loopback0 – Douglas Gourlay's Blog Data Centers, Virtualization, and Cloud Computing


24
Oct/09
6

Cautious Optimism, Irrational Exuberance, Full-Circle Come-a-bouts, and Economic Recovery

Everything seems to come full circle in IT...

Everything seems to come full circle in IT...

Cautious optimism is a term I have been having many discussions lately with friends and analysts about - whether we are seeing true economic recovery or a bit of a 'W' and whether to make serious investments in planned growth or not.  Candidly, in IT we have compressed capital spending for a while, so it could just be a bit of elasticity - although one major thing strikes me as different.

In the current world order many of the IT investments seem to be directly proportional to short-mid term ROI, sure everyone wants to build for 5-10 years, but they also want to see real business results, right now.

Mostly this means that new project types are getting priority and IT is finding creative and innovative ways of delivering near-term business value without, hopefully, taking their eye off the architectural ball.  Ideally we can do both- deliver short-term value creation, while building towards a longer-term vision that enables IT to reinvent itself and infrastructure to transcend generational shifts. Sadly this is not always the case, some companies and people seem to want to either over-rotate on short-term. Sadder, others refuse to admit the world is changing.  Even worse are those who keep their head in the sand and cannot move at all.  Denying change happens is dooming almost any business to failure, embracing a fickle trend too quickly can be just as painful, and relying on past formulas from previous successes doesn't always work.

You may wonder where I am going with this.  Over the past thirteen years I have seen a lot of things change and come full circle- Cut-Through Switching, Lossless L2 Networks, Ring Topologies, Hosting/Cloud/Insource/Outsource.  Universal truth - things change and open and experienced minds that can capture this change tend to prevail.

Architectures have to change with the trend, the old way of doing things is not always the best- althought there are always viable lessons to be learned and due respect should be paid to past success.

Looking at networking, especially in the data center there are a lot of architectural changes in play.  Obviously, the changes being driven to effect convergence between Ethernet and FibreChannel is a big one.  The other is the collapsing of layers and efforts to simplify the topologies while increasing the scale of operations - I think in my next post or two I will have to explore these some more, what are your thoughts on other architectural changes in the data center network?

dg

1
Aug/09
12

America’s Data Center Top 40 for QoS Implementations

a rainbow of fruit flavors for the network

a rainbow of fruit flavors for the network

Do you really use network quality of service within the glass house data center?  This is a question I have been pondering pretty much all day- I get that on the costly WAN links we almost all use it.  I also completely acknowledge that most engineers plan to use some of the QoS features within the data center, but what do we really use?  Which features are necessary and which are just... well...  extraneous?

Here's my thought-   I think there are some  QoS 'hit songs' implemented in the data center, and some that just are not that appropriate for this iMix.

Bad Boy Bad Boy, whatcha gonna queue....
No matter how many security products we surround ourselves with we all have the 'time out queue' for traffic that just isn't, well, good.  It's for those pesky control plane DoS attacks, the misbehaving broadcast storm NIC, or that one application that just won't shut up.

Voices Carry
For some reason I have the 80's Til Tuesday song in my head by the same name- but most engineers I have talked with seem to set up, by default, a strict priority queue for voice traffic, then never,ever touch it again because it just works.

Marky Mark and the Funky Bunch

We all use markers, especially at the edge where we can have maximum visibility to what comes from a single host, and mark it up appropriately so that we don't have to continuously re-inspect and re-apply policy everywhere.  Tag at the edge, queue in the core, shape on the WAN or point of massive congestion/cost.

(Don't) Drop it like it's Hot

No matter what religious SCSI encapsulation you love- FibreChannel, FibreChannel over Ethernet, iSCSI or even file based alternatives such as NFS or CIFS storage just performs better when is is not dropped.  As much as I love technologies like Aspera for moving large files from A-to-B, until it is integrated in my host stack (which would monumentally rock for copying my iTunes library to S3 btw, hint...) the world will be dependent on TCP-guaranteed or PFC/BCN guaranteed storage delivery for a while - so we need a "lossless don't-drop-me-please" queue.

Shape ya Tailfeather
As we hit congestion and contention for bandwidth in a well designed data center this happens at the cost-center link, i.e. the edge router: this is where I apply large buffers, shapers, and policers to ensure that traffic can en-queue long enough to get onto the link, we can also adhere to Peter Lothberg's long-time rule of providing enough buffer to handle a round-trip-response of around the world or ~300ms.  Shaping the previously marked-up traffic onto the link while trying to get all the right traffic on the link is the job of the core routers, usually not fixed switches.

That's it for now, any songs I should also think about including into the Data Center QoS Playlist?

dg

28
Jul/09
3

Tagging Redux – Curse of the Bifurcated Standards Bodies

Moooo-re Siloes Please!

Moooo-re Siloes Please!

I have received a few private comments that I was unduly critical of tagging mechanisms in my recent post about the VN-Tag/VN-Link debate between Cisco and HP.  I don't have a problem with tagging mechanisms, in fact I am a fan of them - however, I have an issue with too many tagging mechanisms.  There ends up being a tag for about everything - we have Security Group Tags, MPLS Tags, VLAN Tags, QoS Tags at L2 and separate ones at L3, we also now have tags that identify the Virtual Machine or the NIC on the Virtual Machine so we can create new NICs in software.

Let's just own up to it for a second that there are way too many tags; however, the problem is not always the industry and companies that want to create a new tag. Now before I go offering a solution that is highly likely to not be flame-retardant let me get on my proverbial soapbox for a second, so bear with me...

I think a root cause problem is that there is a schism in the relevant standards bodies.  I have to go to one standards group for Ethernet (IEEE), another to work at the IP layers (IETF), yet a third if I want to carry storage traffic (ANSI T.11) - and this is the tip of the iceberg.  It is hard enough to shepherd a great idea through the bureaucratic morass that are created anytime you put that many strongly opinionated, diametrically opposed, and discretely compensated individuals on a board/council/committee and try to get them working together but now with things getting 'blurry' between the once rigid lines between siloed standards bodies it is near impossible.

So while I often catch the brunt of 'why isn't this done in the standards bodies' question and have fielded it for years I don't think we have done a fair job of analyzing how efficient the standards bodies are and whether they should look at reorganizing around how their customers use the technologies they ratify.  The world is changing from 20 years ago - are the standards bodies changing with it?  Are they part of the legacy of siloed information technology that needs to change with the new world order where lines are crossed, borders are blurred, and service is more important than boxes?

As far as how to solve this embedded multi-vectored tagging problem I don't think there is one 'right' answer.  But one thing I always with was available was a single extensible tagging structure, that could be layered/embedded, that could be re-purposed for different functions and the first few bits (say 8 or so) delimited what 'kind' of tag it is (QoS, App Descriptor, Segmentation, Forwarding, Cloud Storage Access Rights, etc), and the next 16-24 bits were used to delimit something relevant within that kind of tag.  Done in an extensible fashion we could keep the extended header short of 128-256 bits which would mean it could pretty easily be parsed with one of the more advanced stream processing methods on the market.

One standard to rule them all, one to bind them...

One standard to rule them all, one to bind them...

Yes, it is sort of 'too simple' and is probable a bit like Dark Lord Sauron forging the 'One Ring to Rule them All, One Ring to Bind Them' (and the fact that I am doing that from memory should be a strong signal that I missed ComicCon)  but a tagging construct that 'crossed party lines' between IEEE, IETF, ANSI, ISO, W3C, etc could be quite interesting.  It's a shame there is not standards body to run it through....

dg