loopback0 – Douglas Gourlay's Blog Data Centers, Virtualization, and Cloud Computing

The Year of 10Gb Ethernet?

When will 10Gb become 'default' on servers and data center hosts?

Since 2002 I have been attending trade-shows like Interop and answering the same question, "Is this the year of 10Gb Ethernet?"

I never really understood the question, but after perennially trying to answer, and usually being inaccurate to what that individual questioner wanted to hear I started thinking about this a lot more, at least so I could have a reasonably rehearsed answer.  Running into Bruce Tolley from SolarFlare today, who hosted the panel I was on for at least the last couple of years, reminded me that we are still not quite in 'the year of 10GbE' so I started wondering why, and when it would be.

What's it going to take?  What will it take for 10GbE to take off on the same growth pace that GbE had?

In no particular order, I will start with density.  When I was first asked this in 2002 there was a single-port 10Gb Ethernet module shipping in the Catalyst 6500 and a roadmap to deliver a four-port module the next year.  Density matters, and how dense a network device is can allow different use-cases for the nascent technology.  I remember making a PowerPoint slide once that said essentially the following:

Single Port- nice demo
Two Ports- very expensive, nice for metro connections and dark-fiber long hauls
Four Ports- useful in Distribution to Core switch interconnection
Eight Ports- starts aggregating access layer switches
Sixteen Ports- a good density of campus wiring closet aggregation
Twenty-Four Ports- high performance server aggregation
Forty-Eight Ports- server aggregation

Not surprisingly as silicon densities increase, port densities tend to increase and costs can come down through volumes, value engineering, and better utilization of common equipment.

Orthogonal Upgrades.  Not all budget cycles align.  Sometimes you get budget for a server refresh, other times you have have budget for physical plant upgrades, and on others you have some room for new network equipment.  Most 10Gb implementations today require 'symmetric upgrades' i.e. you have to uplift the cabling, the servers, and the switches all at the same time in a coordinated manner.  This is usually a pretty big inhibitor.  You need to be able to add switches that support past, present, and future speeds, on top of existing cable plant, then allow the servers to upgrade when their budget cycle allows and computational/transport demands require it.

Price.  Yes, it always matters and many times it all comes down to price.  Let's set a magic mark of 10% and 3-4x depending on how you look at this.  It strikes me that when the price of a complete system (servers, NICs, cabling, switching) at speed X*10 is only 10% more than speed X the market moves pretty quickly to connecting the hosts at X*10.  The other way to look at it is that when X*10 is 3-4x the price of X in a myopic pricing arrangement the technology also tends to rapidly shift.

In many cases across most major vendors you can get two or all three of the above today.  The outlier for data center host conversion and 'cross-over' from GbE to 10GbE is 'LAN on Main Board'.  When all other factors align this tends to come out, and then the world changes and in the span of 12-24 months the new transport closes the gap and then starts passing its predecessor.

When do I think we will see the transition and cross-over (caveat- in the data center.  I see no current trajectory towards demands for 10GbE to the desktop or phone in a campus.  For all that video is the vaunted next wave of upgrades I can put several HDTV signals over 100Mb pretty effectively and it won't make a dent in GbE much less 10Gb to the desktop)  I saw some data recently showing 100% growth in 10Gb to the server quarter over quarter, although largely driven by HP Blades with their virtual connect and flex-10 offerings.  Given this trajectory and the desire for the server manufacturers to have a compelling value proposition against HP and Cisco's UCS I would imagine they are rapidly moving to offering 100/1G/10G LOM offers on their bladed and stand-alone server systems over the next twelve months.  Then we finally, after seven years of speculation, can have a 'Year of 10Gb.'

Author: Douglas Gourlay

high-tech executive with interests in networking, virtualization, cloud computing, and IT/Tech government policy. VP of Marketing at Arista Networks - this blog reflects Doug's personal views and opinions and not necessarily those of Arista Networks.