Comments on: America’s Data Center Top 40 for QoS Implementations http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/ Data Centers, Virtualization, and Cloud Computing Sun, 02 May 2010 01:04:25 +0000 hourly 1 http://wordpress.org/?v=3.0 By: Douglas Gourlay http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-101 Douglas Gourlay Tue, 04 Aug 2009 05:46:05 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-101 think I edited it up right now :) check me on this one... think I edited it up right now :) check me on this one…

]]> By: Jim Fenton http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-100 Jim Fenton Tue, 04 Aug 2009 05:43:55 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-100 Still missing an "l" at the end of the URL: index.html, not index.htm Still missing an “l” at the end of the URL: index.html, not index.htm

]]> By: Douglas Gourlay http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-85 Douglas Gourlay Mon, 03 Aug 2009 06:04:50 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-85 fixed it I think - embedded the link into 'Davenport as well' fixed it I think – embedded the link into ‘Davenport as well’

]]> By: Fred Baker http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-83 Fred Baker Mon, 03 Aug 2009 06:01:31 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-83 It didn't display the attached website :-(. ftp://ftpeng.cisco.com/fred/RTT/index.html It didn’t display the attached website :-( . ftp://ftpeng.cisco.com/fred/RTT/index.html

]]> By: Fred Baker http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-82 Fred Baker Mon, 03 Aug 2009 06:00:46 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-82 I guess my problem is that I think real time applications are fundamentally different than elastic - elastic will adapt while real time traffic just gets messed up. So we agree on the elastic traffic, but I would take some pains with voice, video, and sensor traffic. On WAN links, loss is your friend. As the "<a href="ftp://ftpeng.cisco.com/fred/RTT/index.html" rel="nofollow">Davenport</a>" example in the attached web site shows, a WAN service that tries to prevent loss can wind up with multiple instances of the same packet in queue, with disastrous effects on both goodput and RTT. It would be better to drop a packet and bring TCP back into line than let that happen. In the data center, virtually infinite bandwidth is virtually free, and your customer is gratified by millisecond-scale response time. To me, this has two implications. As you say, let's not accept loss as a parameter - I would rather see a delay-based TCP like CalTech FAST than Reno/NewReno/Cubic to keep RTTs low and goodput high. I would also suggest that TCP's initial burst be set to put the average file in flight in the first RTT. Usually this is set to 2*MSS, which means that a loss on the first transmission results in a timeout (bad in a data center); it has to be at least four to get fast-retransmit into play in the first RTT. But I think you want to use 8K segments (9K datagrams) if you can, and put the first five or six segments in flight to fully utilize that first burst. 6*8192=49K, which should serve the average file nicely. I guess my problem is that I think real time applications are fundamentally different than elastic – elastic will adapt while real time traffic just gets messed up. So we agree on the elastic traffic, but I would take some pains with voice, video, and sensor traffic.

On WAN links, loss is your friend. As the “Davenport” example in the attached web site shows, a WAN service that tries to prevent loss can wind up with multiple instances of the same packet in queue, with disastrous effects on both goodput and RTT. It would be better to drop a packet and bring TCP back into line than let that happen.

In the data center, virtually infinite bandwidth is virtually free, and your customer is gratified by millisecond-scale response time. To me, this has two implications. As you say, let’s not accept loss as a parameter – I would rather see a delay-based TCP like CalTech FAST than Reno/NewReno/Cubic to keep RTTs low and goodput high. I would also suggest that TCP’s initial burst be set to put the average file in flight in the first RTT. Usually this is set to 2*MSS, which means that a loss on the first transmission results in a timeout (bad in a data center); it has to be at least four to get fast-retransmit into play in the first RTT. But I think you want to use 8K segments (9K datagrams) if you can, and put the first five or six segments in flight to fully utilize that first burst. 6*8192=49K, which should serve the average file nicely.

]]> By: Fred Baker http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-81 Fred Baker Mon, 03 Aug 2009 05:50:13 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-81 I think there is value in that. I think there is value in that.

]]> By: Douglas Gourlay http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-80 Douglas Gourlay Mon, 03 Aug 2009 05:31:57 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-80 For one customer I was playing with a switching QoS mode where I would mark packets with a particular DSCP value when link utilization went above 90% and then start a packet capture to a ring-buffer. If the link had drops (i.e. ran out of buffer) then I wanted to inject a packet into the mirrored stream to the ring-buffer and snapshot the buffer so you can get immediate forensics on what traffic type caused the drop. I still think this would be useful some places... For one customer I was playing with a switching QoS mode where I would mark packets with a particular DSCP value when link utilization went above 90% and then start a packet capture to a ring-buffer. If the link had drops (i.e. ran out of buffer) then I wanted to inject a packet into the mirrored stream to the ring-buffer and snapshot the buffer so you can get immediate forensics on what traffic type caused the drop.

I still think this would be useful some places…

]]> By: Jim Fenton http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-79 Jim Fenton Mon, 03 Aug 2009 05:05:12 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-79 I would put even less effort into it than Fred. Watch like a hawk for dropped packets, and then decide what to do about it -- either provide a bigger pipe or selectively apply QoS at the point of congestion. Otherwise QoS is going to make a choke point less visible, even though there may be a performance impact. On WAN links QoS is necessary because it's not economical to provide as much bandwidth as everyone would like, but in the data center you want to discover the problem, not work around it. I would put even less effort into it than Fred. Watch like a hawk for dropped packets, and then decide what to do about it — either provide a bigger pipe or selectively apply QoS at the point of congestion. Otherwise QoS is going to make a choke point less visible, even though there may be a performance impact. On WAN links QoS is necessary because it’s not economical to provide as much bandwidth as everyone would like, but in the data center you want to discover the problem, not work around it.

]]> By: Fred Baker http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-77 Fred Baker Sun, 02 Aug 2009 18:35:37 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-77 You asked me to comment on QoS technology in the data center. Here is my viewpoint. In a data center I would think you would overprovision like mad, because doing so is effectively free. I would recommend that real time traffic be separated from elastic traffic, either literally or by putting real time traffic in a priority queue, but I wouldn't put any more effort into it than that. See http://www.ietf.org/rfc/rfc5127.txt 5127 Aggregation of DiffServ Service Classes. K. Chan, J. Babiarz, F. Baker. February 2008. (Format: TXT=43751 bytes) (Status: INFORMATIONAL) You asked me to comment on QoS technology in the data center. Here is my viewpoint.

In a data center I would think you would overprovision like mad, because doing so is effectively free. I would recommend that real time traffic be separated from elastic traffic, either literally or by putting real time traffic in a priority queue, but I wouldn’t put any more effort into it than that. See

http://www.ietf.org/rfc/rfc5127.txt
5127 Aggregation of DiffServ Service Classes. K. Chan, J. Babiarz, F.
Baker. February 2008. (Format: TXT=43751 bytes) (Status:
INFORMATIONAL)

]]> By: David Kubica http://www.douglasgourlay.com/blog/2009/08/americas-data-center-top-40-for-qos-implementations/comment-page-1/#comment-75 David Kubica Sun, 02 Aug 2009 11:54:45 +0000 http://www.douglasgourlay.com/blog/?p=105#comment-75 The key thought process to remember in the Data Center is that there is no "one size fits all" or in this case no one song that will always top the chart... The overall decision of what traffic is more important will always be debated, however a multitude of solutions would need to be implement at different levels to get the job done right the first time. If you already have the technology paid for then it might be time to bring someone in to help plan which songs should be on your personal data center play list, as well as how to implement them properly for best results. The key thought process to remember in the Data Center is that there is no “one size fits all” or in this case no one song that will always top the chart… The overall decision of what traffic is more important will always be debated, however a multitude of solutions would need to be implement at different levels to get the job done right the first time. If you already have the technology paid for then it might be time to bring someone in to help plan which songs should be on your personal data center play list, as well as how to implement them properly for best results.

]]>