Re: Output Drops on Interface

I beg to disagree about the statement that output drops on a gig interfaces
are not common.
Not only that, but on 10G interfaces as well.

This is one of the main reasons why Cisco has created implemented such a
huge buffers on 6708 linecard, and why 6704 linecard is so problematic. The
normal recommendation when there is burstiness is to upgrade 4-port 6704 to
8-port 6708 and shut down half of the ports (run card in dedicated mode).
I've seen several cases where on 6704 linecards it was impossible to exceed
4Gbps per port, because of buffers.
When replaced with 6708, 10G ports utilization jumped to 7Gbps almost
instanteneously.

Ask any ISP that runs IPTV and are trying to deploy HDTV with HE with
non-real-time-os (e.g. not Scientific Atlanta), and have metro rings built
out of 3400's. There are problems like one 13mbps stream not able to fit
into empty 100Mbps pipe, all because of burstiness.

Just imagine a machine sending Constant Bit Rate (CBR) stream out of it's
1Gig interface. Because it's OS is non real time, that means that some app
that also runs in the background will suddenly require CPU, so machine
pauses its sending of CBR to the network and waits for the other app to
release the cpu. After 5000ms for example the CPU is released, so the
machine can resume sending CBR. But all this time the packets were not being
sent and cummulated in the queue. So now machine sends all of it's queue out
of the interface - it's 1G, remember? So it sends 500ms worth of 13Mbps CBR
out of 1G interface, meaning about 600 packets, most of which are 1500 bytes
each (total of 406KB of data). This burst of 600 packets sent at 1Gbps
(interface always sends data at it's rated speed, it cannot send data at any
other speed) means when it reaches first 100M interface, some packets will
be sent, but most will have to be buffered. And buffers will overflow,
because they are not big enough to hold all the packets. Maximum queue depth
of 3400 is 544 packets, so is 2950 and 2960. So about 20-30 packets will get
dropped. And this happens all the time, and the example I'm here describing
is only for one measily stream. Now imagine numerous such streams, bursts of
data, bursts of voice, VOD, etc etc...

Anywhere there is speed mismatch, we will see drops.
2x 1gig interfaces are input and 1Gig is output? drops
15x100M interfaces input and 1G output? drops
2x10G input and 1x10G output? drops

drops drops drops.

Solution?

QoS QoS QoS

....or per-end-station shaping and using NICs with own real-time OS, not
requiring interrupts to send data.

On Tue, Apr 7, 2009 at 10:41 AM, Jafar T <jafar_at_paris.com> wrote:

> Hi
>
> output drops on a gig interface, is not common,
> did you rule out the other side that this interface talks to ?
> hardocde speed and duplex on both sides,
>
> also keep in mind that on a lan interface you could be maxing out the link
> if you reach close to %40, but you have a gig interface !!
> anyway,
> if both sides are ok,
> hardcoded,
> you also do not think you are maxing out the media,
> the drops are not much considering the over all traffic,
> and it is not critical traffic, such as voice...etc
> and your overall memory is ok, you have enough free memory.
>
> if all above is ok, then do this:
>
> you can increase the hold queue outbound, on the interface,
> command would be: hold-queue < increase it to %20 each time > out
>
>
>
> let me know how that goes please.
>
> cheers
>
> JT
>
>
>
> ----- Original Message -----
> From: "Pavel Bykov"
> To: "Mark Stephanus Chandra"
> Cc: ccieindo_at_yahoogroups.com, "CCIE R/S, Groupstudy" , "Dale Shaw" , "sidd
> ant"
> Subject: Re: Output Drops on Interface
> Date: Tue, 7 Apr 2009 10:24:20 +0200
>
>
> If drops would be more pronounced, you would need to look into reducing
> burstiness of the traffic.
> But as others pointed out - the drops you have there do not look like
> something too interesting is happening.
>
> Remember, when you are looking at the counters, always consider the last
> time when they were erased.
> Do not be afraid to erase counters, but watch out not to clear (reset)
> interface itself.
>
> DO: "clear counters [interface x/x]" --- clears counters
> DO NOT: "clear interface x/x" --- resets the interface
>
> On Tue, Apr 7, 2009 at 8:57 AM, Mark Stephanus Chandra <
> mark.chandra_at_gmail.com> wrote:
>
> > Guys,
> >
> >
> >
> > If I spotted a lot of output drops on an interface. If I manage to
> > configure
> > the input max queue to ba larger than default
> >
> >
> >
> > Does it make any effect ?
> >
> >
> >
> > What should I do if I face this kind of problem
> >
> >
> >
> >
> >
> > GigabitEthernet6/35 is up, line protocol is up (connected)
> >
> > Hardware is Gigabit Ethernet Port, address is 001b.d5d2.8b32 (bia
> > 001b.d5d2.8b32)
> >
> > Description: Server NCBS CBSPRD2 -- 4287
> >
> > MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
> >
> > reliability 255/255, txload 1/255, rxload 1/255
> >
> > Encapsulation ARPA, loopback not set
> >
> > Keepalive set (10 sec)
> >
> > Full-duplex, 1000Mb/s, link type is auto, media type is 10/100/1000-TX
> >
> > input flow-control is on, output flow-control is on
> >
> > ARP type: ARPA, ARP Timeout 04:00:00
> >
> > Last input 00:00:00, output never, output hang never
> >
> > Last clearing of "show interface" counters never
> >
> > Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops:
> > 35272
> >
> > Queueing strategy: fifo
> >
> > Output queue: 0/40 (size/max)
> >
> > 5 minute input rate 889000 bits/sec, 460 packets/sec
> >
> > 5 minute output rate 2889000 bits/sec, 861 packets/sec
> >
> > 8432221024 packets input, 2072360096590 bytes, 0 no buffer
> >
> > Received 49212654 broadcasts (49097827 multicast)
> >
> > 0 runts, 0 giants, 0 throttles
> >
> > 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
> >
> > 0 input packets with dribble condition detected
> >
> > 8586775162 packets output, 2846565821543 bytes, 0 underruns
> >
> > 0 output errors, 0 collisions, 0 interface resets
> >
> > 0 babbles, 0 late collision, 0 deferred
> >
> > 0 lost carrier, 0 no carrier
> >
> > 0 output buffer failures, 0 output buffers swapped out
> >
> >
> >
> >
> >
> > Regards
> >
> > Mark Stephanus Chandra - CCIE#23887
> >
> >
> > Blogs and organic groups at http://www.ccie.net
> >
> > _______________________________________________________________________
> > Subscription information may be found at:
> > http://www.groupstudy.com/list/CCIELab.html
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> --
> Pavel Bykov
> ----------------
> Don't forget to help stopping the braindumps, use of which reduces value of
> your certifications. Sign the petition at http://www.stopbraindumps.com/
>
>
> Blogs and organic groups at http://www.ccie.net
>
> _______________________________________________________________________
> Subscription information may be found at:
> http://www.groupstudy.com/list/CCIELab.html
>
>
> -- Be Yourself @ mail.com!
> Choose From 200+ Email Addresses
> Get a *Free* Account at www.mail.com <http://www.mail.com/Product.aspx>!
>

-- 
Pavel Bykov
----------------
Don't forget to help stopping the braindumps, use of which reduces value of
your certifications. Sign the petition at http://www.stopbraindumps.com/
Blogs and organic groups at http://www.ccie.net