Re: LLQ- help (with elephants)

ALL,

I spoke with my friend Wendell and he still had notes on my test. No
configssorry:(

The one thing I forgot was that the link was 128K that I was testing on at the
time. Very similar to the the 128K test that Carlos accidentally forced.

Therefore, when I placed 4 calls that tootled approx 112K of the Pipe. All
worked well. The 5th call went through fine cause it was not enough to congest
the link. The 6th call had to have had congested the link which kicked in the
BURST parameter set in the Priority Queue and traffic dropped traffic out of
the VOICE CLASS.

WHen I used Video, I used enough Video traffic to barely congest the link but
the BURST was much greater to cause more drops.

I also have in my notes that the POLICE action worked very similar to a Single
Rate 2 Color Policer. '

The conclusion was that the Priority Class Burst Parameter will drop traffic
only when the link is congested, even if it's by a little depending on the
traffic.

This is how the BURST parameter works.

I believe Mr Neiberger said this exact same thing in a previous post. (Good
Job Neiberger).

Marko was correct about the congestion but I never argued about the CBWFQ
portion of this. Just the Priority Queue behavior with the BURST parameter.
It's just that the BURST parameter ONLY kicks in when there is congestion on
the physical interface. I believe Carlos also agrees with this but I will let
him speak for himself. Not sure if Marko agrees since he admitted he did not
know how the BURST worked. (Not that there is anything wrong with that) :-)

Paul

Paul Negron
CCIE# 14856
negron.paul_at_gmail.com
303-725-8162

On Dec 19, 2012, at 7:28 AM, Carlos G Mendioroz <tron_at_huapi.ba.ar> wrote:

> Oh my.
> I found why the policer was dropping in FR: the serial was clocked at 128K
on the other side :(
>
> (This was cloaked by SNMP showing TX at 200k for as long as I tested, w/o
the policy... this is indeed a heads up on SNMP metering for me).
>
> Sorry about that.
>
> On the other hand, I would like to share an example on one of the reasons we
need bursting:
>
> Assume we have someone that has to send some elephants through a path that
crosses a river with a bridge. The brisge has a guard that is told not to let
more than 3 elephants per minute through, and just kill any violating
elephants that may arrive at bridge entrance.
>
> Knowing the bridge policy, the herd owner ask you to be carefull not to send
more than 3 epm. (elephants per minute).
>
> So you start your clock and send at times:
> 0'10
> 0'30
> 0'50
> 1'00
> 1'15
> 1'30
> 2'12
> 2'40
> 2'55
>
> Right ? No minute had more than 3. All elephants walk preciselly at speed of
light :) and arrive w/o jitter at the bridge, but the gatekeeper's watch is
not synchronized, so let's add 15 seconds and see:
>
> 0'25
> 0'45
> 1'05
> 1'15
> 1'30
> *1'45* Kill it, 4th in a minute!
> 2'27
> 2'55
> 3'10
>
>
> What went wrong ? Well, metering. That's one of the reasons we need
bursting.
>
> Hope nobody gets under one elephant foot :)
>
> -Carlos
> P.S.
> No animal was hurt in this test!
>
>
>
>
> Carlos G Mendioroz @ 19/12/2012 06:50 -0300 dixit:
>> Paul,
>> I would kindly ask you to apply some policing on words like CRAP. Those
>> raise the emotions w/o adding any technical data...
>>
>> Now, it would be great if we agree on what we say. Congestion is a not
>> very precise word. It might be that your are dropping packets, that you
>> have some packets backlog (queue), or that your processor is running at
>> top speed. In cisco's queue management, it's usually used to denote that
>> one interface's TX ring is full, so the router can not just FIFO packets
>> at interrupt time and has to "engage" software queueing.
>>
>> Marko, I and a lot of people at cisco (e.g. Tim Szigeti) kind of konw
>> that CBWFQ or LLQ vanish when not congested, for good reason.
>> Well, not if you have "police" commands. Those work always.
>>
>> So one or the arguments (main one ?) is that the "priority xx yy" is not
>> going to produce drops unless somehoy you force LLQ to engage.
>>
>> It seems that there is some other way that cisco implements that also
>> takes the "priority xx yy" into consideration that I stumbled on, while
>> trying to prove this with a test.
>>
>> Now, real phone stream is not golden test data. Cisco has special images
>> to generate test data and if you have a programmer side, you can do that
>> too, or use one of the many tools to do CBR, or some VBR.
>> As long as you send below your output TX rate, you should not congest,
>> and testing with low values of priority should be enough to see test
>> results.
>>
>> BTW, the reason for the implicit policer is fairness. The priority queue
>> is served always first, and there is no scheduler check for that. That's
>> why you need to prevent much traffic from entering the queue or else it
>> will starve others. "Queue" here is a control mechanism, it cisco says
>> it has not packet count limit.
>>
>> -Carlos
>>
>>
>> Paul Negron @ 19/12/2012 01:30 -0300 dixit:
>>> Guys,
>>>
>>> The reason why we are going through this is because of a statement I
>>> said a while back in the thread that Marko is disagreeing with.
>>>
>>> I tested this crap with REAL Voice packets from a real phone from a
>>> multiple real FXS ports.
>>> Not your crappy UDP tests that may not set up real BURST behavior like
>>> real Video.
>>>
>>> I remember trying to prove that the Priority Queue would NOT drop
>>> packets if I sent more Call traffic through it in the event that there
>>> was NO congestion. Very similar to what Marko is saying. To my surprise
>>> I saw the Priority Queue actually drop traffic when I had placed 2 more
>>> calls than the "Priority" command was configured for.
>>>
>>> Marko said I had done an" Invalid test" which I felt was very Cocky on
>>> his part especially when he could not recall how the Burst command
>>> actually worked. It's kinda like saying someone else is wrong before you
>>> know ALL the facets involved. He could be right but I guarantee he is
>>> NOT performing the same test with REAL traffic from REAL devices. He is
>>> NOT listening he is just trying to win an argument. WHATEVER!!
>>>
>>> The reason why the Priority Queue introduced the policer was to
>>> protect the Priority Class from using too much traffic in the event that
>>> other classes were not using their share. Now this was when QOS was
>>> first introduced to the code and I was testing on Frame-Relay links so
>>> it COULD have been flawed side it looked like Carlos's test showed but I
>>> am NOT completely sure since I am getting different results from
>>> different people now (like Carlos and Marko). I remember what I saw
>>> cause I repeated the same test with REAL Video Traffic and got similar
>>> results. The priority Queue would drop traffic prior to congesting the
>>> link due to very high burst in a very small period of time. The behavior
>>> was almost like CAR not Class Based Policing.
>>>
>>> I may be getting old and soft in the brain but I remember that test. It
>>> was the Original QOS lab that SKYLINE would run for QOS back in 2002 I
>>> think. One 2600 with FXS ports connected to another 2600 with FXS ports.
>>> We would set up an HTTP server and create Video Flows from Cisco IPTV to
>>> congest the link.
>>>
>>> I put in a call to Wendell Odem to refresh my memory since we set up
>>> that lab together.
>>>
>>> I almost hope he agrees with Marko cause it would make more sense that
>>> way since it would behave most like a pure Queuing mechanism. John
>>> Neiberger also added very accurate insight that agreed with Marko but I
>>> KNOW he is listening to the whole conversation with an unbiased eye.
>>> (thanks man). This discussion is waking up my brain on this subject and
>>> I actually enjoy the dialogue.
>>>
>>>
>>> Paul Negron
>>> CCIE# 14856
>>> negron.paul_at_gmail.com <mailto:negron.paul_at_gmail.com>
>>>
>>>
>>>
>>> On Dec 18, 2012, at 6:46 PM, Marko Milivojevic <markom_at_ipexpert.com
>>> <mailto:markom_at_ipexpert.com>> wrote:
>>>
>>>> That's exactly my point. We don't (you talked about single-rate
>>>> policer, not me, btw.). This is what the description of the command
>>>> says:
>>>>
>>>> "(Optional) Burst size in bytes. The burst size configures the network
>>>> to accommodate temporary bursts of traffic. The default burst value,
>>>> which is computed as 200 milliseconds of traffic at the configured
>>>> bandwidth rate, is used when theburst argument is not specified. The
>>>> range of the burst is from 32 to 2000000 bytes."
>>>>
>>>> Further down, there's an example:
>>>>
>>>> "Examples
>>>>
>>>> The following example shows how to configure PQ with a guaranteed
>>>> bandwidth of 50 kbps and a one-time allowable burst size of 60 bytes
>>>> for the policy map named policy1:
>>>>
>>>> Router(config)# policy-map policy1
>>>> Router(config-pmap)# class voice
>>>> Router(config-pmap-c)# priority 50 60"
>>>>
>>>> Now, look me in the eye and say this sounds like Bc to you? To me,
>>>> this sounds more like a Be (which implies two rate policer. This is
>>>> not the three color policer, as there seems to be no option to specify
>>>> three actions, hence my comment that it sounds like be, but not even
>>>> that, as it lacks the functionality I'd expect to see...
>>>>
>>>> Now, there are two possibilites:
>>>>
>>>> 1) Cisco documentation is wrong and this indeed specifies Bc for a
>>>> single rate, two color policer. It would actually make sense, as it
>>>> allows the possibility to adjust metering on transmission side to the
>>>> metering on the receiving side (other side of the link), should they
>>>> be mismatched. This is not an unlikely scenario and it's what you're
>>>> saying.
>>>>
>>>> 2) This is indeed a temporary "excess burst", but with the implied
>>>> "transmit" action, meaning that it can be used to create a two rate,
>>>> two color policer. According to the documentation, this is what it is.
>>>>
>>>>
>>>> --
>>>> Marko Milivojevic - CCIE #18427 (SP R&S)
>>>> Senior CCIE Instructor - IPexpert
>>>>
>>>> On Tue, Dec 18, 2012 at 3:39 PM, Narbik Kocharians <narbikk_at_gmail.com
>>>> <mailto:narbikk_at_gmail.com>> wrote:
>>>>> Since when do we have a Be in single rate two color policer?
>>>>>
>>>>>
>>>>> On Tue, Dec 18, 2012 at 3:33 PM, Marko Milivojevic
>>>>> <markom_at_ipexpert.com <mailto:markom_at_ipexpert.com>>
>>>>> wrote:
>>>>>>
>>>>>> Well, let's talk about the burst parameter, because I believe we
>>>>>> reached a conclusion it does not work like Bc (i.e. to determine the
>>>>>> Tc). Rather it works more closely to Be, but not even that... :-)
>>>>>>
>>>>>> I'm not confused about it - I just can't seem to find the information
>>>>>> what it does. I can *believe* what it does, but that doesn't explain
>>>>>> it to me ;-)
>>>>>>
>>>>>> --
>>>>>> Marko Milivojevic - CCIE #18427 (SP R&S)
>>>>>> Senior CCIE Instructor - IPexpert
>>>>>>
>>>>>> On Tue, Dec 18, 2012 at 3:30 PM, Narbik Kocharians
>>>>>> <narbikk_at_gmail.com <mailto:narbikk_at_gmail.com>>
>>>>>> wrote:
>>>>>>> I have not, why would you say that? I am not going to participate
>>>>>>> in you
>>>>>>> said he said stuff. The burst in LLQ works the way I explained it,
>>>>>>> and I
>>>>>>> still say that, when did I say the burst does not work the way i
>>>>>>> mentioned?
>>>>>>>
>>>>>>> The burst in LLQ works just like the Bc in a single rate 2 color
>>>>>>> policer.
>>>>>>> NOW...whether it's behavior is that way during congestion or not it
>>>>>>> needs to
>>>>>>> be tested, I believe it works that way during congestion, but i do
>>>>>>> not
>>>>>>> have
>>>>>>> the testing gear to provide results, the tests shown in some of the
>>>>>>> examples
>>>>>>> are ridiculous to test the complete package, things like burst and
>>>>>>> playing
>>>>>>> with the burst value.
>>>>>>>
>>>>>>> It was you that was confused about the burst parameter and not me. Go
>>>>>>> back
>>>>>>> and read what you wrote.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Dec 18, 2012 at 3:15 PM, Marko Milivojevic
>>>>>>> <markom_at_ipexpert.com <mailto:markom_at_ipexpert.com>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Thank you for agreeing with what I was saying all along (even though
>>>>>>>> it kind-of contradicts your previous burst statement) ;-)
>>>>>>>>
>>>>>>>> Now, testing LLQ is tricky and making calls is not exactly the best
>>>>>>>> way to test it, should you intend to use it for something other than
>>>>>>>> voice. Any throughput, queueing, performance testing is best left to
>>>>>>>> the professional tools designed for that purpose. Human ears are
>>>>>>>> not a
>>>>>>>> good tool :-).
>>>>>>>>
>>>>>>>> For anyone with (serious) production concerns about the LLQ and a
>>>>>>>> (serious) testing budget, I can only highly recommend Spirent
>>>>>>>> SmartBits or TestCenter products. They are *unparalleled* to what
>>>>>>>> you'll be able to learn about your routers and switches and their
>>>>>>>> (lack of) performance... :-)
>>>>>>>>
>>>>>>>> --
>>>>>>>> Marko Milivojevic - CCIE #18427 (SP R&S)
>>>>>>>> Senior CCIE Instructor - IPexpert
>>>>>>>>
>>>>>>>> On Tue, Dec 18, 2012 at 3:09 PM, Narbik Kocharians
>>>>>>>> <narbikk_at_gmail.com <mailto:narbikk_at_gmail.com>>
>>>>>>>> wrote:
>>>>>>>>> Totally agree with what was stated in that link.
>>>>>>>>> We have to remember that PQ, CBWFQ, are all congestion management
>>>>>>>>> tool.
>>>>>>>>> but
>>>>>>>>> the best way to test LLQ is to actually make calls and test the
>>>>>>>>> quality.
>>>>>>>>>
>>>>>>>>> On Tue, Dec 18, 2012 at 2:40 PM, Joe Sanchez <marco207p_at_gmail.com
>>>>>>>>> <mailto:marco207p_at_gmail.com>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Here is a great view:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
http://www.cisco.com/en/US/docs/ios/12_0t/12_0t7/feature/guide/pqcbwfq.html#w
p5329
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> JS
>>>>>>>>>>
>>>>>>>>>> On Tue, Dec 18, 2012 at 3:28 PM, Marko Milivojevic
>>>>>>>>>> <markom_at_ipexpert.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> For fun, I added another source of the traffic and lowered
>>>>>>>>>>> bandwidth
>>>>>>>>>>> on the interface to 768000 (in an attempt to actually create a
>>>>>>>>>>> congestion). This is by no means an exhaustive test of LLQ and
>>>>>>>>>>> the
>>>>>>>>>>> quality, but it shows a conditional policer in action.
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------8<------------------------------
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------8<------------------------------
>>>>>>>>>>>
>>>>>>>>>>> If you look closely at the pattern of drops above, you will
>>>>>>>>>>> observe
>>>>>>>>>>> the regular intervals at which they appear. This *can* be an
>>>>>>>>>>> indicator
>>>>>>>>>>> of active traffic conditioner. Let's see what our policy says.
>>>>>>>>>>> Keep
>>>>>>>>>>> in
>>>>>>>>>>> mind that the other traffic generator is much more aggressive
>>>>>>>>>>> than
>>>>>>>>>>> the
>>>>>>>>>>> first (timeout 0).
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------8<------------------------------
>>>>>>>>>>> R2#show policy-map interface Serial0/2/0 output class PRIORITY
>>>>>>>>>>>
>>>>>>>>>>> Serial0/2/0
>>>>>>>>>>>
>>>>>>>>>>> Service-policy output: LLQ
>>>>>>>>>>>
>>>>>>>>>>> queue stats for all priority classes:
>>>>>>>>>>>
>>>>>>>>>>> queue limit 64 packets
>>>>>>>>>>> (queue depth/total drops/no-buffer drops) 0/0/0
>>>>>>>>>>> (pkts output/bytes output) 158281/238054624
>>>>>>>>>>>
>>>>>>>>>>> Class-map: PRIORITY (match-all)
>>>>>>>>>>> 344671 packets, 518385060 bytes
>>>>>>>>>>> 30 second offered rate 10173000 bps, drop rate 9649000 bps
>>>>>>>>>>> Match: protocol icmp
>>>>>>>>>>> Priority: 100 kbps, burst bytes 2500, b/w exceed drops:
>>>>>>>>>>> 186359
>>>>>>>>>>> ------------------------------8<------------------------------
>>>>>>>>>>>
>>>>>>>>>>> The problem is that this test is flawed when it comes to
>>>>>>>>>>> reproducing
>>>>>>>>>>> the real congestion, as here we also have to take into account
>>>>>>>>>>> that
>>>>>>>>>>> the traffic is arriving from the same input interface, so we have
>>>>>>>>>>> 1:1
>>>>>>>>>>> input:output mapping (again, something I mentioned very early in
>>>>>>>>>>> this
>>>>>>>>>>> thread as an important factor).
>>>>>>>>>>>
>>>>>>>>>>> I could spend even more time labbing it up for you, but I suppose
>>>>>>>>>>> I've
>>>>>>>>>>> done enough train-the-trainer for one day :-).
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Marko Milivojevic - CCIE #18427 (SP R&S)
>>>>>>>>>>> Senior CCIE Instructor - IPexpert
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Dec 18, 2012 at 1:16 PM, Marko Milivojevic
>>>>>>>>>>> <markom_at_ipexpert.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> On Tue, Dec 18, 2012 at 1:14 PM, Narbik Kocharians
>>>>>>>>>>>> <narbikk_at_gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> How did you prove that LLQ kicks in during congestion? How did
>>>>>>>>>>>>> you
>>>>>>>>>>>>> measure
>>>>>>>>>>>>> the quality of these packets?
>>>>>>>>>>>>
>>>>>>>>>>>> Well, that's not at all what I was testing, as the test was
>>>>>>>>>>>> built
>>>>>>>>>>>> specifically to avoid congestion to show how there's no policing
>>>>>>>>>>>> when
>>>>>>>>>>>> congestion does not occur. Apples and Oranges.
>>>>>>>>>>>>
>>>>>>>>>>>> Testing the quality of these packets would be difficult (not
>>>>>>>>>>>> impossible) to measure using IOS-only and Voice SLA probes as
>>>>>>>>>>>> traffic
>>>>>>>>>>>> generators, but as I said... this was not the point here.
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Marko Milivojevic - CCIE #18427 (SP R&S)
>>>>>>>>>>>> Senior CCIE Instructor - IPexpert
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Blogs and organic groups at http://www.ccie.net
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
Received on Wed Dec 19 2012 - 18:43:13 ART