RE: Traffic extremely slow in one direction

From: Rik Guyler (rik@guyler.net)
Date: Tue Feb 12 2008 - 19:20:32 ARST


All too true Steve, especially working in an Enterprise shop where you're
always understaffed and overworked. I adopted a policy of adjusting MSS to
1400 and clearing the DF-bit some time ago when PMTUD became totally
worthless to me for my 50+ VPN remotes. After that, performance may not
have been optimal but my phone was much less active. ;-)

Rik

-----Original Message-----
From: nobody@groupstudy.com [mailto:nobody@groupstudy.com] On Behalf Of
Steve
Sent: Tuesday, February 12, 2008 3:15 AM
To: Joseph Brunner
Cc: Luan Nguyen; Paul Cosgrove; Jian Gu; ccie forum
Subject: Re: Traffic extremely slow in one direction

The theory is good but more and more I am finding security products are
blocking ICMP wholesale.
I have lost count on the number of times I have had to clear DF bit simply
because end to end hosts have various AV/FW products installed that don't
allow for path discovery.
Many system admins have been taught from birth "ICMP is evil......block it
at all costs..." so even if you identify the issue, you often have a hard
time convincing them to adjust security policies to allow the ICMP through.

Yes - this does cause fragments....is this the right way to run good
networks - no.......by clearing DF bit, does the throughput increase so
users are happy? - yes.

Sigh...such is life.

Steve

On 12 Feb 2008, at 05:35, Joseph Brunner wrote:

> Why would you clear the df bit?
>
> You DON'T WANT FRAGMENTS!!!! perhaps you should put the ip mtu/tcp mss
> commands on the LAN FACING interface...
>
> IMHO the 2801 is the WORST of the bunch... I tried using it as a large
> scale dmvpn hub... LOL the 2811 actually held up quite nicely, all the
> way to
> 20Mbps+ what killed it was the sheer number of dmvpn tunnels...
>
> -Joe
>
> -----Original Message-----
> From: nobody@groupstudy.com [mailto:nobody@groupstudy.com] On Behalf
> Of Luan Nguyen
> Sent: Monday, February 11, 2008 9:20 PM
> To: Paul Cosgrove
> Cc: Jian Gu; ccie forum
> Subject: Re: Traffic extremely slow in one direction
>
> For me, setting ip mtu, and ip tcp adjust-mss are never enough. I
> have to clear the df bit as well.
> access-list 111 permit tcp any any
> route-map clear-df-bit permit 10
> match ip address 111
> set ip df 0
> int WAN
> ip policy route-map clear-df-bit
>
> On a side note, the 2811 is the worst performer of the 2800 series
> family of ISR regarding GRE/IPSEC. We had to drop it even with the
> AIM-VPN/
> SSL-2
> card. And don't take performance number for its face value...it's
> unidirectional and best case scenario. You should take that number
> divide by 4 to get real world value :)
>
> -lmn
>
> On Feb 11, 2008 3:30 PM, Paul Cosgrove <paul.cosgrove@heanet.ie>
> wrote:
>
>> Jian Gu wrote:
>>> hi, all,
>>>
>>> I have a real world problem that I am scratching my head to find the
>> root
>>> cause. We have a local data center at site A and a remote backup
>>> server
>> at
>>> site B, site A and site B has its own service provider, the
>>> connection bandwidth is well above 50M in either site, traffic will
>>> traverse a GRE/IPsec tunnel which is configured between two 2800
>>> routers, both
>> routers
>>> have hardware accelerated IPSec encryption/decrytion module
>>> installed.
>>>
>>> It is interesting that traffic from site A to site B is extremely
>>> slow, performance is only around 10s packets/second, but FTP
>>> transferring data from site B to site A is reasonably fast, which is
>>> around 8Mb/sec.
>>> I am
>>> pretty sure service provider of site A is not rate limiting incoming
>> traffic
>>> (even it does, noway they will rate limit to 10s packets/sec). What
>> could be
>>> the root cause of such poor performance, any server experts here can
>> give me
>>> a clue?
>>>
>>> Thanks,
>>> Jian
>>>
>>> ____________________________________________________________________
>>> ___ Subscription information may be found at:
>>> http://www.groupstudy.com/list/CCIELab.html
>>>
>>
>> Hi Jian Gu,
>>
>> Perhaps you do not notice delay or drops with FTP transfer because
>> you do not see the individual packets being dropped/retransmitted?
>> With pings this will be more readily apparent. How are you testing
>> the link?
>> Which direction does most of the traffic flow in?
>>
>> The number of packets per second is very important for encryption,
>> more so than the throughput in terms of Kbps. Sending lots of small
>> packets will normally have more of an impact than sending a few large
>> ones (such as FTP).
>>
>> Other than pps, also watch for fragmentation. Check what GRE IP MTU
>> you are using, as unless your GRE IP MTU allows for the encryption
>> header being added later your GRE packets will be fragmented.
>>
>> GRE adds 24 bytes, IPSEC is variable because of the padding but (I
>> think) is about 80 bytes.
>>
>> Also keep in mind that the router will also have to perform
>> fragmentation if the incoming IP packet is larger than the GRE IP
>> MTU, and setting the TCP MSS lower using "ip tcp adjust-mss" can help
>> with that (but only for TCP). Path MTU discovery on end
>> hosts/servers is also a good way to reduce fragmentation, and thereby
>> reduce load on the device.
>>
>> According to the folowing link the 2811 is capable of 55 Mbps with
>> AES or 3DES, but you often have to quite cautious when shown
>> encryption stats without pps or packet size.
>>
>>
>>
>
http://www.cisco.com/web/partners/downloads/765/tools/quickreference/vpn_per
> formance_eng.pdf
>>
>>
>> Paul.
>> --
>> Paul Cosgrove
>> HEAnet Limited, Ireland's Education and Research Network
>> 1st Floor, 5 George's Dock, IFSC, Dublin 1
>> Registered in Ireland, no 275301
>> tel: +353-1-660 9040 fax: +353-1-660 3666
>> web: http://www.heanet.ie/
>>
>> _______________________________________________________________________
>> Subscription information may be found at:
>> http://www.groupstudy.com/list/CCIELab.html
>
> _______________________________________________________________________
> Subscription information may be found at:
> http://www.groupstudy.com/list/CCIELab.html
>
> _______________________________________________________________________
> Subscription information may be found at:
> http://www.groupstudy.com/list/CCIELab.html



This archive was generated by hypermail 2.1.4 : Sat Mar 01 2008 - 16:54:48 ARST