From: Larry Roberts (groupstudy@american-hero.com)
Date: Thu Mar 03 2005 - 18:52:41 GMT-3
Right now this actually appears to be the problem.
Turns out we recently ( month ago ) doubled the amount of imaging that
was being done, and as all the traffic goes to one server to get
initially processed I decided to take a look at the port.
I had over 30K drops in 2 days, but it appears that most appear during
the period of time from 10-4
I'm trying to get a window in replace the server NIC with a GB one.
*fingers crossed that this is the problem*
William Chu wrote:
> Make sure you don't have oversubscribing conditions
> between LAN ports. I have seen issues like that before
> if multiple 100Mbit ports were hammering a single
> 100Mbit port; or a GE port sending bursty traffic to a
> 100Mbit port. Remember, it needs not to be constantly
> oversubscribing, but just enough high burst to fill up
> the output buffer queues on the lower speed port to
> cause frames to drop (and thus causing retransmission
> to occur).
>
> Looks for Out-Discards, output buffer failures on the
> switch port. This document on CCO might help.
>
> http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a008015bfd6.shtml
>
>
> My $0.02.
>
> William
>
> --- Larry Roberts <groupstudy@american-hero.com>
> wrote:
>
>
>>The 6509 is the core of the network. I do have a
>>couple routers that are
>>in the DC subnet, as well as a couple of PIX's also
>>in the DC Subnet.
>>They only provide a couple of subnets each, all of
>>which are either
>>off-site, or part of a dial-in pool. No real way of
>>a routing loop.
>>
>>
>>
>>I don't know if asym. routing could cause the
>>problem with me seeing the
>>problem between hosts in the same VLAN on the same
>>module in the 6509.
>>
>>It may just need a good reboot at this point....
>>
>>Larry
>>
>>
>>Walker, James - Is wrote:
>>
>>>I've seen something similar before when there is
>>
>>some kind of asynmmetical
>>
>>>routing going on.Besides the 6509 MSFC, is there
>>
>>another device(s) routing for
>>
>>>that subnet? Do you only have 1 ways in and 1 way
>>
>>out? Same path?
>>
>>>
>>>
>>>-----Original Message-----
>>>From: nobody@groupstudy.com
>>
>>[mailto:nobody@groupstudy.com]On Behalf Of
>>
>>>wes@stevens.name
>>>Sent: Wednesday, March 02, 2005 7:38 PM
>>>To: Larry Roberts; Groupstudy - Security;
>>
>>Groupstudy R&S
>>
>>>Subject: Re: real world problem
>>>
>>>
>>>Larry, look at the time between the original
>>
>>packet and the
>>
>>>retransmission. Is the app really timing out or is
>>
>>the retransmission
>>
>>>early? If it is timming out put a sniffer on the
>>
>>other side and see if
>>
>>>the client is acking the packet or if it is
>>
>>timming out on the client
>>
>>>side.
>>>
>>>What you are seeing could be just bad apps (early
>>
>>retransmission) or
>>
>>>overloaded clients or servers. I could be that it
>>
>>has nothing to do with
>>
>>>the network (I see this all the time out in our
>>
>>DMZ).
>>
>>>----- Original Message -----
>>>From: "Larry Roberts"
>>>To: "Groupstudy - Security" , "Groupstudy R&S"
>>>Subject: real world problem
>>>Date: Wed, 02 Mar 2005 15:52:26 -0500
>>>
>>>
>>>
>>>>Ok folks I need some collective brain power.
>>>>
>>>>Recently I started noticing issues surfing to
>>
>>certain websites during
>>
>>>the day.
>>>
>>>
>>>>After sticking a sniffer on the network I noticed
>>
>>an excessive number
>>
>>>of
>>>
>>>
>>>>retransmissions happening. I dug a little deeper
>>
>>and noticed that it
>>
>>>didn't
>>>
>>>
>>>>matter what the protocol was ( www,ssl,3389...etc)
>>
>>I was getting these
>>
>>>>retransmissions, and it didn't matter how close
>>
>>the systems were. Only
>>
>>>thing
>>>
>>>
>>>>that mattered was time of day ( Aha!..sorta )
>>
>>During business hours it
>>
>>>was
>>>
>>>
>>>>much more pronounced but it was also happening
>>
>>after hours as well.
>>
>>>>I can go and sit on my DC VLAN, plugged directly
>>
>>into my 6509 and
>>
>>>connect to
>>>
>>>
>>>>an apache box also on the DC subnet and still get
>>
>>them.
>>
>>>>I setup spanning and noticed that EVERYONE is
>>
>>seeing them.
>>
>>>>With traffic internal experiencing it I can rule
>>
>>out the FW or the
>>
>>>Internet
>>>
>>>
>>>>circuit. I believe the issue is related to the
>>
>>6509 and its
>>
>>>configuration.
>>>
>>>
>>>>I'm looking for a little guidance on how to best
>>
>>troubleshoot this
>>
>>>traffic.
>>>
>>>
>>>>Other than seeing excessive retran's I don't get
>>
>>any data from the
>>
>>>sniffer,
>>>
>>>
>>>>and the 6509 shows its utilization at 3% over 5
>>
>>min.
>>
>>>>The 6509 has 2 SupII/MSFC2's as well as 2 8 port
>>
>>GBIC's that connect to
>>
>>>3
>>>
>>>
>>>>3508's on 3 separate floors. Each 3508 has 2
>>
>>uplinks to the 6509, and 6
>>
>>>3550's
>>>
>>>
>>>>connected to the other GBIC's.
>>>>
>>>>Traffic utilization is minimal on the Fiber and I
>>
>>show no input/output
>>
>>>errors
>>>
>>>
>>>>on them.
>>>>
>>>>I'm running c6sup22-dsv-mz.121-22.E1 on the 6509.
>>>>
>>>>I have IPX bridged across every VLAN ( Not by
>>
>>choice ) as well 5
>>
>>>separate
>>>
>>>
>>>>VLAN's.
>>>>
>>>>
>>>>Any thoughts on how to best go about
>>
>>troubleshooting this issue?
>>
>>>>Nothing has changed recently that I am aware of on
>>
>>the network, but
>>
>>>about a
>>>
>>>
>>>>month ago the problem appeared.
>>>>
>>>>
>>>>
>>>>-- Thanks,
>>>>
>>>>Larry
>>>>
>>
>>>_______________________________________________________________________
>>>
>>>>Subscription information may be found at:
>>>>http://www.groupstudy.com/list/CCIELab.html
>>>
>>>
>>>
> _______________________________________________________________________
>
>>>Subscription information may be found at:
>>>http://www.groupstudy.com/list/CCIELab.html
>>
>>--
>>Thanks,
>>
>>Larry
>>
>>
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> _______________________________________________________________________
> Subscription information may be found at:
> http://www.groupstudy.com/list/CCIELab.html
-- Thanks,Larry
This archive was generated by hypermail 2.1.4 : Sun Apr 03 2005 - 17:56:40 GMT-3