Re: Port channel goes in suspension state 6500

From: ALL From_NJ (all.from.nj@gmail.com)
Date: Tue Mar 17 2009 - 16:31:51 ART

Yep, the VLANs have to be the same, pruning, allowed, etc ... everything has
to be the same across all the ports in an etherchannel.

Since the initial problem listed in the first email takes a little time to
occur, I figure it is the connected switch that is the problem. Changing
topologies or something, is causing the control protocol on the local switch
to see something it does not like. The answer should be indicated via the
logs ...

Pretty cool to see you are working this real time! Nice!

It is also nice to see a way to trigger this and simulate this in our labs.

Andrew Lee Lissitz

On Tue, Mar 17, 2009 at 2:43 PM, Nitin Venugopal <nitinsworld@gmail.com>wrote:

> Hi,
>
> While the mails were floating about port channel (suspension), just
> happen to get a encounter a realtime issue with a call from customer:)
>
> 6500 - Port channel 1 has (Te 1/1 and 2/1 - Differnt Modules) running
> 12.18SXF (15A)
>
> 1) By mistake customer added a new allowed vlan on one of the port , Te 1/1
> --( commad switch port trunk allowed vlan add 96)
> 2) Soon after that the whole Port Channel went down.
>
> I have just started the investigation, but wondering why did the port
> channel go down and why did it suspend the other port 2/1 also.
>
> Best Regard
> Nitin
>
>
> ---------Logs
>
> Mar 17 19:55:04.139 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te1/1 is not compatible
> with Te2/1 and will be suspended (vlan mask is different)
> 001253: Mar 17 19:55:39.671 UAE: %STANDBY-3-DUPADDR: Duplicate address
> 172.24.22.2 on Vlan71, sourced by 001c.0f5f.ef00
> 001254: Mar 17 19:55:50.924 UAE: %OSPF-5-ADJCHG: Process 1, Nbr
> 172.24.100.8 on Vlan100 from FULL to DOWN, Neighbor Down: Dead timer expired
> 001255: Mar 17 19:55:50.928 UAE: %OSPF-5-ADJCHG: Process 1, Nbr
> 172.24.100.8 on Vlan30 from FULL to DOWN, Neighbor Down: Dead timer expired
> 001256: Mar 17 19:55:51.117 UAE: %OSPF-5-ADJCHG: Process 1, Nbr
> 172.24.100.8 on Vlan94 from FULL to DOWN, Neighbor Down: Dead timer expired
> 001257: Mar 17 19:56:09.805 UAE: %STANDBY-3-DUPADDR: Duplicate address
> 172.24.22.66 on Vlan72, sourced by 001c.0f5f.ef00
> 001258: Mar 17 19:56:23.506 UAE: %SVCLC-5-FWTRUNK: Firewalled VLANs
> configured on trunks
> Mar 17 19:56:23.501 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te1/1 is not compatible
> with Po1 and will be suspended (vlan mask is different)
> Mar 17 19:56:23.501 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te1/1 is not compatible
> with Po1 and will be suspended (vlan mask is different)
> Mar 17 19:56:23.501 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te2/1 is not compatible
> with Po1 and will be suspended (vlan mask is different)
> Mar 17 19:56:23.513 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
> TenGigabitEthernet2/1, changed state to down
> Mar 17 19:56:23.517 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
> Port-channel1, changed state to down
> 001259: Mar 17 19:56:23.598 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
> Interface TenGigabitEthernet2/1, changed state to down
> 001260: Mar 17 19:56:23.602 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
> Interface Port-channel1, changed state to down
> 001261: Mar 17 19:56:23.618 UAE: %LINK-3-UPDOWN: Interface Port-channel1,
> changed state to down
> Mar 17 19:56:23.553 UAE: %LINK-SP-3-UPDOWN: Interface Port-channel1,
> changed state to down
> 001262: Mar 17 19:56:29.743 UAE: %OSPF-5-ADJCHG: Process 1, Nbr
> 172.24.100.8 on Vlan100 from LOADING to FULL, Loading Done
> 001263: Mar 17 19:56:37.643 UAE: %OSPF-5-ADJCHG: Process 1, Nbr
> 172.24.100.8 on Vlan94 from LOADING to FULL, Loading Done
> 001264: Mar 17 19:56:59.625 UAE: %OSPF-5-ADJCHG: Process 1, Nbr
> 172.24.100.8 on Vlan48 from FULL to DOWN, Neighbor Down: Dead timer expired
> 001265: Mar 17 19:56:59.625 UAE: %OSPF-5-ADJCHG: Process 1, Nbr
> 172.24.100.8 on Vlan32 from FULL to DOWN, Neighbor Down: Dead timer expired
> 001266: Mar 17 19:57:13.919 UAE: %LINK-5-CHANGED: Interface
> TenGigabitEthernet1/1, changed state to administratively down
> Mar 17 19:57:13.857 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te2/1 is not compatible
> with Po1 and will be suspended (vlan mask is different)
> Mar 17 19:57:13.909 UAE: %LINK-SP-5-CHANGED: Interface
> TenGigabitEthernet1/1, changed state to administratively down
> 001267: Mar 17 19:57:15.533 UAE: %LINK-3-UPDOWN: Interface
> TenGigabitEthernet1/1, changed state to down
> Mar 17 19:57:15.533 UAE: %LINK-SP-3-UPDOWN: Interface
> TenGigabitEthernet1/1, changed state to down
> 001268: Mar 17 19:57:16.639 UAE: %LINK-3-UPDOWN: Interface
> TenGigabitEthernet1/1, changed state to up
> Mar 17 19:57:16.622 UAE: %LINK-SP-3-UPDOWN: Interface
> TenGigabitEthernet1/1, changed state to up
> 001269: Mar 17 19:57:20.104 UAE: %SYS-5-CONFIG_I: Configured from console
> by console
> Mar 17 19:57:19.571 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te1/1 is not compatible
> with Po1 and will be suspended (vlan mask is different)
> 001270: .Mar 17 19:58:17.089 UAE: %SVCLC-5-FWTRUNK: Firewalled VLANs
> configured on trunks
> 001271: .Mar 17 19:58:17.097 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
> Interface TenGigabitEthernet1/1, changed state to up
> 001272: .Mar 17 19:58:17.109 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
> Interface TenGigabitEthernet2/1, changed state to up
> 001273: .Mar 17 19:58:17.141 UAE: %LINK-3-UPDOWN: Interface Port-channel1,
> changed state to up
> 001274: .Mar 17 19:58:17.145 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
> Interface Port-channel1, changed state to up
> Mar 17 19:58:17.062 UAE: %EC-SP-5-COMPATIBLE: Te1/1 is compatible with
> port-channel members
> Mar 17 19:58:17.078 UAE: %EC-SP-5-COMPATIBLE: Te2/1 is compatible with
> port-channel members
> Mar 17 19:58:17.106 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
> TenGigabitEthernet1/1, changed state to up
> Mar 17 19:58:17.106 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
> TenGigabitEthernet2/1, changed state to up
> Mar 17 19:58:17.258 UAE: %LINK-SP-3-UPDOWN: Interface Port-channel1,
> changed state to up
> Mar 17 19:58:17.266 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
> Port-channel1, changed state to up
> ---------------Logs-------------
>
>
>
> On Tue, Mar 17, 2009 at 9:10 PM, ALL From_NJ <all.from.nj@gmail.com>wrote:
>
>> Would be nice to see the logs for when the ports are going into suspended
>> state ... messages displayed at this time are key.
>>
>> Suspended state occurs due to the control plane protocols putting the port
>> into it. What are the control protocols seeing that they do not like?
>> Examples of control protocols are STP, CDP (CDP carries a lot of control
>> information inside it too), etc ...
>>
>> Could the switch be seeing it's own packets coming back to it? Probably
>> no
>> way to tell over email and writing a book will not help ... but I would
>> suggest to monitor both switches and watch for the log messages to occur.
>>
>> I do not think it is a config problem on this switch since it comes up and
>> stays up for some time. Perhaps the switch it connects to is
>> misconfigured
>> or has issues w/ flapping links or something like this? I would also
>> suggest to look at your topology and see where a control plane / protocol
>> problem could be happening ... and start there.
>>
>> HTH,
>>
>> Andrew Lee Lissitz
>>
>>
>>
>> On Tue, Mar 17, 2009 at 12:03 PM, <andy@cloud9.net> wrote:
>>
>> > My first thought would be a spanning tree issue but I'm thinking
>> > there should be a reason why the etherchannel goes into a suspended
>> > state.
>> >
>> > One other thing to look for is if you have QoS enable and the port
>> > in the etherchannel are in different module. QoS makes sure that the
>> > hardware buffers are the same and if they are not will suspend some
>> > ports. There is a work around for this issue "no mls qos
>> > channel-consistency" on the port-channel will fix this problem.
>> >
>> > If neither of these help you I will be glad to look at show tech's
>> > from both switches. Just zip them and send them to me.
>> >
>> > Andy
>> >
>> > Quoting Network places <uniquemoonlight@gmail.com>:
>> >
>> > > Hi All,
>> > > i have an etherchannel configured between 6500's. Which stays
>> up
>> > for
>> > > a while when you no shut but after a while it goes in the suspension
>> > state
>> > > and stays there connectivity is lost as well. Cant find a reason for
>> > that.
>> > > Any help would be highly appreciated.
>> > >
>> > >
>> > > Thanks
>> > >
>> > >
>> > > Blogs and organic groups at http://www.ccie.net[1]
>> > >
>> > >
>> _______________________________________________________________________
>> > > Subscription information may be found at:
>> > > http://www.groupstudy.com/list/CCIELab.html[2]<http://www.groupstudy.com/list/CCIELab.html%5B2%5D>
>> <http://www.groupstudy.com/list/CCIELab.html%5B2%5D>
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> >
>> >
>> > Links:
>> > ------
>> > [1] http://www.ccie.net/
>> > [2] http://www.groupstudy.com/list/CCIELab.html
>> >
>> >
>> > Blogs and organic groups at http://www.ccie.net
>> >
>> > _______________________________________________________________________
>> > Subscription information may be found at:
>> > http://www.groupstudy.com/list/CCIELab.html
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>> --
>> Andrew Lee Lissitz
>> all.from.nj@gmail.com
>>
>>
>> Blogs and organic groups at http://www.ccie.net
>>
>> _______________________________________________________________________
>> Subscription information may be found at:
>> http://www.groupstudy.com/list/CCIELab.html
>>
>>
>>
>>
>>
>>
>>
>>
> 

-- 
Andrew Lee Lissitz
all.from.nj@gmail.com

Blogs and organic groups at http://www.ccie.net

This archive was generated by hypermail 2.1.4 : Mon Apr 06 2009 - 06:44:05 ART