From: Nitin Venugopal (nitinsworld@gmail.com)
Date: Tue Mar 17 2009 - 15:43:50 ART
Hi,
While the mails were floating about port channel (suspension), just happen
to get a encounter a realtime issue with a call from customer:)
6500 - Port channel 1 has (Te 1/1 and 2/1 - Differnt Modules) running
12.18SXF (15A)
1) By mistake customer added a new allowed vlan on one of the port , Te 1/1
--( commad switch port trunk allowed vlan add 96)
2) Soon after that the whole Port Channel went down.
I have just started the investigation, but wondering why did the port
channel go down and why did it suspend the other port 2/1 also.
Best Regard
Nitin
---------Logs
Mar 17 19:55:04.139 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te1/1 is not compatible
with Te2/1 and will be suspended (vlan mask is different)
001253: Mar 17 19:55:39.671 UAE: %STANDBY-3-DUPADDR: Duplicate address
172.24.22.2 on Vlan71, sourced by 001c.0f5f.ef00
001254: Mar 17 19:55:50.924 UAE: %OSPF-5-ADJCHG: Process 1, Nbr 172.24.100.8
on Vlan100 from FULL to DOWN, Neighbor Down: Dead timer expired
001255: Mar 17 19:55:50.928 UAE: %OSPF-5-ADJCHG: Process 1, Nbr 172.24.100.8
on Vlan30 from FULL to DOWN, Neighbor Down: Dead timer expired
001256: Mar 17 19:55:51.117 UAE: %OSPF-5-ADJCHG: Process 1, Nbr 172.24.100.8
on Vlan94 from FULL to DOWN, Neighbor Down: Dead timer expired
001257: Mar 17 19:56:09.805 UAE: %STANDBY-3-DUPADDR: Duplicate address
172.24.22.66 on Vlan72, sourced by 001c.0f5f.ef00
001258: Mar 17 19:56:23.506 UAE: %SVCLC-5-FWTRUNK: Firewalled VLANs
configured on trunks
Mar 17 19:56:23.501 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te1/1 is not compatible
with Po1 and will be suspended (vlan mask is different)
Mar 17 19:56:23.501 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te1/1 is not compatible
with Po1 and will be suspended (vlan mask is different)
Mar 17 19:56:23.501 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te2/1 is not compatible
with Po1 and will be suspended (vlan mask is different)
Mar 17 19:56:23.513 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet2/1, changed state to down
Mar 17 19:56:23.517 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
Port-channel1, changed state to down
001259: Mar 17 19:56:23.598 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
Interface TenGigabitEthernet2/1, changed state to down
001260: Mar 17 19:56:23.602 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
Interface Port-channel1, changed state to down
001261: Mar 17 19:56:23.618 UAE: %LINK-3-UPDOWN: Interface Port-channel1,
changed state to down
Mar 17 19:56:23.553 UAE: %LINK-SP-3-UPDOWN: Interface Port-channel1, changed
state to down
001262: Mar 17 19:56:29.743 UAE: %OSPF-5-ADJCHG: Process 1, Nbr 172.24.100.8
on Vlan100 from LOADING to FULL, Loading Done
001263: Mar 17 19:56:37.643 UAE: %OSPF-5-ADJCHG: Process 1, Nbr 172.24.100.8
on Vlan94 from LOADING to FULL, Loading Done
001264: Mar 17 19:56:59.625 UAE: %OSPF-5-ADJCHG: Process 1, Nbr 172.24.100.8
on Vlan48 from FULL to DOWN, Neighbor Down: Dead timer expired
001265: Mar 17 19:56:59.625 UAE: %OSPF-5-ADJCHG: Process 1, Nbr 172.24.100.8
on Vlan32 from FULL to DOWN, Neighbor Down: Dead timer expired
001266: Mar 17 19:57:13.919 UAE: %LINK-5-CHANGED: Interface
TenGigabitEthernet1/1, changed state to administratively down
Mar 17 19:57:13.857 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te2/1 is not compatible
with Po1 and will be suspended (vlan mask is different)
Mar 17 19:57:13.909 UAE: %LINK-SP-5-CHANGED: Interface
TenGigabitEthernet1/1, changed state to administratively down
001267: Mar 17 19:57:15.533 UAE: %LINK-3-UPDOWN: Interface
TenGigabitEthernet1/1, changed state to down
Mar 17 19:57:15.533 UAE: %LINK-SP-3-UPDOWN: Interface TenGigabitEthernet1/1,
changed state to down
001268: Mar 17 19:57:16.639 UAE: %LINK-3-UPDOWN: Interface
TenGigabitEthernet1/1, changed state to up
Mar 17 19:57:16.622 UAE: %LINK-SP-3-UPDOWN: Interface TenGigabitEthernet1/1,
changed state to up
001269: Mar 17 19:57:20.104 UAE: %SYS-5-CONFIG_I: Configured from console by
console
Mar 17 19:57:19.571 UAE: %EC-SP-5-CANNOT_BUNDLE2: Te1/1 is not compatible
with Po1 and will be suspended (vlan mask is different)
001270: .Mar 17 19:58:17.089 UAE: %SVCLC-5-FWTRUNK: Firewalled VLANs
configured on trunks
001271: .Mar 17 19:58:17.097 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
Interface TenGigabitEthernet1/1, changed state to up
001272: .Mar 17 19:58:17.109 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
Interface TenGigabitEthernet2/1, changed state to up
001273: .Mar 17 19:58:17.141 UAE: %LINK-3-UPDOWN: Interface Port-channel1,
changed state to up
001274: .Mar 17 19:58:17.145 UAE: %LINEPROTO-5-UPDOWN: Line protocol on
Interface Port-channel1, changed state to up
Mar 17 19:58:17.062 UAE: %EC-SP-5-COMPATIBLE: Te1/1 is compatible with
port-channel members
Mar 17 19:58:17.078 UAE: %EC-SP-5-COMPATIBLE: Te2/1 is compatible with
port-channel members
Mar 17 19:58:17.106 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet1/1, changed state to up
Mar 17 19:58:17.106 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet2/1, changed state to up
Mar 17 19:58:17.258 UAE: %LINK-SP-3-UPDOWN: Interface Port-channel1, changed
state to up
Mar 17 19:58:17.266 UAE: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface
Port-channel1, changed state to up
---------------Logs-------------
On Tue, Mar 17, 2009 at 9:10 PM, ALL From_NJ <all.from.nj@gmail.com> wrote:
> Would be nice to see the logs for when the ports are going into suspended
> state ... messages displayed at this time are key.
>
> Suspended state occurs due to the control plane protocols putting the port
> into it. What are the control protocols seeing that they do not like?
> Examples of control protocols are STP, CDP (CDP carries a lot of control
> information inside it too), etc ...
>
> Could the switch be seeing it's own packets coming back to it? Probably no
> way to tell over email and writing a book will not help ... but I would
> suggest to monitor both switches and watch for the log messages to occur.
>
> I do not think it is a config problem on this switch since it comes up and
> stays up for some time. Perhaps the switch it connects to is misconfigured
> or has issues w/ flapping links or something like this? I would also
> suggest to look at your topology and see where a control plane / protocol
> problem could be happening ... and start there.
>
> HTH,
>
> Andrew Lee Lissitz
>
>
>
> On Tue, Mar 17, 2009 at 12:03 PM, <andy@cloud9.net> wrote:
>
> > My first thought would be a spanning tree issue but I'm thinking
> > there should be a reason why the etherchannel goes into a suspended
> > state.
> >
> > One other thing to look for is if you have QoS enable and the port
> > in the etherchannel are in different module. QoS makes sure that the
> > hardware buffers are the same and if they are not will suspend some
> > ports. There is a work around for this issue "no mls qos
> > channel-consistency" on the port-channel will fix this problem.
> >
> > If neither of these help you I will be glad to look at show tech's
> > from both switches. Just zip them and send them to me.
> >
> > Andy
> >
> > Quoting Network places <uniquemoonlight@gmail.com>:
> >
> > > Hi All,
> > > i have an etherchannel configured between 6500's. Which stays
> up
> > for
> > > a while when you no shut but after a while it goes in the suspension
> > state
> > > and stays there connectivity is lost as well. Cant find a reason for
> > that.
> > > Any help would be highly appreciated.
> > >
> > >
> > > Thanks
> > >
> > >
> > > Blogs and organic groups at http://www.ccie.net[1]
> > >
> > > _______________________________________________________________________
> > > Subscription information may be found at:
> > > http://www.groupstudy.com/list/CCIELab.html[2]<http://www.groupstudy.com/list/CCIELab.html%5B2%5D>
> <http://www.groupstudy.com/list/CCIELab.html%5B2%5D>
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> > Links:
> > ------
> > [1] http://www.ccie.net/
> > [2] http://www.groupstudy.com/list/CCIELab.html
> >
> >
> > Blogs and organic groups at http://www.ccie.net
> >
> > _______________________________________________________________________
> > Subscription information may be found at:
> > http://www.groupstudy.com/list/CCIELab.html
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> --
> Andrew Lee Lissitz
> all.from.nj@gmail.com
>
>
> Blogs and organic groups at http://www.ccie.net
>
> _______________________________________________________________________
> Subscription information may be found at:
> http://www.groupstudy.com/list/CCIELab.html
Blogs and organic groups at http://www.ccie.net
This archive was generated by hypermail 2.1.4 : Mon Apr 06 2009 - 06:44:05 ART