From: MADMAN (dmadlan@qwest.com)
Date: Tue Apr 20 2004 - 10:26:11 GMT-3
I forget all the steps that have been takin but I have another idea
since it just happended to me the other day. I was in the lab
connecting up a 3500 to another 3500 that was connected to a 6500. All
connections were trunks. The new 3500 could ping the adjacent 3500 but
no further. The adjacent 3500 could ping the new 3500 and anywhere
else. After wasting an hour screwing around and getting totally puzzled
as this was very simple I pulled the plug on the adjacent 3500 and reset
it.
Everything worked fine after the reset!
Dave
Brian McGahan wrote:
> Kevin,
>
> Try changing the MAC addresses of the hosts. I've seen some odd
> bugs where certain MAC specific addresses can cause problems depending
> on traffic load, so it's not unfathomable that the direction of flow
> could cause it as well.
>
>
> HTH,
>
> Brian McGahan, CCIE #8593
> bmcgahan@internetworkexpert.com
>
> Internetwork Expert, Inc.
> http://www.InternetworkExpert.com
> Toll Free: 877-224-8987 x 705
> Outside US: 775-826-4344 x 705
>
>
>
>>-----Original Message-----
>>From: Minihane Kevin-kminih01 [mailto:Kevin.Minihane@motorola.com]
>>Sent: Monday, April 19, 2004 7:44 PM
>>To: 'Kenneth Wygand'; Brian McGahan; Church, Chuck
>>Subject: RE: Strange trunking issue
>>
>>Hi Kenneth
>>
>>Thanks for all the help from everyone.
>>When I said I set up a packet sniffer, I SPANned the affected port to
>>another port on the 3550, and used a packet analyser to look at the
>>traffic passing between the hosts. I saw nothing untoward at all.
>
> I've
>
>>also cleared the counters on the interfaces concerned, and I'm not
>
> seeing
>
>>any CRC errors, dropped packets etc
>>
>>I'm not performing any QoS at all! I can open a TAC case, which I was
>>going to do in the morning, as this is starting to depress me!
>>
>>It must be nearly quitting time for all you guys over there in the
>>US....it's coming up to 1:45 am here.
>>
>>Going to stick with this another little while, and see if I cant crack
>
> it.
>
>>If not, will have to open a case with TAC
>>
>>Thanks
>>
>>Kevin
>>
>>-----Original Message-----
>>From: Kenneth Wygand [mailto:KWygand@customonline.com]
>>Sent: 20 April 2004 01:33
>>To: Minihane Kevin-kminih01; Brian McGahan; Church, Chuck
>>Cc: ccielab@groupstudy.com
>>Subject: RE: Strange trunking issue
>>
>>
>>Kevin,
>>
>>A few days ago you said:
>>
>><snip>
>>I've used a packet sniffer to check for windowing (or rather,
>>
>>>for continuous changes in the windowing) and everything appears
>>
>>normal.
>>
>>>I'm not seeing excessive traffic, or changes in the window sizes.
>>
>></snip>
>>
>>Did you SPAN a port to a packet sniffer on another workstation or just
>>load a packet sniffer on one of the devices? Try SPANning the
>
> affected
>
>>port (monitor command) on the 3550 to another port on the 3550 and
>
> sniff
>
>>that. Do you see any other types of traffic other than TCP traffic?
>
> Do
>
>>you see CRC errors or retransmissions? How about the interface
>>statistics on the affected port of the 3550?
>>
>>I'm also assuming you aren't doing ANY type of QoS on the equipment...
>>do you have any type of WRED configuration that could've gone nuts?
>
> Any
>
>>type of QoS configuration at all that could be bug infested?
>>
>>Last but not least, do you have a SmartNet contract? Can you open a
>
> TAC
>
>>case? Is it feasible to post your configurations here after masking
>>your sensitive IP's, descriptions, hostnames, etc.?
>>
>>Kenneth E. Wygand
>>Systems Engineer, Project Services
>>CISSP #37102, CCNP, CCDP, ACSP, Cisco IPT Design Specialist, MCP, CNA,
>>Network+, A+
>>Custom Computer Specialists, Inc.
>>"The only unattainable goal is the one not attempted."
>>-Anonymous
>>
>>-----Original Message-----
>>From: nobody@groupstudy.com [mailto:nobody@groupstudy.com] On Behalf
>
> Of
>
>>Minihane Kevin-kminih01
>>Sent: Monday, April 19, 2004 8:14 PM
>>To: 'Brian McGahan'; MinihaneKevin-kminih01; Church, Chuck
>>Cc: ccielab@groupstudy.com
>>Subject: RE: Strange trunking issue
>>
>>Hi Brian
>>
>>I've put a host onto the 5500, in the same VLAN as hostB. I've also
>>placed all the GBIC's into the same access VLAN. The slowness is
>
> still
>
>>there.
>>
>>I've placed another host onto the 3550, in the same VLAN as hostB, and
>>the speed is what you would expect (10Mb per sec).
>>
>>At times, the speed between the 5500 and the 3550 is down to 300k per
>>second. At it's worst, I've seen speeds of 24k per sec! And again,
>
> this
>
>>is only in one direction! It happends when I try to "push" something
>>from hostA to hostB. If I try to push from hostB to hostA, then
>>everything is ok!!!!!!
>>
>>
>>
>>-----Original Message-----
>>From: Brian McGahan [mailto:bmcgahan@internetworkexpert.com]
>>Sent: 20 April 2004 00:21
>>To: Minihane Kevin-kminih01; Church, Chuck
>>Cc: ccielab@groupstudy.com
>>Subject: RE: Strange trunking issue
>>
>>
>>Kevin,
>>
>> Try setting both hosts and the GBICs between the switches in the
>>same access VLAN. This will let you know whether or not it is a
>>trunking related issue.
>>
>>HTH,
>>
>>Brian McGahan, CCIE #8593
>>bmcgahan@internetworkexpert.com
>>
>>Internetwork Expert, Inc.
>>http://www.InternetworkExpert.com
>>Toll Free: 877-224-8987 x 705
>>Outside US: 775-826-4344 x 705
>>
>>
>>
>>>-----Original Message-----
>>>From: nobody@groupstudy.com [mailto:nobody@groupstudy.com] On Behalf
>>
>>Of
>>
>>>Minihane Kevin-kminih01
>>>Sent: Monday, April 19, 2004 6:01 PM
>>>To: 'Church, Chuck'
>>>Cc: 'ccielab@groupstudy.com'
>>>Subject: RE: Strange trunking issue
>>>
>>>I've even tried a different 3550 !....At the moment, I'm running
>>
>>version
>>
>>>12.1(19) on both my 3550's. I've tried ftp'ing between two hosts on
>>
>>the
>>
>>>3550, and it works perfectly.
>>>
>>>-----Original Message-----
>>>From: Church, Chuck [mailto:cchurch@wamnetgov.com]
>>>Sent: 19 April 2004 23:56
>>>To: Minihane Kevin-kminih01; Kenneth Wygand; Robert Fischer; MADMAN
>>>Cc: ccielab@groupstudy.com
>>>Subject: RE: Strange trunking issue
>>>
>>>
>>>Maybe try a different GBIC port and/or copper port on the 3550?
>
> Newer
>
>>>code for the 3550?
>>>
>>>
>>>Chuck Church
>>>Lead Design Engineer
>>>CCIE #8776, MCNE, MCSE
>>>Wam!Net Government Services - Design & Implementation Team
>>>13665 Dulles Technology Dr. Ste 250
>>>Herndon, VA 20171
>>>Office: 703-480-2569
>>>Cell: 703-819-3495
>>>cchurch@wamnetgov.com
>>>PGP key:
>>>
>>
> http://pgp.mit.edu:11371/pks/lookup?op=index&search=cchurch%40wamnetgov.
>
>>>com
>>>-----Original Message-----
>>>From: Minihane Kevin-kminih01 [mailto:Kevin.Minihane@motorola.com]
>>>Sent: Monday, April 19, 2004 6:38 PM
>>>To: 'Kenneth Wygand'; Robert Fischer; Church, Chuck; MADMAN
>>>Cc: ccielab@groupstudy.com
>>>Subject: RE: Strange trunking issue
>>>
>>>Well, this is really doing my head in now! I've replaced the fibre,
>>>GBIC's on the 3550. Have changed the encapsulation to ISL (from
>>
>>dot1q).
>>
>>>have reversed the fibres, etc. I've just placed hostB onto the 5500
>>>again, and the ftp takes off like a rocket. I'm beginning to
>
> suspect
>
>>>the 3550> Has anyone ever seen "slowness" on a device connected to
>
> a
>
>>>3550? I'd have thought it would have been ok....
>>>
>>>
>>>
>>>-----Original Message-----
>>>From: nobody@groupstudy.com [mailto:nobody@groupstudy.com]On Behalf
>
> Of
>
>>>Minihane Kevin-kminih01
>>>Sent: 19 April 2004 21:15
>>>To: 'Kenneth Wygand'; Robert Fischer; Church, Chuck; MADMAN
>>>Cc: ccielab@groupstudy.com
>>>Subject: RE: Strange trunking issue
>>>
>>>
>>>Hi
>>>Sorry for the delay in this. Was sent away with work. I've tried
>>>swapping the strands around, still no joy. I've used a different
>>
>>cable,
>>
>>>and different GBIC's as well in the 3550. I'm going to take Chuck's
>>>advice, and try changing the encapsulation to ISL, just in case
>
> there
>
>>is
>>
>>>some sort of bug causing me to go grey prematurely. I'll let you
>
> know
>
>>>how that works out. It's 9 pm here in ireland, so I'll try this
>
> later
>
>>>on tonight and let u all know the result
>>>
>>>Thanks for all the help so far!
>>>
>>>Kevin
>>>
>>>-----Original Message-----
>>>From: Kenneth Wygand [mailto:KWygand@customonline.com]
>>>Sent: 16 April 2004 12:46
>>>To: Minihane Kevin-kminih01; Robert Fischer; Church, Chuck; Minihane
>>>Kevin-kminih01; MADMAN
>>>Cc: ccielab@groupstudy.com
>>>Subject: RE: Strange trunking issue
>>>
>>>
>>>Try swapping the fiber strands on both sides (take the receive port
>
> on
>
>>>the 5500 and swap it with the transmit port on the 5500, then do the
>>>same thing on the 3550 side). Then try your transmissions again and
>>
>>let
>>
>>>us know.
>>>
>>>Good luck,
>>>Ken
>>>
>>> -----Original Message-----
>>> From: nobody@groupstudy.com on behalf of Minihane Kevin-kminih01
>>>
>>> Sent: Fri 4/16/2004 5:13 AM
>>> To: 'Robert Fischer'; 'Church, Chuck'; Minihane Kevin-kminih01;
>>>'MADMAN'
>>> Cc: ccielab@groupstudy.com
>>> Subject: RE: Strange trunking issue
>>>
>>>
>>>
>>> One of the hosts is running Solaris, (hostB), while HostA is a
>>>Linux (ReHat) machine. I've also put another Solaris host onto the
>>>5500, with the same effect.
>>>
>>> I've used a packet sniffer to check for windowing (or rather,
>>>for continuous changes in the windowing) and everything appears
>>
>>normal.
>>
>>>I'm not seeing excessive traffic, or changes in the window sizes.
>>>
>>> All hosts are running at 100 Mbs Full duplex. I've played
>>>around with auto settings on each switch, and still have the same
>>>result.
>>>
>>> The mtu on both is 1500, and (using a packet sniffer) I've
>>>checked the packet size of the FTP data. This is 1518, which is
>>>correct? (to include the headers etc?)
>>>
>>> Both hosts are in the same VLAN.
>>>
>>> This is doing my head in!
>>>
>>>
>>> -----Original Message-----
>>> From: Robert Fischer [mailto:rfischer@broadcom.com]
>>> Sent: 16 April 2004 01:09
>>> To: 'Church, Chuck'; 'Minihane Kevin-kminih01'; 'MADMAN'
>>> Cc: ccielab@groupstudy.com
>>> Subject: RE: Strange trunking issue
>>>
>>>
>>> What are the hosts running, Windows/ linux? What are the host
>>>uplinks,
>>> 10/100? The host connected to the 5500 may be overrunning the
>>>5500's port
>>> buffer. Capture some packets on the 5500 size and watch for the
>>>windowing
>>> size to continuously change and the host to retransmit. Newer
>>>versions of
>>> Windows like XP are defaulting to 64K window sizes and causing
>>>port buffer
>>> issues with older gear. Some versions of Red Hat have difficulty
>>>negotiating
>>> this too.
>>>
>>> -Robb
>>>
>>> -----Original Message-----
>>> From: nobody@groupstudy.com [mailto:nobody@groupstudy.com] On
>>>Behalf Of
>>> Church, Chuck
>>> Sent: Thursday, April 15, 2004 10:35 AM
>>> To: Minihane Kevin-kminih01; MADMAN
>>> Cc: ccielab@groupstudy.com
>>> Subject: RE: Strange trunking issue
>>>
>>>
>>> So now put the Host B on the 3550, and see if the problem
>>>exists. Any
>>> chance it's an MTU issue? Are A and B in the same VLAN? If
>>>not, who's
>>> routing, the 3550?
>>>
>>>
>>> Chuck Church
>>> Lead Design Engineer
>>> CCIE #8776, MCNE, MCSE
>>> Wam!Net Government Services - Design & Implementation Team 13665
>>>Dulles
>>> Technology Dr. Ste 250 Herndon, VA 20171
>>> Office: 703-480-2569
>>> Cell: 703-819-3495
>>> cchurch@wamnetgov.com
>>> PGP key:
>>>
>>>
>>
> http://pgp.mit.edu:11371/pks/lookup?op=index&search=cchurch%40wamnetgov.
>
>>> com
>>> -----Original Message-----
>>> From: Minihane Kevin-kminih01
>>>[mailto:Kevin.Minihane@motorola.com]
>>> Sent: Thursday, April 15, 2004 12:23 PM
>>> To: 'MADMAN'; Minihane Kevin-kminih01
>>> Cc: ccielab@groupstudy.com
>>> Subject: RE: Strange trunking issue
>>>
>>> Hiya
>>>
>>> Thanks for the input. I moved hostB onto the 5500, and the
>>>problem
>>> disapeared. I put it back onto the 3550, and the same problem
>>>exists
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: MADMAN [mailto:dmadlan@qwest.com]
>>> Sent: 15 April 2004 17:04
>>> To: Minihane Kevin-kminih01
>>> Cc: ccielab@groupstudy.com
>>> Subject: Re: Strange trunking issue
>>>
>>>
>>>
>>> Your assuming you have a switching problem accross the trunk.
>>>Plug host
>>> B into the 5500 and see if you still have the problem.
>>>
>>> Dave
>>>
>>> Minihane Kevin-kminih01 wrote:
>>>
>>> > Hi
>>> >
>>> > Apologies in advance if this seems long winded, but I wanted
>>>to attach
>>> the configurations, and output of various commands.
>>> >
>>> > I'm experiencing a very strange trunking issue, and was
>>>wondering if
>>> anyone could perhaps have some input which might help?
>>> >
>>> >
>>> > The following is my layout.
>>> >
>>> >
>>> > Host A ------- Cat5500 ------------------------------ Cat3550
>>> > ------------ Host B
>>> >
>>> > Each host is connected to it's switch using a 100Mb full
>>>duplex
>>> > connection
>>> >
>>> > I have two switches (cat 5500 running CatOS, and a 3550). I'm
>>> trunking between both using dot1q encapsulation. Vtp
>>>information is flowing
>>> freely between them, and I can configure vlans on both,
>>>propagate the vlans
>>> between each switch, etc. The trunk between the switches is a
>>>gigabit fibre
>>> connection.
>>> >
>>> > The problem I am having is when I try to ftp from host A to
>>>host B,
>>> and vice versa.
>>> >
>>> > If I ftp from Host A to Host B, I experience very slow speeds
>>> > (sometimes as low as 24k per second). If I come from Host B
>>>to host
>>> > A, the speed increases to 8Mbs per second! I've checked the
>>>usual
>>> > (speed, duplex, check for CRC, dropped packets) and I get
>>>nothing
>>> >
>>> > The following is the config of the interface g0/1 on the 3550
>>> >
>>> > Lab_3550_1#show run int g0/1
>>> > Building configuration...
>>> >
>>> > Current configuration : 169 bytes
>>> > !
>>> > interface GigabitEthernet0/1
>>> > description Trunk to main lab 5500 port 1/1 switchport trunk
>>> > encapsulation dot1q switchport mode dynamic desirable no ip
>>>address
>>> > end
>>> >
>>> > This is the output when I type the "show int g0/1" command on
>>>the 3550
>>> >
>>> > Lab_3550_1#show int g0/1
>>> > GigabitEthernet0/1 is up, line protocol is up (connected)
>>> > Hardware is Gigabit Ethernet, address is 000d.bc49.44b1 (bia
>>> 000d.bc49.44b1)
>>> > Description: Trunk to main lab 5500 port 1/1
>>> > MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
>>> > reliability 255/255, txload 1/255, rxload 1/255
>>> > Encapsulation ARPA, loopback not set
>>> > Keepalive set (10 sec)
>>> > Full-duplex, 1000Mb/s, link type is auto, media type is SX
>>> > output flow-control is off, input flow-control is off
>>> > ARP type: ARPA, ARP Timeout 04:00:00
>>> > Last input 00:00:00, output 00:00:00, output hang never
>>> > Last clearing of "show interface" counters 01:52:54
>>> > Input queue: 0/75/0/0 (size/max/drops/flushes); Total output
>>>drops:
>>> 0
>>> > Queueing strategy: fifo
>>> > Output queue: 0/40 (size/max)
>>> > 5 minute input rate 1962000 bits/sec, 471 packets/sec
>>> > 5 minute output rate 1707000 bits/sec, 428 packets/sec
>>> > 2818750 packets input, 1023156797 bytes, 0 no buffer
>>> > Received 293096 broadcasts (0 multicast)
>>> > 0 runts, 0 giants, 0 throttles
>>> > 1 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored
>>> > 0 watchdog, 224737 multicast, 0 pause input
>>> > 0 input packets with dribble condition detected
>>> > 2635991 packets output, 1397912368 bytes, 0 underruns
>>> > 0 output errors, 0 collisions, 0 interface resets
>>> > 0 babbles, 0 late collision, 0 deferred
>>> > 0 lost carrier, 0 no carrier, 0 PAUSE output
>>> > 0 output buffer failures, 0 output buffers swapped out
>>> >
>>> > And finally, on the 3550, I "show int trunk"
>>> >
>>> > Lab_3550_1#show int trunk
>>> >
>>> > Port Mode Encapsulation Status Native
>>>vlan
>>> > Fa0/45 desirable isl trunking 44
>>> > Fa0/46 desirable isl trunking 44
>>> > Fa0/47 on isl trunking 41
>>> > Gi0/1 desirable 802.1q trunking 1
>>> > Gi0/2 desirable 802.1q trunking 41
>>> >
>>> > Port Vlans allowed on trunk
>>> > Fa0/45 1-4094
>>> > Fa0/46 1-4094
>>> > Fa0/47 1-4094
>>> > Gi0/1 1-4094
>>> > Gi0/2 1-4094
>>> >
>>> > Port Vlans allowed and active in management domain
>>> > Fa0/45
>>>
>>>
>>
> 1,10,12,20,25,28,30-33,35-41,44-45,48,52,54-55,60,130,341,351-352,361,36
>
>>> 9,400-406,419,995
>>> > Fa0/46
>>>
>>>
>>
> 1,10,12,20,25,28,30-33,35-41,44-45,48,52,54-55,60,130,341,351-352,361,36
>
>>> 9,400-406,419,995
>>> > Fa0/47
>>>
>>>
>>
> 1,10,12,20,25,28,30-33,35-41,44-45,48,52,54-55,60,130,341,351-352,361,36
>
>>> 9,400-406,419,995
>>> > Gi0/1
>>>
>>>
>>
> 1,10,12,20,25,28,30-33,35-41,44-45,48,52,54-55,60,130,341,351-352,361,36
>
>>> 9,400-406,419,995
>>> > Gi0/2
>>>
>>>
>>
> 1,10,12,20,25,28,30-33,35-41,44-45,48,52,54-55,60,130,341,351-352,361,36
>
>>> 9,400-406,419,995
>>> >
>>> > Port Vlans in spanning tree forwarding state and not
>>>pruned
>>> > Fa0/45 1,41,44,351,361,400
>>> > Fa0/46 1
>>> > Fa0/47
>>>
>>>
>>
> 1,10,12,20,25,28,30-33,35-41,44-45,48,52,54-55,60,130,341,351-352,361,36
>
>>> 9,400-406,419,995
>>> > Gi0/1
>>>
>>>
>>
> 1,10,12,20,25,28,30-33,35-41,44-45,48,52,54-55,60,130,341,351-352,361,36
>
>>> 9,400-406,419,995
>>> > Gi0/2 1,41,403-406
>>> >
>>> >
>>> >
>>> > On my 5500 I do the following:
>>> >
>>> > 5500-OMCLAB (enable) show port capabilities 1/1
>>> > Model WS-X5530
>>> > Port 1/1
>>> > Type 1000BaseSX
>>> > Speed 1000
>>> > Duplex full
>>> > Trunk encap type 802.1Q,ISL
>>> > Trunk mode on,off,desirable,auto,nonegotiate
>>> > Channel no
>>> > Broadcast suppression percentage(0-100)
>>> > Flow control
>>> receive-(off,on,desired),send-(off,on,desired)
>>> > Security no
>>> > Membership static
>>> > Fast start yes
>>> > QOS n/a
>>> > Rewrite no
>>> > UDLD Capable
>>> >
>>> >
>>> >
>>> >
>>> > 5500-OMCLAB (enable) show port 1/1
>>> > Port Name Status Vlan Level Duplex
>>>Speed
>>> Type
>>> > ----- ------------------ ---------- ---------- ------ ------
>>>-----
>>> ------------
>>> > 1/1 Trunk to lab_3550_ connected trunk normal full
>>>1000
>>> 1000BaseSX
>>> >
>>> > Port Trap IfIndex
>>> > ----- -------- -------
>>> > 1/1 disabled 224
>>> >
>>> > Port Broadcast-Limit Broadcast-Drop
>>> > -------- --------------- --------------
>>> > 1/1 - 0
>>> >
>>> > Port Send FlowControl Receive FlowControl RxPause
>>>TxPause
>>> Unsupported
>>> > admin oper admin oper
>>> opcodes
>>> > ----- -------- -------- -------- -------- -------
>>>-------
>>> -----------
>>> > 1/1 desired off off off 0 0
>>>0
>>> >
>>> >
>>> > Port Align-Err FCS-Err Xmit-Err Rcv-Err UnderSize
>>> > ----- ---------- ---------- ---------- ---------- ---------
>>> > 1/1 0 0 0 0 0
>>> >
>>> > Port Single-Col Multi-Coll Late-Coll Excess-Col Carri-Sen
>>>Runts
>>> Giants
>>> > ----- ---------- ---------- ---------- ---------- ---------
>>>---------
>>> ---------
>>> > 1/1 0 0 0 0 0
>>>0
>>> -
>>> >
>>> > Last-Time-Cleared
>>> > --------------------------
>>> > Thu Apr 15 2004, 10:04:41
>>> >
>>> >
>>> > 5500-OMCLAB (enable) show trunk
>>> > Port Mode Encapsulation Status Native
>>>vlan
>>> > -------- ----------- ------------- ------------
>>>-----------
>>> > 1/1 on dot1q trunking 1
>>> > 1/2 on dot1q trunking 1
>>> >
>>> >
>>>
> ______________________________________________________________________
>
>>> > _ Please help support GroupStudy by purchasing your study
>>>materials
>>> > from:
>>> > http://shop.groupstudy.com
>>> >
>>> > Subscription information may be found at:
>>> > http://www.groupstudy.com/list/CCIELab.html
>>> >
>>>
>>> --
>>> David Madland
>>> CCIE# 2016
>>> Sr. Network Engineer
>>> Qwest Communications
>>> 612-664-3367
>>>
>>> "Emotion should reflect reason not guide it"
>>>
>>>
>>>
>>
> _______________________________________________________________________
>
>>> Please help support GroupStudy by purchasing your study
>>>materials from:
>>> http://shop.groupstudy.com
>>>
>>> Subscription information may be found at:
>>> http://www.groupstudy.com/list/CCIELab.html
>>>
>>>
>>>
>>
> _______________________________________________________________________
>
>>> Please help support GroupStudy by purchasing your study
>>>materials from:
>>> http://shop.groupstudy.com
>>>
>>> Subscription information may be found at:
>>> http://www.groupstudy.com/list/CCIELab.html
>>>
>>>
>>>
>>
> _______________________________________________________________________
>
>>> Please help support GroupStudy by purchasing your study
>>>materials from:
>>> http://shop.groupstudy.com
>>>
>>> Subscription information may be found at:
>>> http://www.groupstudy.com/list/CCIELab.html
>>>
>>>
>>
> _______________________________________________________________________
>
>>>Please help support GroupStudy by purchasing your study materials
>>
>>from:
>>
>>>http://shop.groupstudy.com
>>>
>>>Subscription information may be found at:
>>>http://www.groupstudy.com/list/CCIELab.html
>>>
>>>
>>
> _______________________________________________________________________
>
>>>Please help support GroupStudy by purchasing your study materials
>>
>>from:
>>
>>>http://shop.groupstudy.com
>>>
>>>Subscription information may be found at:
>>>http://www.groupstudy.com/list/CCIELab.html
>>
>>
> _______________________________________________________________________
>
>>Please help support GroupStudy by purchasing your study materials
>
> from:
>
>>http://shop.groupstudy.com
>>
>>Subscription information may be found at:
>>http://www.groupstudy.com/list/CCIELab.html
>
>
> _______________________________________________________________________
> Please help support GroupStudy by purchasing your study materials from:
> http://shop.groupstudy.com
>
> Subscription information may be found at:
> http://www.groupstudy.com/list/CCIELab.html
>
-- David Madland CCIE# 2016 Sr. Network Engineer Qwest Communications 612-664-3367"Emotion should reflect reason not guide it"
This archive was generated by hypermail 2.1.4 : Mon May 03 2004 - 19:48:50 GMT-3