Thanks guys for the info... here's the config of my router scrubbed
for sensitive info.
Current configuration : 14524 bytes
!
version 12.4
service timestamps debug datetime
service timestamps log datetime localtime
service password-encryption
!
hostname !!!!
!
boot-start-marker
boot-end-marker
!
logging message-counter syslog
logging buffered 4096
enable secret 5
!
aaa new-model
!
!
aaa authentication login default group tacacs+ local
aaa authorization commands 15 default group tacacs+ if-authenticated
aaa accounting commands 15 default stop-only group tacacs+
!
!
aaa session-id common
clock timezone GMT 0
!
dot11 syslog
no ip source-route
ip wccp 61 redirect-list WAAS_PERMIT_ANY
ip wccp 62 redirect-list WAAS_PERMIT_ANY
!
!
ip cef
!
!
ip multicast-routing
!
no ipv6 cef
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
voice-card 0
no dspfarm
!
!
!
!
!
archive
log config
hidekeys
!
!
!
!
!
!
!
class-map match-all Class-GRE-AF31
match protocol gre
match dscp af31
class-map match-any Class-MissionCritical
match access-group name ACL-SAP
match access-group name ACL-Voice-Signaling
match access-group name ACL-Voice-Intercluster
match access-group name ACL-WWW
match access-group name ACL-Sametime
match class-map Class-GRE-AF31
class-map match-all Class-GRE-AF11
match protocol gre
match dscp af11
class-map match-all Class-GRE-BE
match protocol gre
match dscp default
class-map match-any Class-BusinessBatch
match access-group name ACL-Notes
match class-map Class-GRE-AF11
class-map match-all Class-GRE-AF21
match protocol gre
match dscp af21
class-map match-all Class-GRE-AF41
match protocol gre
match dscp af41
class-map match-all Class-GRE-EF
match protocol gre
match dscp ef
class-map match-any Class-Video
match access-group name ACL-Webcast
match class-map Class-GRE-AF41
class-map match-any Class-Voice
match access-group name ACL-Voice
match class-map Class-GRE-EF
class-map match-any Class-BusinessCritical
match access-group name ACL-Webcache
match class-map Class-GRE-AF21
class-map match-any Class-BestEffort
match any
match class-map Class-GRE-BE
!
!
policy-map ETM
class Class-Video
set dscp af41
bandwidth remaining percent 25
random-detect
class Class-MissionCritical
set dscp af31
bandwidth remaining percent 40
random-detect
class Class-BusinessCritical
set dscp af21
bandwidth remaining percent 20
random-detect
class Class-BusinessBatch
set dscp af11
bandwidth remaining percent 10
random-detect
class Class-BestEffort
set dscp default
bandwidth remaining percent 4
random-detect
class Class-Voice
set ip dscp ef
priority 8
!
!
!
!
!
interface Loopback0
ip address 192.168.133.21 255.255.255.255
!
interface Multilink1
bandwidth 4096
ip address
ip wccp 62 redirect in
ip flow ingress
ip flow egress
no peer neighbor-route
ppp chap hostname
ppp multilink
ppp multilink links minimum 1
ppp multilink group 1
ppp multilink fragment disable
service-policy output ETM
!
interface FastEthernet0/0
ip address 192.168.70.161 255.255.255.252 secondary
ip address 10.160.1.3 255.255.255.0
ip wccp 61 redirect in
ip pim sparse-dense-mode
ip cgmp
duplex full
speed 100
standby 1 ip 10.160.1.2
standby 1 timers 5 15
standby 1 priority 105
standby 1 preempt
standby 1 track Multilink1
!
interface FastEthernet0/1
no ip address
shutdown
duplex auto
speed auto
!
interface Serial0/0/0
no ip address
encapsulation ppp
ppp chap hostname
ppp multilink
ppp multilink group 1
!
interface Serial0/0/1
no ip address
encapsulation ppp
ppp chap hostname
ppp multilink
ppp multilink group 1
!
interface Integrated-Service-Engine1/0
description WAAS
ip address 10.160.32.1 255.255.255.0
service-module ip address 10.160.32.2 255.255.255.0
service-module ip default-gateway 10.160.32.1
no keepalive
!
router bgp 64606
no synchronization
bgp log-neighbor-changes
redistribute connected
redistribute static
neighbor
no auto-summary
!
ip forward-protocol nd
ip route 10.160.0.0 255.255.255.0 10.160.1.1
ip route 10.160.1.0 255.255.255.0 10.160.1.1
ip route 10.160.2.0 255.255.255.0 10.160.1.1
ip route 10.160.3.0 255.255.255.0 10.160.1.1
ip route 10.160.4.0 255.255.255.0 10.160.1.1
ip route 10.160.5.0 255.255.255.0 10.160.1.1
ip route 10.160.6.0 255.255.255.0 10.160.1.1
ip route 10.160.7.0 255.255.255.0 10.160.1.1
ip route 10.160.8.0 255.255.255.0 10.160.1.1
ip route 10.160.9.0 255.255.255.0 10.160.1.1
ip route 10.160.10.0 255.255.255.0 10.160.1.1
ip route 10.160.11.0 255.255.255.0 10.160.1.1
ip route 10.160.12.0 255.255.255.0 10.160.1.1
ip route 10.160.13.0 255.255.255.0 10.160.1.1
ip route 10.160.14.0 255.255.255.0 10.160.1.1
ip route 10.160.15.0 255.255.255.0 10.160.1.1
ip route 10.160.16.0 255.255.248.0 10.160.1.1
ip route 10.160.16.0 255.255.255.0 10.160.1.1
ip route 10.160.17.0 255.255.255.0 10.160.1.1
ip route 10.160.18.0 255.255.255.0 10.160.1.1
ip route 10.160.19.0 255.255.255.0 10.160.1.1
ip route 10.160.20.0 255.255.255.0 10.160.1.1
ip route 10.160.21.0 255.255.255.0 10.160.1.1
ip route 10.160.22.0 255.255.255.0 10.160.1.1
ip route 10.160.23.0 255.255.255.0 10.160.1.1
ip route 10.160.24.0 255.255.255.0 10.160.1.1
ip route 10.226.0.0 255.255.0.0 10.160.1.1
ip route 10.231.2.0 255.255.255.0 10.160.1.1
ip route 10.232.4.0 255.255.255.0 10.160.1.1
ip route 172.16.48.0 255.255.255.0 10.160.1.1
ip route 172.17.16.0 255.255.255.0 10.160.1.1
ip route 172.18.24.0 255.255.255.0 10.160.1.1
ip route 196.34.143.0 255.255.255.0 10.160.1.1
ip route 196.35.69.8 255.255.255.248 10.160.1.1
ip route 196.37.50.0 255.255.255.0 10.160.1.1
no ip http server
no ip http secure-server
!
!
ip pim rp-address
!
ip access-list extended ACL-Notes
remark Lotus Notes
permit tcp any eq 1352 any
permit tcp any any eq 1352
ip access-list extended ACL-SAP
remark SAP uses these range of tcp ports
remark Created from doc "TCP/IP Ports Used by SAP Applications"
permit tcp any range 3200 3399 any
permit tcp any range 3600 3699 any
permit tcp any range 8000 8079 any
permit tcp any range 8081 8599 any
permit tcp any range 30000 39904 any
permit tcp any range 50000 59910 any
permit tcp any any range 3200 3399
permit tcp any any range 3600 3699
permit tcp any any range 8000 8079
permit tcp any any range 8081 8599
permit tcp any any range 30000 39904
permit tcp any any range 50000 59910
ip access-list extended ACL-Sametime
remark Lotus Sametime
permit tcp any eq 1533 any
permit tcp any any eq 1533
ip access-list extended ACL-Voice
remark VoIP traffic
permit udp any range 16384 32767 any range 16384 32767
ip access-list extended ACL-Voice-Intercluster
permit tcp any any eq 1090
permit tcp any eq 1090 any
permit tcp any any eq 1099
permit tcp any eq 1099 any
permit tcp any any range 1500 1501
permit tcp any range 1500 1501 any
permit tcp any any eq 1515
permit tcp any eq 1515 any
permit tcp any any range 2551 2556
permit tcp any range 2551 2556 any
permit tcp any any range 8000 8010
permit tcp any range 8000 8010 any
ip access-list extended ACL-Voice-Signaling
remark VoIP signaling traffic
permit udp any any range 1719 1720
permit udp any range 1719 1720 any
permit tcp any any eq 2000
permit tcp any eq 2000 any
permit udp any any eq 2427
permit udp any eq 2427 any
permit tcp any any eq 2428
permit tcp any eq 2428 any
permit tcp any any range 5060 5061
permit tcp any range 5060 5061 any
permit udp any any range 5060 5061
permit udp any range 5060 5061 any
permit tcp any any eq 2748
permit tcp any eq 2748 any
permit tcp any any eq 2789
permit tcp any eq 2789 any
permit udp any any range 3223 3224
permit udp any range 3223 3224 any
permit udp any any eq 4321
permit udp any eq 4321 any
ip access-list extended ACL-WWW
remark Web port 80 traffic
permit tcp any eq www any
permit tcp any any eq www
remark HTTPS
permit tcp any eq 443 any
permit tcp any any eq 443
ip access-list extended ACL-Webcache
remark Webcache traffic
permit tcp any eq 8080 any
permit tcp any any eq 8080
ip access-list extended ACL-Webcast
remark Stratosfour webcasts use these udp multicasts
permit udp any 239.192.0.0 0.0.255.255
ip access-list extended DENY
deny ip any any
ip access-list extended WAAS
remark **1
remark **2 Webcache, Notes, WWW, SAP
remark **3
permit tcp any eq 8080 any
permit tcp any any eq 8080
permit tcp any eq 1352 any
permit tcp any any eq 1352
permit tcp any eq www any
permit tcp any any eq www
permit tcp any range 3200 3399 any
permit tcp any range 3600 3699 any
permit tcp any range 8000 8079 any
permit tcp any range 8081 8599 any
permit tcp any range 30000 39904 any
permit tcp any range 50000 59910 any
permit tcp any any range 3200 3399
permit tcp any any range 3600 3699
permit tcp any any range 8000 8079
permit tcp any any range 8081 8599
permit tcp any any range 30000 39904
permit tcp any any range 50000 59910
ip access-list extended WAAS_PERMIT_ANY
permit ip any any
ip access-list extended any
permit ip any any
!
ip sla responder
access-list 10 permit 167.228.170.200
access-list 10 permit 167.228.1.48
access-list 10 permit 167.228.1.55
access-list 10 permit 209.114.69.96
access-list 10 permit 167.228.2.231
access-list 10 permit 167.228.2.230
access-list 10 permit 167.228.2.232
access-list 10 permit 167.228.248.13
access-list 10 permit 209.114.69.95
access-list 51 permit 159.24.0.0 0.0.255.255
access-list 51 permit 205.223.96.0 0.0.15.255
access-list 51 permit 170.127.71.128 0.0.0.127
access-list 51 permit 146.170.64.0 0.0.7.255
access-list 51 permit 146.170.100.0 0.0.3.255
access-list 51 permit 146.170.124.0 0.0.3.255
!
!
!
!
!
control-plane
!
!
!
!
!
!
!
!
!
line con 0
exec-timeout 5 0
line aux 0
exec-timeout 5 0
line 66
no activation-character
no exec
transport preferred none
transport input all
transport output pad telnet rlogin lapb-ta mop udptn v120 ssh
line vty 0 4
exec-timeout 30 0
transport input telnet
!
scheduler allocate 20000 1000
end
On Sat, May 1, 2010 at 7:28 PM, Kambiz Agahian <kagahian_at_ccbootcamp.com> wrote:
> Carlos,
>
> My typo. I was thinking of VoIP "control" ports (e.g. Skinny TCP 2000)
> and not RTP, I worded that as UDP...actually the 61/62 thing works in
> the "TCP-promiscuous" mode. That's a part of its name :) - Anyway as the
> best practice the forwarding decision should be inclusive and not
> exclusive. So you pick only the things that you really need.
>
> But let's go back to the problem:
>
> 1- We need to verify CEF. This can be done through the commands I asked
> for.
> 2- We need to make sure that WCCP is configure properly. That's why I
> asked for the configuration of the "integrated" interface.
> 3- We however still not quite sure whether any optimization is achieved
> or not. The previous item has to be verified first.
> 4- Read through the output provided by the original poster you see this
> "Total Packets s/w Redirected", I know it's a bit strange but the "s/w"
> keyword stands for "Software"! if you configure hardware switching (with
> the latest version of the IOS is possible to some extend) then that
> counter should not be going up at all. So we need to know a bit of
> config of the NME module as well.
>
> As soon as he posts some more details we can look into the issue but
> generally speaking the following items are among the most important CPU
> spike cases in WCCP installations:
>
> 1- Bad WCCP configuration/Design (e.g. wrong interface, outbound
> interception etc.)
> 2- CEF issues
> 3- Software switching and GRE overhead. The L2 redirection is a nice
> remedy.
> 4- Bugs
>
> Needless to say, all these things are case-based. So let's wait and see
> what he says.
>
> HTH
>
> --------------------------
> Kambiz Agahian
> CCIE (R&S), CCSI, WAASSE, RSSSE
> Technical Instructor
> CCBOOTCAMP - Cisco Learning Solutions Partner (CLSP)
> Email: kagahian_at_ccbootcamp.com
> Toll Free: 877-654-2243
> International: +1-702-968-5100
> Skype: skype:ccbootcamp?call
> FAX: +1-702-446-8012
> YES! We take Cisco Learning Credits!
> Training And Remote Racks: http://www.ccbootcamp.com
>
> -----Original Message-----
> From: Carlos G Mendioroz [mailto:tron_at_huapi.ba.ar]
> Sent: Saturday, May 01, 2010 6:02 AM
> To: Kambiz Agahian
> Cc: Group Study; Cisco certification
> Subject: Re: WCCP and WAE question regarding high cpu utilization.
>
> Kambiz,
> on #1, wccp group 61/62 only redirect TCP traffic, so RTP is not going
> to get caught AFAIK, and though SIP can run over TCP, I would usually
> associate "voip traffic" to RTP.
>
> General question: can this be done in CEF at all ? I.e. TCP redirection.
> I know cisco says to enable CEF in the routers, etc.
> But isn't CEF a fast destination IP controlled thing ? So if the router
> has to differentiate TCP from the rest, it would have to punt the
> traffic.
>
> Enter the architecture dependent world. Some chasis do have extensions
> to deal with L4 on hardware for things like QoS and ACLs, but here we
> need differentiated forwarding based on L4. I'm suspicious that this is
> a no no for an ISR.
>
> Thoughts ?
> -Carlos
>
> Kambiz Agahian @ 1/05/2010 6:54 -0300 dixit:
>> Hi there,
>>
>> Your platform is one of the suitable/recommended models for WCCP
>> redirection to/from NME's but here are my suggestions:
>>
>> 1- Never ever use a permit any ACL with Cisco WAAS - you certainly
> don't
>> want to kick your voip traffic over to the WAAS module. First off, you
>> need a TCP only ACL but in this case start with a simple ACL just to
>> pick some "interesting" traffic. HTTP is usually a good choice.
>>
>> 2- I need more info to troubleshoot this. The output of the "show ip
> int
>> xxx" and "show int xxx" commands is obviously necessary. If you're not
>> limited by confidentiality policies I also need a full config of the
>> router, for instance I need to know what your QoS policy is doing.
>>
>> 3- If you have any, please take all the CEF killers off the config,
> I'm
>> quite sure you're aware of this, but do get rid of things like ACLs
> with
>> the log option enabled.
>>
>> * If you're not comfortable with posting more details here feel free
> to
>> contact me off-list or as a best practice open a case with Cisco TAC.
>>
>>
>> HTH
>>
>> --------------------------
>> Kambiz Agahian
>> CCIE (R&S), CCSI, WAASSE, RSSSE
>> Technical Instructor
>> CCBOOTCAMP - Cisco Learning Solutions Partner (CLSP)
>> Email: kagahian_at_ccbootcamp.com
>> Toll Free: 877-654-2243
>> International: +1-702-968-5100
>> Skype: skype:ccbootcamp?call
>> FAX: +1-702-446-8012
>> YES! We take Cisco Learning Credits!
>> Training And Remote Racks: http://www.ccbootcamp.com
>>
>>
>>
>>
>> -----Original Message-----
>> From: nobody_at_groupstudy.com [mailto:nobody_at_groupstudy.com] On Behalf
> Of
>> Group Study
>> Sent: Friday, April 30, 2010 1:43 PM
>> To: Cisco certification
>> Subject: WCCP and WAE question regarding high cpu utilization.
>>
>> I'm using a nme-wae card in an ISR router and using wccp to redirect
>> traffic, all traffic.
>>
>> I notice that the CPU utilization gets to 100% and when I do a "show
>> ip wccp" i notice CEF switched packets are zero and process switched
>> packets are many, leading me to believe that's the reason for high
>> cpu...
>>
>> Any suggestions on how to fix this?
>>
>> Global WCCP information:
>> Router information:
>> Router Identifier: 192.168.133.21
>> Protocol Version: 2.0
>>
>> Service Identifier: 61
>> Number of Service Group Clients: 1
>> Number of Service Group Routers: 1
>> Total Packets s/w Redirected: 141892759
>> Process: 141892759
>> CEF: 0
>> Service mode: Open
>> Service Access-list: -none-
>> Total Packets Dropped Closed: 0
>> Redirect Access-list: WAAS_PERMIT_ANY
>> Total Packets Denied Redirect: 0
>> Total Packets Unassigned: 7881
>> Group Access-list: -none-
>> Total Messages Denied to Group: 0
>> Total Authentication failures: 0
>> Total Bypassed Packets Received: 784
>>
>> Service Identifier: 62
>> Number of Service Group Clients: 1
>> Number of Service Group Routers: 1
>> Total Packets s/w Redirected: 138317602
>> Process: 137859756
>> CEF: 457846
>> Service mode: Open
>> Service Access-list: -none-
>> Total Packets Dropped Closed: 0
>> Redirect Access-list: WAAS_PERMIT_ANY
>> Total Packets Denied Redirect: 168333787
>> Total Packets Unassigned: 9094
>> Group Access-list: -none-
>> Total Messages Denied to Group: 0
>> Total Authentication failures: 0
>> Total Bypassed Packets Received: 732
>>
>>
>>
>> !
>> interface FastEthernet0/0
>> ip address 192.168.70.161 255.255.255.252 secondary
>> ip address 10.160.1.3 255.255.255.0
>> ip wccp 61 redirect in
>> ip pim sparse-dense-mode
>> ip cgmp
>> duplex full
>> speed 100
>> standby 1 ip 10.160.1.2
>> standby 1 timers 5 15
>> standby 1 priority 105
>> standby 1 preempt
>> standby 1 track Multilink1
>> end
>>
>> interface Multilink1
>> description
>> bandwidth 4096
>>
>> ip wccp 62 redirect in
>> ip flow ingress
>> ip flow egress
>> no peer neighbor-route
>> ppp chap hostname abc
>> ppp multilink
>> ppp multilink links minimum 1
>> ppp multilink group 1
>> ppp multilink fragment disable
>> service-policy output QOS
>> end
>>
>>
>> Blogs and organic groups at http://www.ccie.net
>>
>>
> _______________________________________________________________________
>> Subscription information may be found at:
>> http://www.groupstudy.com/list/CCIELab.html
>>
>>
>> Blogs and organic groups at http://www.ccie.net
>>
>>
> _______________________________________________________________________
>> Subscription information may be found at:
>> http://www.groupstudy.com/list/CCIELab.html
>>
>>
>>
>>
>>
>>
>>
>
> --
> Carlos G Mendioroz <tron_at_huapi.ba.ar> LW7 EQI Argentina
Blogs and organic groups at http://www.ccie.net
Received on Mon May 03 2010 - 10:05:06 ART
This archive was generated by hypermail 2.2.0 : Tue Jun 01 2010 - 07:09:52 ART