High CPU process on 3560G-48PS Switch

From: Irfan Sid <lifeoverip_at_gmail.com>
Date: Wed, 10 Aug 2011 10:30:52 +0300

*I am getting 20% constant cpu untiliztion on my access switch 3560G-48PS.
The processes which are taking up cpu processing are in the show output
below. *

*the Hulc LED Process is taking 13%. Does anyone knwo what this is? Also the
Fifo Error detection is also one of concern.*

*Can someone please look at the below and advise, if this is normal or any
anomalies here?*

SW2#show processes cpu | e 0.00%

CPU utilization for five seconds: 22%/0%; one minute: 23%; five minutes: 22%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

   4 2129937 372942 5711 0.47% 0.06% 0.05% 0 Check heaps

  48 39773744 166915955 238 2.39% 2.86% 2.90% 0 Fifo Error
Detec

  81 19405517 3589252 5406 0.47% 0.63% 0.62% 0 hpm counter
proc

*109 570200132 75644467 7537 13.09% 13.19% 13.22% 0 Hulc LED
Process*

117 12157268 724020 16791 0.31% 0.31% 0.31% 0 HQM Stack
Proces

165 7666982 39552706 193 0.15% 0.20% 0.16% 0 Spanning Tree

191 550559 8014478 68 0.15% 0.06% 0.02% 0 Marvell wk-a
Pow

228 224 185 1210 0.15% 0.03% 0.01% 1 Virtual Exec

GPT_16F_SW2#

SW2#show processes cpu history

    2222222222222222222222222222222222222222222222222222222222
    2233333333332222222222222222222222222444442222233332222233
100
90
80
70
60
50
40
30
20 **********************************************************
10 **********************************************************
   0....5....1....1....2....2....3....3....4....4....5....5....
             0 5 0 5 0 5 0 5 0 5
               CPU% per second (last 60 seconds)

    2222222222222222222222222222222222222222222222222222222222
    4434334433345344333333344344333434333332334233233334333333
100
90
80
70
60
50
40
30 *
20 ##########################################################
10 ##########################################################
   0....5....1....1....2....2....3....3....4....4....5....5....
             0 5 0 5 0 5 0 5 0 5
               CPU% per minute (last 60 minutes)
              * = maximum CPU% # = average CPU%

    2222223222222332222222222222222222222242242222222222222222222222222422
    5658990988987008889788887998888776655933323433343333333433333333334533
100
90
80
70
60
50 *
40 * * *
30 *************************************** * *
20 ######################################################################
10 ######################################################################
   0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
             0 5 0 5 0 5 0 5 0 5 0 5 0
                   CPU% per hour (last 72 hours)
                  * = maximum CPU% # = average CPU%

SW2#

SW2#show processes cpu

CPU utilization for five seconds: 23%/0%; one minute: 23%; five minutes: 22%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

   1 0 7 0 0.00% 0.00% 0.00% 0 Chunk Manager

   2 808 726150 1 0.00% 0.00% 0.00% 0 Load Meter

   3 675 1426 473 0.00% 0.00% 0.00% 0 SpanTree
Helper

   4 2129870 372930 5711 0.47% 0.07% 0.05% 0 Check heaps

   5 290 742 390 0.00% 0.00% 0.00% 0 Pool Manager

   6 0 2 0 0.00% 0.00% 0.00% 0 Timers

   7 0 2 0 0.00% 0.00% 0.00% 0 Net Input

   8 0 1 0 0.00% 0.00% 0.00% 0 Crash writer

   9 1840 65864 27 0.00% 0.00% 0.00% 0 ARP Input

  10 0 1 0 0.00% 0.00% 0.00% 0
AAA_SERVER_DEADT

  11 0 2 0 0.00% 0.00% 0.00% 0 AAA
high-capacit

  12 0 1 0 0.00% 0.00% 0.00% 0 Policy
Manager

  13 34 3 11333 0.00% 0.00% 0.00% 0 Entity MIB
API

  14 0 1 0 0.00% 0.00% 0.00% 0 IFS Agent
Manage

  15 358 60605 5 0.00% 0.00% 0.00% 0 IPC Dynamic
Cach

  16 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone
Manager

  17 189566 3590141 52 0.00% 0.00% 0.00% 0 IPC Periodic
Tim

  18 0 1 0 0.00% 0.00% 0.00% 0 IPC Managed
Time

  19 141529 3590094 39 0.00% 0.00% 0.00% 0 IPC Deferred
Por

  20 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat
Manager

  21 0 1 0 0.00% 0.00% 0.00% 0 IPC Session
Serv

  22 68887 1085152 63 0.00% 0.00% 0.00% 0 HC Counter
Timer

  23 160205 3590143 44 0.15% 0.01% 0.00% 0 Dynamic ARP
Insp

  24 0 1 0 0.00% 0.00% 0.00% 0 ARP Snoop

  25 142485 3590113 39 0.00% 0.00% 0.00% 0 GraphIt

  26 0 2 0 0.00% 0.00% 0.00% 0 XML Proxy
Client

  27 0 1 0 0.00% 0.00% 0.00% 0 Critical
Bkgnd

  28 476338 5185632 91 0.00% 0.03% 0.04% 0 Net
Background

  29 0 1 0 0.00% 0.00% 0.00% 0 IDB Work

  30 417 58678 7 0.00% 0.00% 0.00% 0 Logger

  31 162772 3590114 45 0.15% 0.01% 0.00% 0 TTY
Background

  32 579291 3590518 161 0.00% 0.00% 0.00% 0 Per-Second
Jobs

  33 637722 61335 10397 0.00% 0.01% 0.00% 0 Per-minute
Jobs

  34 0 7 0 0.00% 0.00% 0.00% 0 AggMgr
Process

  35 481628 724011 665 0.00% 0.00% 0.00% 0 Compute load
avg

  36 0 3 0 0.00% 0.00% 0.00% 0 Collection
proce

  37 32076 72006566 0 0.00% 0.00% 0.00% 0
DownWhenLooped

  38 0 1 0 0.00% 0.00% 0.00% 0 HRPC lpip
reques

  39 0 2 0 0.00% 0.00% 0.00% 0 HLPIP Sync
Proce

  40 6410 723983 8 0.00% 0.00% 0.00% 0 Hulc LED
Alchemy

  41 1648003 75460772 21 0.00% 0.07% 0.06% 0 RedEarth Tx
Mana

  42 214914 142425297 1 0.00% 0.03% 0.00% 0 RedEarth Rx
Mana

  43 170 24243 7 0.00% 0.00% 0.00% 0 HULC Thermal
Pro

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

  44 0 1 0 0.00% 0.00% 0.00% 0 HRPC
asic-stats

  45 0 1 0 0.00% 0.00% 0.00% 0 HRPC hsm
request

  46 0 7 0 0.00% 0.00% 0.00% 0 Stack Mgr

  47 210 6 35000 0.00% 0.00% 0.00% 0 Stack Mgr
Notifi

  48 39770016 166910344 238 2.87% 3.00% 2.92% 0 Fifo Error
Detec

  49 3991 427 9346 0.00% 0.00% 0.00% 0 Adjust
Regions

  50 486364 3590187 135 0.00% 0.00% 0.00% 0 hrpc ->
response

  51 40774 723997 56 0.00% 0.00% 0.00% 0 hrpc ->
request

  52 50878 723998 70 0.00% 0.00% 0.00% 0 hrpc <-
response

  53 0 1 0 0.00% 0.00% 0.00% 0 HRPC hcomp
reque

  54 0 3 0 0.00% 0.00% 0.00% 0 HULC Device
Mana

  55 0 3 0 0.00% 0.00% 0.00% 0 HRPC hdm non
blo

  56 0 2 0 0.00% 0.00% 0.00% 0 HRPC hdm
blockin

  57 7399 723982 10 0.00% 0.00% 0.00% 0 HIPC bkgrd
proce

  58 2426 194391 12 0.00% 0.00% 0.00% 0 Hulc
Port-Securi

  59 0 1 0 0.00% 0.00% 0.00% 0 HRPC hpsecure
re

  60 0 1 0 0.00% 0.00% 0.00% 0 HRPC hlfm
reques

  61 455265 106169670 4 0.00% 0.00% 0.00% 0 HLFM address
lea

  62 760871 3590193 211 0.00% 0.00% 0.00% 0 HLFM aging
proce

  63 82554 106266314 0 0.00% 0.00% 0.00% 0 HLFM address
ret

  64 0 1 0 0.00% 0.00% 0.00% 0 HRPC hrcmd
reque

  65 0 1 0 0.00% 0.00% 0.00% 0 HRPC hulc
misc r

  66 0 1 0 0.00% 0.00% 0.00% 0 HRPC system
mtu

  67 22670 1210254 18 0.00% 0.00% 0.00% 0 HVLAN main
bkgrd

  68 0 2 0 0.00% 0.00% 0.00% 0 HVLAN Mapped
Vla

  69 0 2 0 0.00% 0.00% 0.00% 0 Vlan shutdown
Pr

  70 0 1 0 0.00% 0.00% 0.00% 0 HRPC vlan
reques

  71 0 1 0 0.00% 0.00% 0.00% 0 HULC VLAN REF
Ba

  72 0 1 0 0.00% 0.00% 0.00% 0 HRPC hfbm
reques

  73 168 20203 8 0.00% 0.00% 0.00% 0 HCMP sync
proces

  74 0 1 0 0.00% 0.00% 0.00% 0 HRPC ilp
request

  75 753470 3590160 209 0.00% 0.01% 0.00% 0 Hulc ILP
Alchemy

  76 0 1 0 0.00% 0.00% 0.00% 0 HPM Msg Retry
Pr

  77 539 30303 17 0.00% 0.00% 0.00% 0 DHCPD Timer

  78 365635 29588942 12 0.00% 0.00% 0.00% 0 hpm main
process

  79 60 2992 20 0.00% 0.00% 0.00% 0 HPM Stack
Sync P

  80 0 1 0 0.00% 0.00% 0.00% 0 HRPC pm
request

  81 19404733 3589131 5406 0.47% 0.63% 0.62% 0 hpm counter
proc

  82 0 1 0 0.00% 0.00% 0.00% 0 HRPC
pm-counters

  83 0 1 0 0.00% 0.00% 0.00% 0 hpm vp events
ca

  84 0 1 0 0.00% 0.00% 0.00% 0 HRPC hcmp
reques

  85 26 1736 14 0.00% 0.00% 0.00% 0 HCEF ADJ
Refresh

  86 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl3mm
reque

  87 0 1 0 0.00% 0.00% 0.00% 0
hl3md_rpfq_thrl_

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

  88 42116 2528881 16 0.00% 0.00% 0.00% 0 hl3mm

  89 0 1 0 0.00% 0.00% 0.00% 0 HACL Queue
Proce

  90 0 1 0 0.00% 0.00% 0.00% 0 HRPC acl
request

  91 0 17 0 0.00% 0.00% 0.00% 0 HACL Acl
Manager

  92 0 1 0 0.00% 0.00% 0.00% 0 HRPC backup
inte

  93 0 1 0 0.00% 0.00% 0.00% 0 SNMP Timers

  94 0 1 0 0.00% 0.00% 0.00% 0 HRPC cdp
request

  95 0 1 0 0.00% 0.00% 0.00% 0 HRPC dot1x
reque

  96 0 4 0 0.00% 0.00% 0.00% 0 HULC DOT1X
Proce

  97 0 1 0 0.00% 0.00% 0.00% 0 HRPC sdm
request

  98 155197 17946011 8 0.00% 0.00% 0.00% 0 Hulc Storm
Contr

  99 0 2 0 0.00% 0.00% 0.00% 0 HSTP Sync
Proces

100 0 1 0 0.00% 0.00% 0.00% 0 HRPC stp_cli
req

101 0 1 0 0.00% 0.00% 0.00% 0 HRPC
stp_state_s

102 0 2 0 0.00% 0.00% 0.00% 0 S/W Bridge
Proce

103 0 1 0 0.00% 0.00% 0.00% 0 HRPC hudld
reque

104 0 1 0 0.00% 0.00% 0.00% 0 HRPC vqpc
reques

105 0 1 0 0.00% 0.00% 0.00% 0 BCM cable
diags

106 0 1 0 0.00% 0.00% 0.00% 0 HRPC
iec_load_ba

107 0 1 0 0.00% 0.00% 0.00% 0 HRPC l2pt qnq
rp

108 0 1 0 0.00% 0.00% 0.00% 0 HRPC hled
reques

109 570183116 75641769 7537 13.25% 13.25% 13.22% 0 Hulc LED
Process

110 187814 2656078 70 0.00% 0.00% 0.00% 0 HL3U bkgrd
proce

111 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl3u
reques

112 10171 830073 12 0.00% 0.00% 0.00% 0 HL3U PBR bkgrd
p

113 24 30302 0 0.00% 0.00% 0.00% 0 HL3U PBR n-h
res

114 0 1 0 0.00% 0.00% 0.00% 0 HRPC dtp
request

115 0 1 0 0.00% 0.00% 0.00% 0 HRPC
show_forwar

116 0 1 0 0.00% 0.00% 0.00% 0 HRPC snmp
reques

117 12156867 723996 16791 0.31% 0.31% 0.31% 0 HQM Stack
Proces

118 24305 1447975 16 0.00% 0.00% 0.00% 0 HRPC qos
request

119 0 1 0 0.00% 0.00% 0.00% 0 HRPC span
reques

120 0 1 0 0.00% 0.00% 0.00% 0 HRPC system
post

121 0 1 0 0.00% 0.00% 0.00% 0 Hulc Reload
Mana

122 0 1 0 0.00% 0.00% 0.00% 0 HRPC
hrcli-event

123 9956 1030066 9 0.00% 0.00% 0.00% 0 DHCPD Database

124 0 2 0 0.00% 0.00% 0.00% 0 image mgr

125 0 3 0 0.00% 0.00% 0.00% 0 HL2MCM

126 0 3 0 0.00% 0.00% 0.00% 0 HL2MCM

127 0 2 0 0.00% 0.00% 0.00% 0 EAPoUDP
Process

128 0 3 0 0.00% 0.00% 0.00% 0 CEF switching
ba

129 1845115 3590286 513 0.00% 0.00% 0.00% 0 PI MATM Aging
Pr

130 34 1943 17 0.00% 0.00% 0.00% 0 Switch Backup
In

131 223 60604 3 0.00% 0.00% 0.00% 0 MMN bkgrd
proces

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

132 0 2 0 0.00% 0.00% 0.00% 0 Dot1x Mgr
Proces

133 0 1 0 0.00% 0.00% 0.00% 0 MAB Framework

134 24 2501 9 0.00% 0.00% 0.00% 0 802.1x switch

135 438 60604 7 0.00% 0.00% 0.00% 0 802.1x MDA
Aging

136 0 1 0 0.00% 0.00% 0.00% 0 802.1x Webauth
F

137 0 1 0 0.00% 0.00% 0.00% 0 802.1x
Critical

138 49491 605579 81 0.00% 0.00% 0.00% 0 DTP Protocol

139 0 1 0 0.00% 0.00% 0.00% 0 EAP Framework

140 0 1 0 0.00% 0.00% 0.00% 0 HRPC dai
request

141 0 1 0 0.00% 0.00% 0.00% 0 HULC DAI
Process

142 0 1 0 0.00% 0.00% 0.00% 0 HRPC dhcp
snoopi

143 0 4 0 0.00% 0.00% 0.00% 0 HULC DHCP
Snoopi

144 0 1 0 0.00% 0.00% 0.00% 0 HRPC ip source
g

145 0 1 0 0.00% 0.00% 0.00% 0 HULC IP Source
g

146 46833 3636501 12 0.00% 0.00% 0.00% 0 UDLD

147 15845 206606 76 0.00% 0.00% 0.00% 0 Port-Security

148 114549 3590135 31 0.00% 0.00% 0.00% 0 MDFS LC
Download

149 0 2 0 0.00% 0.00% 0.00% 0 Switch IP Host
T

150 0 1 0 0.00% 0.00% 0.00% 0 Link State
Group

151 2670 363385 7 0.00% 0.00% 0.00% 0 Ethchnl

152 419 1754 238 0.00% 0.00% 0.00% 0 VMATM Callback

153 40924 36011983 1 0.00% 0.00% 0.00% 0 MDFS MFIB
Proces

154 0 1 0 0.00% 0.00% 0.00% 0 IPv6 RIB
Redistr

155 0 2 0 0.00% 0.00% 0.00% 0 AAA Server

156 0 1 0 0.00% 0.00% 0.00% 0 AAA ACCT Proc

157 0 1 0 0.00% 0.00% 0.00% 0 ACCT Periodic
Pr

158 270377 1750003 154 0.00% 0.00% 0.00% 0 CDP Protocol

159 24 318 75 0.00% 0.00% 0.00% 0 SpanTree Flush

160 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl2mcm
igmp

161 0 2 0 0.00% 0.00% 0.00% 0 AAA Dictionary
R

162 736 30309 24 0.00% 0.00% 0.00% 0 DHCP Snooping

163 13645 188108 72 0.00% 0.01% 0.00% 0 IP Input

164 0 1 0 0.00% 0.00% 0.00% 0 ICMP event
handl

165 7666689 39551709 193 0.00% 0.14% 0.14% 0 Spanning Tree

166 364 60607 6 0.00% 0.00% 0.00% 0 Spanning Tree
St

167 6060 72734 83 0.00% 0.00% 0.00% 0 CEF background
p

168 0 1 0 0.00% 0.00% 0.00% 0 IP IRDP

169 0 1 0 0.00% 0.00% 0.00% 0 CEF RF HULC
Conv

170 0 2 0 0.00% 0.00% 0.00% 0 XDR mcast

171 0 1 0 0.00% 0.00% 0.00% 0 IPC LC Message
H

172 0 1 0 0.00% 0.00% 0.00% 0 XDR RP Ping
Back

173 380 30303 12 0.00% 0.00% 0.00% 0 XDR RP
backgroun

174 0 1 0 0.00% 0.00% 0.00% 0 XDR RP Test
Back

175 0 3 0 0.00% 0.00% 0.00% 0 MDFS LC
Process

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

176 128 45457 2 0.00% 0.00% 0.00% 0 Cluster L2

177 4973 363383 13 0.00% 0.00% 0.00% 0 Cluster RARP

178 24425 575477 42 0.00% 0.00% 0.00% 0 Cluster Base

179 295 12734 23 0.00% 0.00% 0.00% 0 TCP Timer

180 27 61 442 0.00% 0.00% 0.00% 0 TCP Protocols

181 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers

182 36 12127 2 0.00% 0.00% 0.00% 0 HTTP CORE

183 0 1 0 0.00% 0.00% 0.00% 0 RARP Input

184 1635 2716 601 0.00% 0.00% 0.00% 0 IGMPSN L2MCM

185 0 1 0 0.00% 0.00% 0.00% 0 IGMPSN MRD

186 527 2696 195 0.00% 0.00% 0.00% 0 IGMPSN

187 0 1 0 0.00% 0.00% 0.00% 0 IGMPQR

188 0 1 0 0.00% 0.00% 0.00% 0 CEF MQC IPC
Back

189 0 2 0 0.00% 0.00% 0.00% 0 L2TRACE SERVER

190 224248 4373808 51 0.00% 0.00% 0.00% 0 Inline Power

191 550475 8014209 68 0.00% 0.04% 0.01% 0 Marvell wk-a
Pow

192 1730 2685 644 0.00% 0.00% 0.00% 0 MLDSN L2MCM

193 0 1 0 0.00% 0.00% 0.00% 0 MRD

194 0 1 0 0.00% 0.00% 0.00% 0 MLD_SNOOP

195 0 1 0 0.00% 0.00% 0.00% 0 HRPC hl2mcm
mlds

196 0 1 0 0.00% 0.00% 0.00% 0 IP RIB Update

197 0 1 0 0.00% 0.00% 0.00% 0 Auth-proxy AAA
B

198 52 12122 4 0.00% 0.00% 0.00% 0 IP Admin SM
Proc

199 0 2 0 0.00% 0.00% 0.00% 0 DHCPD Receive

200 68122 177424 383 0.00% 0.05% 0.02% 0 IP SNMP

201 18522 88390 209 0.00% 0.00% 0.00% 0 PDU DISPATCHER

202 0 2 0 0.00% 0.00% 0.00% 0 AAA Cached
Serve

203 0 2 0 0.00% 0.00% 0.00% 0 LOCAL AAA

204 0 2 0 0.00% 0.00% 0.00% 0 TPLUS

205 0 1 0 0.00% 0.00% 0.00% 0 HRPC x_setup
req

206 0 23 0 0.00% 0.00% 0.00% 0 VTP Trap
Process

207 0 2 0 0.00% 0.00% 0.00% 0 VTPMIB EDIT
BUFF

208 0 2 0 0.00% 0.00% 0.00% 0 DHCP Security
He

209 0 1 0 0.00% 0.00% 0.00% 0 HCD Process

210 0 1 0 0.00% 0.00% 0.00% 0 HRPC cable
diagn

211 0 2 0 0.00% 0.00% 0.00% 0 DiagCard1/-1

212 373243 23594810 15 0.00% 0.00% 0.00% 0 PM Callback

213 2173 35766 60 0.00% 0.00% 0.00% 0 VLAN Manager

214 99173 88348 1122 0.15% 0.12% 0.07% 0 SNMP ENGINE

215 4581 290721 15 0.00% 0.00% 0.00% 0 dhcp snooping
sw

216 0 3 0 0.00% 0.00% 0.00% 0 RADIUS TEST
CMD

217 0 2 0 0.00% 0.00% 0.00% 0 AAA SEND STOP
EV

218 0 1 0 0.00% 0.00% 0.00% 0 Syslog Traps

219 0 1 0 0.00% 0.00% 0.00% 0 SAA MPLSLM
Proce

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

220 0 2 0 0.00% 0.00% 0.00% 0 STP FAST
TRANSIT

221 0 2 0 0.00% 0.00% 0.00% 0 CSRT RAPID
TRANS

222 0 1 0 0.00% 0.00% 0.00% 0 SNMP
ConfCopyPro

223 43729 3659774 11 0.15% 0.01% 0.00% 0 NTP

224 0 43 0 0.00% 0.00% 0.00% 0 hulc cfg mgr
mas

225 46031 241 191000 0.00% 0.00% 0.00% 0 hulc running
con

226 0 1 0 0.00% 0.00% 0.00% 0 SNMP Traps

228 208 169 1230 0.15% 0.01% 0.00% 1 Virtual Exec
SW2#

Blogs and organic groups at http://www.ccie.net
Received on Wed Aug 10 2011 - 10:30:52 ART

This archive was generated by hypermail 2.2.0 : Thu Sep 01 2011 - 06:05:56 ART