Overloaded Cisco Router??
I am wondering where to start looking: C1841#sh processes cpu history C1841 11:44:05 AM Monday Oct 1 2018 UTC 999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
More details. While digging, I have disabled snmp-server and some debug logging configs. Now I have: C1841#sh processes cpu sorted CPU utilization for five seconds: 64%/17%; one minute: 78%; five minutes: 66% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process * 121 467636 134723 3471 45.27% 55.87% 47.09% 0 IP Input* I still think this is high, no? I need to figure out what exactly comprises IP Input On Mon, 1 Oct 2018 at 12:11, Odhiambo Washington <odhiambo@gmail.com> wrote:
I am wondering where to start looking:
C1841#sh processes cpu history
C1841 11:44:05 AM Monday Oct 1 2018 UTC
999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
The silence here ……. On Mon, 1 Oct 2018 at 14:12, Odhiambo Washington via skunkworks < skunkworks@lists.my.co.ke> wrote:
More details.
While digging, I have disabled snmp-server and some debug logging configs. Now I have:
C1841#sh processes cpu sorted CPU utilization for five seconds: 64%/17%; one minute: 78%; five minutes: 66% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process * 121 467636 134723 3471 45.27% 55.87% 47.09% 0 IP Input*
I still think this is high, no? I need to figure out what exactly comprises IP Input
On Mon, 1 Oct 2018 at 12:11, Odhiambo Washington <odhiambo@gmail.com> wrote:
I am wondering where to start looking:
C1841#sh processes cpu history
C1841 11:44:05 AM Monday Oct 1 2018 UTC
999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." _______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
Sometimes the silence really helps! I decided to patiently dig information about this issue and made very good progress: 1. https://www.cisco.com/c/en/us/support/docs/routers/7500-series-routers/41160... 2. https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c02768100 3. https://www.certificationkits.com/cisco-certification/cisco-ccna-640-802-exa... 4. https://community.cisco.com/t5/collaboration-voice-and-video/how-to-resolve-... On Tue, 2 Oct 2018 at 10:06, Alvin Ochola <ajochola@gmail.com> wrote:
The silence here …….
On Mon, 1 Oct 2018 at 14:12, Odhiambo Washington via skunkworks < skunkworks@lists.my.co.ke> wrote:
More details.
While digging, I have disabled snmp-server and some debug logging configs. Now I have:
C1841#sh processes cpu sorted CPU utilization for five seconds: 64%/17%; one minute: 78%; five minutes: 66% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process * 121 467636 134723 3471 45.27% 55.87% 47.09% 0 IP Input*
I still think this is high, no? I need to figure out what exactly comprises IP Input
On Mon, 1 Oct 2018 at 12:11, Odhiambo Washington <odhiambo@gmail.com> wrote:
I am wondering where to start looking:
C1841#sh processes cpu history
C1841 11:44:05 AM Monday Oct 1 2018 UTC
999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." _______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
Yea the silence not good for the community :-) So Wash, is this resolved? Also what IOS version, protocols and services are you running on the box? Regards, Simon Mayoye ________________________________ From: Odhiambo Washington via skunkworks <skunkworks@lists.my.co.ke> Sent: 02 October 2018 10:58 AM To: Alvin Jason Ochieng Cc: Odhiambo Washington; Skunkworks Mailing List Subject: Re: [Skunkworks] Overloaded Cisco Router?? Sometimes the silence really helps! I decided to patiently dig information about this issue and made very good progress: 1. https://www.cisco.com/c/en/us/support/docs/routers/7500-series-routers/41160... 2. https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c02768100 3. https://www.certificationkits.com/cisco-certification/cisco-ccna-640-802-exa... 4. https://community.cisco.com/t5/collaboration-voice-and-video/how-to-resolve-... On Tue, 2 Oct 2018 at 10:06, Alvin Ochola <ajochola@gmail.com<mailto:ajochola@gmail.com>> wrote: The silence here ....... On Mon, 1 Oct 2018 at 14:12, Odhiambo Washington via skunkworks <skunkworks@lists.my.co.ke<mailto:skunkworks@lists.my.co.ke>> wrote: More details. While digging, I have disabled snmp-server and some debug logging configs. Now I have: C1841#sh processes cpu sorted CPU utilization for five seconds: 64%/17%; one minute: 78%; five minutes: 66% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 121 467636 134723 3471 45.27% 55.87% 47.09% 0 IP Input I still think this is high, no? I need to figure out what exactly comprises IP Input On Mon, 1 Oct 2018 at 12:11, Odhiambo Washington <odhiambo@gmail.com<mailto:odhiambo@gmail.com>> wrote: I am wondering where to start looking: C1841#sh processes cpu history C1841 11:44:05 AM Monday Oct 1 2018 UTC 999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." _______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke<mailto:skunkworks@lists.my.co.ke> ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
Hi Simon, Nope, the problem is still there and now I am beginning to think it's probably some worm on the network. Not yet certain. It could be an attack from outside too. Please see a separate mail to you, off list. On Tue, 2 Oct 2018 at 11:07, Simon Mayoye <mayoye@seacom.mu> wrote:
Yea the silence not good for the community :-)
So Wash, is this resolved? Also what IOS version, protocols and services are you running on the box?
Regards,
Simon Mayoye ------------------------------ *From:* Odhiambo Washington via skunkworks <skunkworks@lists.my.co.ke> *Sent:* 02 October 2018 10:58 AM *To:* Alvin Jason Ochieng *Cc:* Odhiambo Washington; Skunkworks Mailing List *Subject:* Re: [Skunkworks] Overloaded Cisco Router??
Sometimes the silence really helps! I decided to patiently dig information about this issue and made very good progress:
1. https://www.cisco.com/c/en/us/support/docs/routers/7500-series-routers/41160... 2. https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c02768100 3. https://www.certificationkits.com/cisco-certification/cisco-ccna-640-802-exa... 4. https://community.cisco.com/t5/collaboration-voice-and-video/how-to-resolve-...
On Tue, 2 Oct 2018 at 10:06, Alvin Ochola <ajochola@gmail.com> wrote:
The silence here …….
On Mon, 1 Oct 2018 at 14:12, Odhiambo Washington via skunkworks < skunkworks@lists.my.co.ke> wrote:
More details.
While digging, I have disabled snmp-server and some debug logging configs. Now I have:
C1841#sh processes cpu sorted CPU utilization for five seconds: 64%/17%; one minute: 78%; five minutes: 66% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process * 121 467636 134723 3471 45.27% 55.87% 47.09% 0 IP Input*
I still think this is high, no? I need to figure out what exactly comprises IP Input
On Mon, 1 Oct 2018 at 12:11, Odhiambo Washington <odhiambo@gmail.com> wrote:
I am wondering where to start looking:
C1841#sh processes cpu history
C1841 11:44:05 AM Monday Oct 1 2018 UTC
999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." _______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
Check ACL, Route maps, NAT overload or flood attack - those are main culplits. Also check dynamic route looping - like OSPF - big consumer. Kind Regards, Wilson./ On Mon, 1 Oct 2018 at 14:12, Odhiambo Washington via skunkworks < skunkworks@lists.my.co.ke> wrote:
More details.
While digging, I have disabled snmp-server and some debug logging configs. Now I have:
C1841#sh processes cpu sorted CPU utilization for five seconds: 64%/17%; one minute: 78%; five minutes: 66% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process * 121 467636 134723 3471 45.27% 55.87% 47.09% 0 IP Input*
I still think this is high, no? I need to figure out what exactly comprises IP Input
On Mon, 1 Oct 2018 at 12:11, Odhiambo Washington <odhiambo@gmail.com> wrote:
I am wondering where to start looking:
C1841#sh processes cpu history
C1841 11:44:05 AM Monday Oct 1 2018 UTC
999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." _______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
If still not fixed based on above input; check if there's a broadcast or multicast storm which could be affecting your dynamic routing - can be fixed using storm-control broadcast/multicast level command on a per interface level or globally (depends on platform). You can also set some control-plane limits to help limit CPU resource utilization. Kennedy On Tue, Oct 2, 2018 at 1:09 PM Thuo Wilson via skunkworks < skunkworks@lists.my.co.ke> wrote:
Check ACL, Route maps, NAT overload or flood attack - those are main culplits. Also check dynamic route looping - like OSPF - big consumer. Kind Regards, Wilson./
On Mon, 1 Oct 2018 at 14:12, Odhiambo Washington via skunkworks < skunkworks@lists.my.co.ke> wrote:
More details.
While digging, I have disabled snmp-server and some debug logging configs. Now I have:
C1841#sh processes cpu sorted CPU utilization for five seconds: 64%/17%; one minute: 78%; five minutes: 66% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process * 121 467636 134723 3471 45.27% 55.87% 47.09% 0 IP Input*
I still think this is high, no? I need to figure out what exactly comprises IP Input
On Mon, 1 Oct 2018 at 12:11, Odhiambo Washington <odhiambo@gmail.com> wrote:
I am wondering where to start looking:
C1841#sh processes cpu history
C1841 11:44:05 AM Monday Oct 1 2018 UTC
999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." _______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
_______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@Kennedy Aseda <samskid5@gmail.com> the IOS is c1841-advsecurityk9-mz.151-4 I welcome suggestions. I could also share access so you can do a probe, if you get some time. On Tue, 9 Oct 2018 at 12:34, Kennedy Aseda via skunkworks < skunkworks@lists.my.co.ke> wrote:
If still not fixed based on above input; check if there's a broadcast or multicast storm which could be affecting your dynamic routing - can be fixed using storm-control broadcast/multicast level command on a per interface level or globally (depends on platform).
You can also set some control-plane limits to help limit CPU resource utilization.
Kennedy
On Tue, Oct 2, 2018 at 1:09 PM Thuo Wilson via skunkworks < skunkworks@lists.my.co.ke> wrote:
Check ACL, Route maps, NAT overload or flood attack - those are main culplits. Also check dynamic route looping - like OSPF - big consumer. Kind Regards, Wilson./
On Mon, 1 Oct 2018 at 14:12, Odhiambo Washington via skunkworks < skunkworks@lists.my.co.ke> wrote:
More details.
While digging, I have disabled snmp-server and some debug logging configs. Now I have:
C1841#sh processes cpu sorted CPU utilization for five seconds: 64%/17%; one minute: 78%; five minutes: 66% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process * 121 467636 134723 3471 45.27% 55.87% 47.09% 0 IP Input*
I still think this is high, no? I need to figure out what exactly comprises IP Input
On Mon, 1 Oct 2018 at 12:11, Odhiambo Washington <odhiambo@gmail.com> wrote:
I am wondering where to start looking:
C1841#sh processes cpu history
C1841 11:44:05 AM Monday Oct 1 2018 UTC
999999999999999999999999999999999999999999999999999999999999 444499999999998888800000888888888888888999999999999999777777 100 *************** ************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6 0 5 0 5 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." _______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
_______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
_______________________________________________ skunkworks mailing list skunkworks@lists.my.co.ke ------------ List info, subscribe/unsubscribe http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------
Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."
participants (5)
-
Alvin Ochola -
Kennedy Aseda -
Odhiambo Washington -
Simon Mayoye -
Thuo Wilson