From: Jim Devane (jim@powerpulse.cc)
Date: Fri Nov 28 2003 - 22:32:33 GMT-3
All,
I had soft-reconfig on 2 of my production routers as well. I was taking 3
full views (701, 6395, 7911) and passing 4 tables downstream ( 22584, 18687,
27496, 3718)
I noticed that my memory changed VERY little as we suspected. Unfortunately,
I didn't realize a sh ip bgp sum was the best data to capture.
I did take a look at "sh mem sum" and "sh proc mem sort hold" ( BGP was
obviously the tope candidate" I actually saw an increase in mem used.
Fortunately, the GRP-B can take 512 MB and when I get some payola I will go
to PRP-1 , and in the meantime bitch about how noisy some ASN's are but for
me, either way it seems like the command really has no effect.
The output are from routers that are identical. GSR12012, GRP-B, 256MB.
76911 comes into core02 and 701 and 6395 come into core 01. I have iBGP
running between them.
I guess more FYI than anything else.
THANKS!
Jim
pwps-core01#sh mem summ
Head Total(b) Used(b) Free(b) Lowest(b)
Largest(b)
Processor 5375D580 210381440 144389172 65992268 41647752
45936792
Fast 5373D580 131080 129816 1264 1264
1212
pwps-core02#sh mem summ
Head Total(b) Used(b) Free(b) Lowest(b)
Largest(b)
Processor 5375D580 210381440 135321040 75060400 49995544
49298436
Fast 5373D580 131080 129816 1264 1264
1212
pwps-core01#sh proc mem sort hold
Total: 210381440, Used: 144391112, Free: 65990328
PID TTY Allocated Freed Holding Getbufs Retbufs Process
156 0 371707392 20959844 114648228 0 0 BGP Router
pwps-core02#sh proc mem sorted hold
Total: 210381440, Used: 135326460, Free: 75054980
PID TTY Allocated Freed Holding Getbufs Retbufs Process
159 0 613600380 126390232 106952464 0 0 BGP Router
AFTER THE RESETS!
pwps-core01#sh mem summ
Head Total(b) Used(b) Free(b) Lowest(b)
Largest(b)
Processor 5375D580 210381440 149174912 61206528 41647752
41580088
Fast 5373D580 131080 129816 1264 1264
1212
pwps-core02#sh mem summ
Head Total(b) Used(b) Free(b) Lowest(b)
Largest(b)
Processor 5375D580 210381440 136519240 73862200 49995544
48314736
Fast 5373D580 131080 129816 1264 1264
1212
pwps-core01#sh proc mem sort hold
Total: 210381440, Used: 149180644, Free: 61200796
PID TTY Allocated Freed Holding Getbufs Retbufs Process
156 0 522355920 35517312 119280452 0 0 BGP Router
pwps-core02#sh proc mem sort hold
Total: 210381440, Used: 136524424, Free: 73857016
PID TTY Allocated Freed Holding Getbufs Retbufs Process
159 0 635935864 134432128 108150468 0 0 BGP Router
-----Original Message-----
From: Scott Morris [mailto:swm@emanon.com]
Sent: Friday, November 28, 2003 1:56 PM
To: 'Anthony Pace'; 'Brian McGahan'; 'Jim Devane'; ccielab@groupstudy.com
Cc: cchurch@wamnetgov.com; marcelo.medina@citigroup.com
Subject: RE: Why Disabling BPG soft-reconfig did not reclaim Memory?
It's my impression that the databases are set up in a particular fashion
by the soft-reconfig command, regardless of whether you are or are not
doing any particular filtering.
How much memory is on your router(s)? Were you having any difficulties
accepting all of the routes before?
Short of going to a default route from your provider, you may consider a
prefix-list idea (perhaps on their end if they will ponder it for you)
to prevent routes of, say more specific than /20 from being sent to you,
or being accepted on your end.
I would be interested in the output of "show ip bgp summary" both with
and without the soft-reconfig options... That'll tell a bit about the
memory being used and how many entries et al. At least if you feel like
resetting and/or rebooting things again! :)
Scott
-----Original Message-----
From: nobody@groupstudy.com [mailto:nobody@groupstudy.com] On Behalf Of
Anthony Pace
Sent: Friday, November 28, 2003 2:21 PM
To: Scott Morris; 'Brian McGahan'; 'Jim Devane'; ccielab@groupstudy.com
Cc: cchurch@wamnetgov.com; marcelo.medina@citigroup.com
Subject: Why Disabling BPG soft-reconfig did not reclaim Memory?
I saw this thread and it was in vien of my situation. I had 2 production
routers doing soft-reconfig and were at 90% memory utilization. They
were taking in full tables from two providers(1 each). I removed
soft-reconfig from both configs, cleared BGP and even rebooted routers
but after they converged they were still using 90% memory.
I now cannot view received routes, validating that the "untouched table
is gone", but the memory utilization is identical.
Is it possibel that instead of storing 2 tables, perhaps the router
marks certain routes or indexes it some how to accomodate your filters?
I was not really filtering it so maybe it never stored a second table?
My next step is to only take in partial routes (my neigbor's neigbors)
Tony Pace
On Mon, 24 Nov 2003 16:17:11 -0500, "Scott Morris" <swm@emanon.com>
said:
> That's because the "policy" of being able to do a soft reset is
> something that has to be set up in the database at the beginning of a
> peering arrangement. So whenever you add it, the database needs to be
> refreshed accordingly. Extra chunks of the database are used to be
> able to do on-the-fly processing of the soft-reset.
>
> It's not so much that memory is held (the way we normally think of
> applications), but more that the BGP database is allocated differently
> in order to allow for better parsing, so that takes up memory. In my
> experience, it hasn't necessarily been a 2:1 increase like someone
> else had mentioned, but it may cause issues on otherwise over-laden
> routers.
>
> Scott Morris, CCIE4 (R&S/ISP-Dial/Security/Service Provider) #4713,
> CISSP, JNCIS, et al. IPExpert CCIE Program Manager
> IPExpert Sr. Technical Instructor
> swm@emanon.com/smorris@ipexpert.net
> http://www.ipexpert.net
>
>
> -----Original Message-----
> From: nobody@groupstudy.com [mailto:nobody@groupstudy.com] On Behalf
> Of Brian McGahan
> Sent: Monday, November 24, 2003 3:52 PM
> To: 'Jim Devane'; ccielab@groupstudy.com
> Subject: RE: BGP Soft Reset (BGP RIB)
>
>
> Jim,
>
> It appears that adding soft-reconfig forces a hard reset, but
> removing it does not:
>
> R1(config)#router bgp 100
> R1(config-router)#nei 204.12.1.254 remote-as 54
> *Mar 4 20:15:07.836: %BGP-5-ADJCHANGE: neighbor 204.12.1.254 Up
> R1(config-router)#nei 204.12.1.254 soft-reconfig in
> *Mar 4 20:15:24.540: %BGP-5-ADJCHANGE: neighbor 204.12.1.254 Down
Soft
> reconfig change *Mar 4 20:16:02.326: %BGP-5-ADJCHANGE: neighbor
> 204.12.1.254 Up
> R1(config-router)#do sh ip bgp nei 204.12.1.254 received-routes
<snip>
> Total number of prefixes 10
> R1(config-router)#no nei 204.12.1.254 soft-reconfig in
> R1(config-router)#do sh ip bgp nei 204.12.1.254 received-routes %
> Inbound soft reconfiguration not enabled on 204.12.1.254
>
>
> It must need a hard clear to populate the 'received' routes in
the
> first place. I'd be interested to see your results of the memory
> savings if any.
>
>
> Brian McGahan, CCIE #8593
> bmcgahan@internetworkexpert.com
>
> Internetwork Expert, Inc.
> http://www.InternetworkExpert.com
> Toll Free: 877-224-8987
> Direct: 708-362-1418 (Outside the US and Canada)
>
>
>
> > -----Original Message-----
> > From: Jim Devane [mailto:jim@powerpulse.cc]
> > Sent: Monday, November 24, 2003 1:49 PM
> > To: 'Brian McGahan'
> > Subject: RE: BGP Soft Reset (BGP RIB)
> >
> >
> > Brian,
> >
> > Actually, I don't believe the hard reset takes place automatically.
> > At least not on a 12012 running 12.0.25T
> >
> > I went through my config's and found 2 neighbors that had the
> > soft-reconfig in place and after I checked to see that the opposite
> > site router was running 12.2 I removed it.. the BGP session has not
> > had any change to it.
> >
> > Unless you meant something else and I am mis-understanding.
> >
> > I hope I can schedule a window this week to reset the session to see
> > if there is any memory savings.
> >
> > Thanks,
> > Jim
> >
> >
> >
> > -----Original Message-----
> > From: nobody@groupstudy.com [mailto:nobody@groupstudy.com] On Behalf
> > Of Brian McGahan
> > Sent: Monday, November 24, 2003 11:38 AM
> > To: 'kurt'; ccielab@groupstudy.com
> > Subject: RE: BGP Soft Reset (BGP RIB)
> >
> > Kurt,
> >
> > Correct, if route-refresh is supported you can 'clear
> > ip bgp x.x.x.x in' without memory implications.
> > Route-refresh is supported by default as of 12.0T.
> >
> > > also memory held by a previous soft (configured) reset will be
> > > held
> > > until a reboot ??
> >
> > This I'm not sure about, but I'm going to guess no. If you add
or
> > remove the 'soft-reconfiguration inbound' statement a hard clear of
> > the BGP neighbor automatically takes place. Therefore any cached
> > received routes will be automatically flushed.
> >
> >
> > HTH,
> >
> > Brian McGahan, CCIE #8593
> > bmcgahan@internetworkexpert.com
> >
> > Internetwork Expert, Inc.
> > http://www.InternetworkExpert.com
> > Toll Free: 877-224-8987
> > Direct: 708-362-1418 (Outside the US and Canada)
> >
> >
> >
> > > -----Original Message-----
> > > From: kurt [mailto:kurt@cybernex.net]
> > > Sent: Monday, November 24, 2003 1:18 PM
> > > To: Brian McGahan
> > > Subject: Re: BGP Soft Reset (BGP RIB)
> > >
> > >
> > >
> > > ok ,
> > >
> > > if i understand this correctly
> > > i had been in the practice of setting
> > > neighbor xxxxx soft-reconfiguration inbound
> > > this causes the memory doubling issue
> > > correct?
> > >
> > > if i do a
> > > sh ip bgp neighbor
> > > and it says route refresh is supported then i can
> > > do a clear ip bgp * soft without memory implications right ??
> > >
> > > also memory held by a previous soft (configured) reset will be
> > > held
> > > until a reboot ??
> > >
> > > main-gw#sbn
> > > BGP neighbor is , remote, external link
> > > BGP version 4, remote router ID BGP state = Established,
> > > up for 7w4d
> > > Last read 00:00:00, hold time is 180, keepalive interval is 60
> > > seconds
> > > Neighbor capabilities:
> > > Route refresh: advertised and received(old & new)
> > > Address family IPv4 Unicast: advertised and received
> > > Message statistics:
> > > InQ depth is 7
> > > OutQ depth is 0
> > > Sent Rcvd
> > > Opens: 2 2
> > > Notifications: 1 0
> > > Updates: 6 3448866
> > > Keepalives: 148547 148540
> > > Route Refresh: 1 0
> > > Total: 148557 3597408
> > > Default minimum time between advertisement runs is 30 seconds
> > >
> > > For address family: IPv4 Unicast
> > > BGP table version 10566014, neighbor version 0
> > > Index 1, Offset 0, Mask 0x2
> > > Inbound path policy configured
> > > Outgoing update prefix filter list is locally-originated
> > >
> > >
> > > ----- Original Message -----
> > > From: "Brian McGahan" <bmcgahan@internetworkexpert.com>
> > > To: "'Kurt Kruegel'" <kurt@cybernex.net>;
> > > <Ken.Farrington@barclayscapital.com>; <ccielab@groupstudy.com>
> > > Sent: Monday, November 24, 2003 1:50 PM
> > > Subject: RE: BGP Soft Reset (BGP RIB)
> > >
> > >
> > > > Kurt,
> > > >
> > > > If route-refresh is supported then both 'clear ip bgp * in'
> > > and 'clear
> > > > ip bgp * soft in' will do route-refresh. If route-refresh is
> > > > not
> > > > supported and soft-reconfiguration is not configured you
> > > will get the
> > > > following error message:
> > > >
> > > > router#clear ip bgp 172.16.4.3 in
> > > > %BGP: Inbound soft reconfig for 172.16.4.3 not possible as it
> > > > has neither refresh capability, nor inbound soft reconfig
> > > >
> > > > router#clear ip bgp 172.16.4.3 soft in
> > > > %BGP: Inbound soft reconfig for 172.16.4.3 not possible as it
> > > > has neither refresh capability, nor inbound soft reconfig
> > > >
> > > >
> > > >
> > > > HTH,
> > > >
> > > > Brian McGahan, CCIE #8593 bmcgahan@internetworkexpert.com
> > > >
> > > > Internetwork Expert, Inc. http://www.InternetworkExpert.com Toll
> > > > Free: 877-224-8987
> > > > Direct: 708-362-1418 (Outside the US and Canada)
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Kurt Kruegel [mailto:kurt@cybernex.net]
> > > > > Sent: Sunday, November 23, 2003 7:45 PM
> > > > > To: Brian McGahan; Ken.Farrington@barclayscapital.com;
> > > > > ccielab@groupstudy.com
> > > > > Subject: Re: BGP Soft Reset (BGP RIB)
> > > > >
> > > > >
> > > > > so ....
> > > > > will
> > > > > clear ip bgp in
> > > > > force a route refresh ???
> > > > >
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > From: "Brian McGahan" <bmcgahan@internetworkexpert.com>
> > > > > To: <Ken.Farrington@barclayscapital.com>;
> > <ccielab@groupstudy.com>
> > > > > Sent: Sunday, November 23, 2003 8:21 PM
> > > > > Subject: RE: BGP Soft Reset (BGP RIB)
> > > > >
> > > > >
> > > > > > Ken,
> > > > > >
> > > > > > "Soft-reconfiguration" is Cisco's old proprietary way
> > > of allowing
> > > > > > changes in the BGP policy without tearing down the TCP
> > > session and
> > > > > > rebuilding it. "Route-refresh" is the standard and
> > > > > preferred method
> > > > > > to do so.
> > > > > >
> > > > > > Soft-reconfiguration must be preconfigured with the
> > > > > 'neighbor x.x.x.x
> > > > > > soft-reconfiguration inbound' bgp subcommand. With
> > > > > soft-reconfig, the
> > > > > > router caches prefixes received from a neighbor, then
> > > installs a
> > > > > > separate copy which is a result of configured inbound
> > > > > > filters. The original copy can be seen by issuing the 'show
> > > > > > ip bgp nei x.x.x.x received-routes' command. Using this
> > > > > > method is not recommended, because the memory required on a
> > > > > > per
> > > neighbor basis
> > > > > > can be
> > > > > effectively
> > > > > > doubled.
> > > > > >
> > > > > > Unlike soft-reconfig, route-refresh does not need to keep
> > > > > two copies
> > > > > > of the BGP table. There is no way to see the
> > > unmodified received
> > > > > > table other than going to the upstream side and doing a 'sh
> > > > > ip bgp nei
> > > > > > x.x.x.x advertised'. Route-refresh is part of the BGP
> > > > > > capabilities exchange, and does not require any additional
> > > > > > configuration.
> > > > > >
> > > > > >
> > > > > > HTH,
> > > > > >
> > > > > > Brian McGahan, CCIE #8593 bmcgahan@internetworkexpert.com
> > > > > >
> > > > > > Internetwork Expert, Inc.
> > http://www.InternetworkExpert.com Toll
> > > > > > Free: 877-224-8987
> > > > > > Direct: 708-362-1418 (Outside the US and Canada)
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: nobody@groupstudy.com
> > [mailto:nobody@groupstudy.com] On
> > > > > > > Behalf Of Bob Sinclair
> > > > > > > Sent: Sunday, November 23, 2003 5:05 PM
> > > > > > > To: Ken.Farrington@barclayscapital.com;
> > ccielab@groupstudy.com
> > > > > > > Subject: Re: BGP Soft Reset (BGP RIB)
> > > > > > >
> > > > > > >
> > > > > > > Ken,
> > > > > > >
> > > > > > > The soft-reconfiguration inbound command is
> > required if you
> > > > > > > want to issue a
> > > > > > > "neighbor received-routes" command. The received routes
> > > > > > > command will show
> > > > > > > you what the neighbor is sending you BEFORE any inbound
> > > > > > > policies are applied. Very helpful!
> > > > > > >
> > > > > > > HTH,
> > > > > > >
> > > > > > > -Bob Sinclair
> > > > > > > CCIE #10427, CISSP, MCSE bsinclair@netmasterclass.net
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > From: <Ken.Farrington@barclayscapital.com>
> > > > > > > To: <ccielab@groupstudy.com>
> > > > > > > Sent: Sunday, November 23, 2003 5:05 PM
> > > > > > > Subject: BGP Soft Reset (BGP RIB)
> > > > > > >
> > > > > > >
> > > > > > > > Guys,
> > > > > > > >
> > > > > > > > Please can I confirm that nowadays you do not have
> > > to use the "
> > > > > > > > neighbor 200.200.1.1 soft-reconfiguration
> > inbound" command.
> > > > > > > >
> > > > > > > > All my routers without this command have the
> > > > > > > > route-refresh
> > > > > > > capability
> > > > > > > > on (please see output below), and as such and I can do a
> > > > > > > >
> > > > > > > > "cle ip bgp 200.200.1.1 soft in" or "cle ip bgp
> > > > > > > 200.200.1.1 in"
> > > > > > > >
> > > > > > > > without configuring soft-reconfiguration under the
> > > BGP process.
> > > > > > > >
> > > > > > > >
> > > > > > > > Also, I assume the router stores two copies of the
> > > RIB, one
> > > > > > > > unmodified
> > > > > > > and
> > > > > > > > one modified with inbound policies. If this is
> > > > > correct, is there
> > > > > > > > a
> > > > > > > > way of showing the unmodified RIB ?
> > > > > > > >
> > > > > > > >
> > > > > > > > TEST2#sh ip bgp nei 200.200.1.1
> > > > > > > > BGP neighbor is 200.200.1.1, remote AS 6500,
> > external link
> > > > > > > > BGP version 4, remote router ID 155.195.16.161
> > > > > > > > BGP state = Established, up for 00:06:48
> > > > > > > > Last read 00:00:48, hold time is 180, keepalive
> > > > > > > > interval
> > > > > > > is 60 seconds
> > > > > > > > Neighbor capabilities:
> > > > > > > > Route refresh: advertised and received(new)
> > > > > > > > <======================Please Note
> > > > > > > > Address family IPv4 Unicast: advertised and received
> > > > > > > > Received 1481 messages, 936 notifications, 0 in queue
> > > > > > > > Sent 1537 messages, 0 notifications, 0 in queue
> > > > > > > > Route refresh request: received 0, sent 6
> > > > > > > > Default minimum time between advertisement runs
> > > is 30 seconds
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > ------------------------------------------------------------------
> > > --
> > > > > > > --
> > > > > > > > --
> > > > > > > > For more information about Barclays Capital, please
> > > > > > > > visit
> > > > > > > > our web site at http://www.barcap.com.
> > > > > > > >
> > > > > > > >
> > > > > > > > Internet communications are not secure and therefore
> > > > > the Barclays
> > > > > > > > Group does not accept legal responsibility for the
> > > > > contents of this
> > > > > > > > message. Although the Barclays Group operates
> > > > > > > > anti-virus
> > > > > > > programmes,
> > > > > > > > it does not accept responsibility for any damage
> > > > > whatsoever that
> > > > > > > > is
> > > > > > > > caused by viruses being passed. Any views or opinions
> > > > > > > presented are
> > > > > > > > solely those of the author and do not necessarily
> > represent
> > > > > > > those of
> > > > > > > > the Barclays Group. Replies to this email may be
> > > > > monitored by the
> > > > > > > > Barclays Group for operational or business reasons.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > ------------------------------------------------------------------
> > > --
> > > > > > > --
> > > > > > > > --
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > __________________________________________________________________
> > > __
> > > > > > > __
> > > > > > > > _
> > > > > > > > Please help support GroupStudy by purchasing your study
> > > > > > > materials from:
> > > > > > > > http://shop.groupstudy.com
> > > > > > > >
> > > > > > > > Subscription information may be found at:
> > > > > > > > http://www.groupstudy.com/list/CCIELab.html
> > > > > > >
> > > > > > >
> > ______________________________________________________________
> > > > > > > _________
> > > > > > > Please help support GroupStudy by purchasing your study
> > > > > > > materials from: http://shop.groupstudy.com
> > > > > > >
> > > > > > > Subscription information may be found at:
> > > > > > > http://www.groupstudy.com/list/CCIELab.html
> > > > > >
> > > > > >
> > > > >
> > >
> > ____________________________________________________________________
> > __
> > > > > > _
> > > > > > Please help support GroupStudy by purchasing your study
> > > > > materials from:
> > > > > > http://shop.groupstudy.com
> > > > > >
> > > > > > Subscription information may be found at:
> > > > > > http://www.groupstudy.com/list/CCIELab.html
> > > >
> > > >
> > > ______________________________________________________________
> > > _________
> > > > Please help support GroupStudy by purchasing your study
> > > materials from:
> > > > http://shop.groupstudy.com
> > > >
> > > > Subscription information may be found at:
> > > > http://www.groupstudy.com/list/CCIELab.html
> >
> > ______________________________________________________________
> > _________
> > Please help support GroupStudy by purchasing your study
> > materials from: http://shop.groupstudy.com
> >
> > Subscription information may be found at:
> > http://www.groupstudy.com/list/CCIELab.html
>
> ______________________________________________________________________
> _
> Please help support GroupStudy by purchasing your study materials
from:
> http://shop.groupstudy.com
>
> Subscription information may be found at:
> http://www.groupstudy.com/list/CCIELab.html
>
> ______________________________________________________________________
> _
> Please help support GroupStudy by purchasing your study materials
from:
> http://shop.groupstudy.com
>
> Subscription information may be found at:
> http://www.groupstudy.com/list/CCIELab.html
-- Anthony Pace anthonypace@fastmail.fm-- http://www.fastmail.fm - Same, same, but different
This archive was generated by hypermail 2.1.4 : Fri Dec 12 2003 - 12:29:19 GMT-3