Re: OT : BGP soft-reconfiguration inbound memory costs

Hi Eric,

Unfortunately I didn't get any measurements as I was trying to fix one issue
on the box and ended up creating another. Turned out this particular 7200
was not like the rest we had and it only had 256m of mem on the G1 instead
of the 1gb we had on all others. I had already tuned BGP to accept as many
routes as possible, and I enabled soft-reconfig to check if we were
filtering a route and ended up almost killing the box (mem-alloc failure,
CEF disabling, etc). Since the box was unreachable by our NMS, I couldn't
collect data, so i'm not sure how much exactly the route-cache takes up. But
now as I'm typing this, I remember doing this 5 or 6 years ago, and I
believe the that soft-reconfig has been tweaked over the years and probably
takes up much less than id did in my experience from before.

As far as trying to measure just how much one peers' paths in the RIB
actually takes up would be very difficult I think (assuming multiple peers
on one router). I know BGP has its own MIB, but I have yet to see anything
that has the memory each peer takes up. I am not aware of any other commands
to show those metrics either (other than the one's you've already
mentioned). Only thing I can think of would be to filter everything on one
of the peers you have (preferably one that has no paths installed in the FIB
and not forwarding to them :), and shutdown the peer turn. Get a
measurement, turn them back on with soft-reconfig enabled and see if the
memory jumps any at all since you not accepting any routes. (Not the best of
methods, I would agree:) It would be interesting to get exactly the values
of a prefix stored in the RIB.

And to be honest, after doing some thinking about it, i'm not even sure
those pre-filtered paths sit in the RIB anymore. They may be stored
differently than RIB routes. Anyone else have a clue??

-
max

On Sun, Feb 27, 2011 at 4:45 PM, eric_at_linux.ca <eric.lauriault_at_gmail.com>wrote:

> Thanks for your replies. My copy of BGP design and implementation is in the
> mail :) The chapter on BGP memory use estimation sounds quite interesting.
>
> Fabian : This is what I used as a basis for my labbing. Maybe I'm not
> monitoring the right counters which would explain why I can't see the
> difference in memory usage.
>
> Max : I think this configuration was added by one of my predecessors to be
> able to run "show ip bgp neighbor x.x.x.x received-routes". You mentioned
> that you could indeed notice that memory usage increased. Which show
> commands/OID did you use to measure this difference?
>
> Regards,
>
> Eric
>
> On Sun, Feb 27, 2011 at 3:16 AM, Max Pierson <nmaxpierson_at_gmail.com>wrote:
>
>> >Does anybody have any experience with this feature?
>>
>> On 12.4.x (something mainline I believe) I do. It does take up memory on a
>> 720xVXR router with a NPE-G1 thats noticeable.
>>
>> As Fabian was alluding to, it's only necessary when testing filters now
>> that BGP supports route-refresh so you don't have to keep a state table.
>> There's no other reason I can think of where you would need to apply it, so
>> I only use it on peers if I need to test filters.
>>
>>
>> -
>> m
>>
>>
>> On Sat, Feb 26, 2011 at 4:39 PM, eric_at_linux.ca <eric.lauriault_at_gmail.com>wrote:
>>
>>> This is precisely what I thought too. I had 3 peers with full feed
>>> configured with the soft-reconfig inboud command. I figured I'd save on
>>> ram
>>> by removing it but couldn't see any difference afterwards...
>>>
>>> Full feeds were on 12.2(33)SXI. I labbed it up under 12.4(15)T.
>>>
>>> Does anybody have any experience with this feature?
>>>
>>> Eric
>>>
>>> On Sat, Feb 26, 2011 at 4:03 PM, Fabian Pucciarelli <fabiangp_at_gmail.com
>>> >wrote:
>>>
>>> > My understanding is that when you enable the feature you need enough
>>> RAM to
>>> > hold 2x the amount of nlri received from that neighbor. If you receive
>>> > internet full tables you would require a lot of memory, if it is a
>>> small
>>> > table you don't require much. I only use the feature to be able to use
>>> show
>>> > ip bgp neighbor x received and compare it to show ip bgp neighbor x
>>> routes
>>> > when testing inbound filters.
>>> >
>>> > F.
>>> > On Feb 25, 2011 2:22 PM, "eric_at_linux.ca" <eric.lauriault_at_gmail.com>
>>> wrote:
>>> > > Hi all,
>>> > >
>>> > > For a long time, word on the street was that "soft-reconfig inbound"
>>> > takes
>>> > > quite a bit of memory and is not as useful as it once was since most
>>> BGP
>>> > > implementations have added route refresh capabilities. I've also read
>>> > that
>>> > > "the code has been optimized and that the memory impact is
>>> insignificant
>>> > > nowdays". Kinda contradictory. I can't seem to find much information
>>> on
>>> > the
>>> > > exact cost in memory of this feature.
>>> > >
>>> > > For some time, we had this feature enabled on a few peers with full
>>> > feeds.
>>> > > One would think that by removing it, we would save quite a bit of
>>> ram. We
>>> > > didn't see any quantifiable difference in RAM usage.
>>> > >
>>> > > I've tried measuring the exact cost of this feature in a lab
>>> environment
>>> > but
>>> > > the following show commands weren't very helpful :
>>> > >
>>> > > show proc mem | i BGP : the bgp processes seem to take the exact same
>>> > amount
>>> > > of RAM
>>> > > show ip bgp sum : the memory and path/bestpath fields seems to be
>>> exactly
>>> > > the same, whether the command is there or not
>>> > > show ip bgp neigh : can't find anything there either
>>> > >
>>> > > Does anybody have any clear understanding of the memory costs of the
>>> > > "soft-reconfiguration inbound" command?
>>> > >
>>> > > Regards,
>>> > >
>>> > > Eric Lauriault, #27521
>>> > >
>>> > >
>>> > > Blogs and organic groups at http://www.ccie.net
>>> > >
>>> > >
>>> _______________________________________________________________________
>>> > > Subscription information may be found at:
>>> > > http://www.groupstudy.com/list/CCIELab.html
>>>
>>>
>>> Blogs and organic groups at http://www.ccie.net
>>>
>>> _______________________________________________________________________
>>> Subscription information may be found at:
>>> http://www.groupstudy.com/list/CCIELab.html

Blogs and organic groups at http://www.ccie.net
Received on Sun Feb 27 2011 - 20:01:18 ART