RE: Internet Traffic load balancing

I did some basic tests on this, and Marko I think what you said before is correct. If the recursive routes point to the same CEF adjacency, it's as if they are just one route. You could simply disable CEF to fix this, but that's not really feasible unless the flow volume is really low. Otherwise with point-to-point links I don't think there's any way around this, but with multipoint links you can force it to have multiple separate CEF adjacencies per route. Here's a working example that I set up that results in a load balancing ratio of 4:1.

SW1#
interface FastEthernet0/3
 ip address 155.28.37.7 255.255.255.0
!
interface Vlan67
 ip address 169.254.0.1 255.255.255.0 secondary
 ip address 155.28.67.7 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 155.28.37.3
ip route 0.0.0.0 0.0.0.0 155.28.67.6
!
! Add additional weighting to Vlan67 exit point
!
ip route 0.0.0.0 0.0.0.0 169.254.0.2
ip route 0.0.0.0 0.0.0.0 169.254.0.3
ip route 0.0.0.0 0.0.0.0 169.254.0.4
!
! Resolve link-local addresses to layer 2 gateway on Vlan67
!
arp 169.254.0.4 0019.56d4.f922 ARPA
arp 169.254.0.3 0019.56d4.f922 ARPA
arp 169.254.0.2 0019.56d4.f922 ARPA

The result of this is that the layer 2 address of the primary gateway (155.28.67.6) has four CEF adjacencies associated with it.

Rack28SW1#show ip cef 0.0.0.0 0.0.0.0 internal
0.0.0.0/0, epoch 2, RIB[S], refcount 7, per-destination sharing
  sources: RIB, D/N, DRH
  feature space:
   Broker: linked
  subblocks:
   DefNet source: 0.0.0.0/0
  ifnums:
   FastEthernet0/3(469): 155.28.37.3
   Vlan67(1031): 155.28.67.6, 169.254.0.2, 169.254.0.3, 169.254.0.4
  path 0335D56C, path list 03351AEC, share 1/1, type recursive nexthop, for IPv4, flags resolved
  recursive via 155.28.37.3[IPv4:Default], fib 033A2748, 1 terminal fib
    path 0335DFD8, path list 03351FAC, share 1/1, type adjacency prefix, for IPv4
    attached to FastEthernet0/3, adjacency IP adj out of FastEthernet0/3, addr 155.28.37.3 03A81E40
  path 0335D5E0, path list 03351AEC, share 1/1, type recursive nexthop, for IPv4, flags resolved
  recursive via 155.28.67.6[IPv4:Default], fib 033A2E30, 1 terminal fib
    path 0335E304, path list 033521C0, share 1/1, type adjacency prefix, for IPv4
    attached to Vlan67, adjacency IP adj out of Vlan67, addr 155.28.67.6 03A821C0
  path 0335D654, path list 03351AEC, share 1/1, type recursive nexthop, for IPv4, flags resolved
  recursive via 169.254.0.2[IPv4:Default], fib 033A2390, 1 terminal fib
    path 0335DE08, path list 03351E7C, share 1/1, type adjacency prefix, for IPv4
    attached to Vlan67, adjacency IP adj out of Vlan67, addr 169.254.0.2 03A80940
  path 0335D7B0, path list 03351AEC, share 1/1, type recursive nexthop, for IPv4, flags resolved
  recursive via 169.254.0.3[IPv4:Default], fib 033A1B98, 1 terminal fib
    path 0335D980, path list 03351BD0, share 1/1, type adjacency prefix, for IPv4
    attached to Vlan67, adjacency IP adj out of Vlan67, addr 169.254.0.3 03A80780
  path 0335D824, path list 03351AEC, share 1/1, type recursive nexthop, for IPv4, flags resolved
  recursive via 169.254.0.4[IPv4:Default], fib 033A1E40, 1 terminal fib
    path 0335D90C, path list 03351B84, share 1/1, type adjacency prefix, for IPv4
    attached to Vlan67, adjacency IP adj out of Vlan67, addr 169.254.0.4 03A805C0
  output chain:
    loadinfo 0333E04C, per-session, 5 choices, flags 0003, 5 locks
    flags: Per-session, for-rx-IPv4
    15 hash buckets
      < 0 > IP adj out of FastEthernet0/3, addr 155.28.37.3 03A81E40
      < 1 > IP adj out of Vlan67, addr 155.28.67.6 03A821C0
      < 2 > IP adj out of Vlan67, addr 169.254.0.2 03A80940
      < 3 > IP adj out of Vlan67, addr 169.254.0.3 03A80780
      < 4 > IP adj out of Vlan67, addr 169.254.0.4 03A805C0
      < 5 > IP adj out of FastEthernet0/3, addr 155.28.37.3 03A81E40
      < 6 > IP adj out of Vlan67, addr 155.28.67.6 03A821C0
      < 7 > IP adj out of Vlan67, addr 169.254.0.2 03A80940
      < 8 > IP adj out of Vlan67, addr 169.254.0.3 03A80780
      < 9 > IP adj out of Vlan67, addr 169.254.0.4 03A805C0
      <10 > IP adj out of FastEthernet0/3, addr 155.28.37.3 03A81E40
      <11 > IP adj out of Vlan67, addr 155.28.67.6 03A821C0
      <12 > IP adj out of Vlan67, addr 169.254.0.2 03A80940
      <13 > IP adj out of Vlan67, addr 169.254.0.3 03A80780
      <14 > IP adj out of Vlan67, addr 169.254.0.4 03A805C0
    Subblocks:
     None

I'm going to do a more detailed write-up on this shortly and I'll post a link to it when it's done.

Brian McGahan, CCIE #8593 (R&S/SP/Security)
bmcgahan_at_INE.com
 
Internetwork Expert, Inc.
http://www.INE.com

-----Original Message-----
From: Marko Milivojevic [mailto:markom_at_ipexpert.com]
Sent: Tuesday, May 03, 2011 12:18 PM
To: Carlos G Mendioroz
Cc: Brian McGahan; masroor ali; Cisco certification
Subject: Re: Internet Traffic load balancing

This is one of the most amazing threads on Group Study in a long while
:-). I just had to say it!

--
Marko Milivojevic - CCIE #18427
Senior Technical Instructor - IPexpert
FREE CCIE training: http://bit.ly/vLecture
Mailto: markom_at_ipexpert.com
Telephone: +1.810.326.1444
Web: http://www.ipexpert.com/
On Tue, May 3, 2011 at 09:16, Carlos G Mendioroz <tron_at_huapi.ba.ar> wrote:
> Marko,
> I made a small test and the thing makes for a lot of fun (and food for
> thought for a R/S expert :)
>
> I have one generator (ostinato), sending steady traffic to 10 different
> addresses, 1.1.1.1 .. 10.10.10.10 (@1000 pps) to a router R1 which has
> 2 serials connected to router R2 acting as a sink.
> (Actually sending pings, so the sink is answering back)
>
> Serial links are 20.1.1.0/30 and 20.2.2.0/30, generated 10 statics to
> 0.0.0.0/0 via 169.254.1.1 .. 7 and 169.254.2.1 .. 3, and respective
> recursive routes to s0/0 and s0/1.
>
> SNMP in R1 shows more or less balanced traffic :(
> Hmm, I got 2 routes out (so ended with 8:1) and all traffic went to one
> interface! What ? Added a second route and I got a 5:2 relation.
> Aha. The 10 different destinations are not getting into 10 different
> CEF hash buckets!
>
> R1 is a 2610XM running 12.4.9T7.
> I played with "ip cef load-sharing algorithm" and the load changes.
>
> So my readings:
>
> works ? yes. (unequal load balancing using CEF, seeded by statics)
> is obvious how ? Not at all.
>
> This version does not has 16 fixed buckets as Ivan documents:
>
> R1#sh ip cef 0.0.0.0 0.0.0.0 internal
> *Jul B 1 12:30:36.866: %SYS-5-CONFIG_I: Configured from console by console
> 0.0.0.0/0, version 50, epoch 0, per-destination sharing
> 0 packets, 0 bytes
> B via 169.254.2.2, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.2.2.2, Serial0/1 via 169.254.2.2/32
> B  B valid adjacency
> B via 169.254.2.1, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.2.2.2, Serial0/1 via 169.254.2.1/32
> B  B valid adjacency
> B via 169.254.1.7, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.1.1.2, Serial0/0 via 169.254.1.7/32
> B  B valid adjacency
> B via 169.254.1.6, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.1.1.2, Serial0/0 via 169.254.1.6/32
> B  B valid adjacency
> B via 169.254.1.5, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.1.1.2, Serial0/0 via 169.254.1.5/32
> B  B valid adjacency
> B via 169.254.1.4, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.1.1.2, Serial0/0 via 169.254.1.4/32
> B  B valid adjacency
> B via 169.254.1.3, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.1.1.2, Serial0/0 via 169.254.1.3/32
> B  B valid adjacency
> B via 169.254.1.2, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.1.1.2, Serial0/0 via 169.254.1.2/32
> B  B valid adjacency
> B via 169.254.1.1, 0 dependencies, recursive
> B  B traffic share 1
> B  B next hop 20.1.1.2, Serial0/0 via 169.254.1.1/32
> B  B valid adjacency
>
> B 0 packets, 0 bytes switched through the prefix
> B tmstats: external 0 packets, 0 bytes
> B  B  B  B  B  internal 0 packets, 0 bytes
> B Load distribution: 0 1 2 3 4 5 6 7 8 (refcount 1)
>
> B Hash B OK B Interface B  B  B  B  B  B  B  B  Address B  B  B  B  Packets
> B 1 B  B  Y B  Serial0/1 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B 2 B  B  Y B  Serial0/1 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B 3 B  B  Y B  Serial0/0 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B 4 B  B  Y B  Serial0/0 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B 5 B  B  Y B  Serial0/0 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B 6 B  B  Y B  Serial0/0 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B 7 B  B  Y B  Serial0/0 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B 8 B  B  Y B  Serial0/0 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B 9 B  B  Y B  Serial0/0 B  B  B  B  B  B  B  B  point2point B  B  B  B  B  0
> B refcount 4318, covered prefixes:
> B  B 169.254.2.3/32
> B  B 169.254.2.2/32
> B  B 169.254.2.1/32
> B  B 169.254.1.7/32
> B  B 169.254.1.6/32
> B  B 169.254.1.5/32
> B  B 169.254.1.4/32
> B  B 169.254.1.3/32
> B  B 169.254.1.2/32
> B  B 169.254.1.1/32
> B  B 172.30.0.0/16
> B  B 20.2.2.0/30
> B  B 20.1.1.0/30
> B  B 224.0.0.0/4
> B  B 255.255.255.255/32
> B  B 0.0.0.0/32
> R1#
>
> -Carlos
>
>
>
> Marko Milivojevic @ 02/05/2011 22:39 -0300 dixit:
>>
>> On Mon, May 2, 2011 at 17:45, Carlos G Mendioroz <tron_at_huapi.ba.ar> wrote:
>>>
>>> This was generated from the switching point ? I.e. it was generated by
>>> the
>>> router having the 5:2 statics ?
>>>
>>> I would test it from a neighbour, so CEF is used for sure. Local trafic
>>> is process switched most probably.
>>
>> Those are valid points, but are contrary to what was originally
>> expected. My argument was that CEF would break this, but I honestly
>> expected that it would work when process-switched. It doesn't seem to.
>> I will add 3rd router behind R2 and re-run the test.
>>
>> --
>> Marko Milivojevic - CCIE #18427
>> Senior Technical Instructor - IPexpert
>>
>> FREE CCIE training: http://bit.ly/vLecture
>>
>> Mailto: markom_at_ipexpert.com
>> Telephone: +1.810.326.1444
>> Web: http://www.ipexpert.com/
>>
>>
>> Blogs and organic groups at http://www.ccie.net
>>
>> _______________________________________________________________________
>> Subscription information may be found at:
>> http://www.groupstudy.com/list/CCIELab.html
>>
>>
>>
>>
>>
>>
>>
>
> --
> Carlos G Mendioroz B <tron_at_huapi.ba.ar> B LW7 EQI B Argentina
Blogs and organic groups at http://www.ccie.net