Re: OT: PFR Internet Inbound/Outbound LB

Dear all,

I would appreciate if everyone share their opinion about the below
parameters for an ISP environment having sensitive customers:

Global Settings:
  trace probe delay 1000
  exit holddown time 60 secs, time remaining 0

Default Policy Settings:
  backoff 150 150 150
  delay relative 50
  holddown 200
  periodic 150
  probe frequency 56
  number of jitter probe packets 100
  mode route control
  mode monitor both
  mode select-exit good
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  resolve utilization priority 1 variance 2
  resolve range priority 2 variance 0

Learn Settings:
  current state : STARTED
  time remaining in current state : 330 seconds
  throughput
  no delay
  inside bgp
  monitor-period 5
  periodic-interval 5
  aggregation-type prefix-length 24
  prefixes 400 appls 400
  expire after time 30

Best Regards,
*Mohammad Moghaddas*

On Fri, Feb 14, 2014 at 12:07 PM, Mohammad Moghaddas <moghaddas.it_at_gmail.com
> wrote:

> I found this, what could be the cause?
>
> show pfr mas traffic-class performance inside
>
> Traffic-class: (inside)
> Destination Prefix : x.x.64.0/18 Source Prefix : N/A
> Destination Port : N/A Source Port : N/A
> DSCP : N Protocol : N/A
> Application Name: : N/A
>
> General:
> Control State : Not Controlled
> Traffic-class status : DISABLED due to unknown reason
> Current Exit : BR Unknown interface Unknown, Tie
> breaker was None
> Time on current exit : 0d 0:0:0
> Time remaining in current state : 0 seconds
> Traffic-class type : Learned
> Improper config : None
>
> Last Out-of-Policy event:
> No Out-of-Policy Event
>
> Average Passive Performance Current Exit: (Average for last 5 minutes)
> Unreachable : 0% -- Threshold: 50%
> Delay : 0% -- Threshold: 50%
> Loss : 0% -- Threshold: 10%
> Egress BW : 0 kbps
> Ingress BW : 0 kbps
> Time since last update : 0d 0:0:0
> .....
>
>
> Best Regards,
> *Mohammad Moghaddas*
>
>
> On Fri, Feb 14, 2014 at 11:17 AM, Mohammad Moghaddas <
> moghaddas.it_at_gmail.com> wrote:
>
>> Dear Tony,
>>
>> thanks for responding.
>> The cause of DOWN status is because I've pasted the info after shutting
>> PFR down.
>> All the traffic is pure internet (all the exits), and as I mentioned
>> before, using PBR customers are routed through different exits, but when
>> one exit become unavailable, EEM changes the configuration (ip sla+track).
>> So there was no need to separate them in different VRFs.
>> There is no ip sla responder, the tcp-connect probe are checking google,
>> yahoo, etc on port 80 from different exits.
>> Inbound Internet optimization is the most important part for me. I know
>> that PFR should prepend the AS-PATH to change the entrance, but it is not
>> behaving so. Is is only doing STATIC routes which affects Outbound traffic.
>> I should note that I've tried removing the PBR and also route-maps
>> assigned to Exit BGP peers, but nothing changed. I think my first post has
>> more complete info for you than this one.
>> I've "no shut" pfr and you find the relative info below:
>>
>> show pfr master:
>> OER state: ENABLED and ACTIVE
>> Conn Status: SUCCESS, PORT: 3949
>> Version: 3.1
>> Number of Border routers: 1
>> Number of Exits: 5
>> Number of monitored prefixes: 290 (max 5000)
>> Max prefixes: total 5000 learn 2500
>> Prefix count: total 290, learn 290, cfg 0
>> PBR Requirements met
>> Nbar Status: Inactive
>>
>> Border Status UP/DOWN AuthFail Version DOWN
>> Reason
>> 172.31.255.14 ACTIVE UP 00:07:31 0 3.1
>>
>> OER master in special monitor mode
>> ......
>>
>> !
>>
>> show pfr border active-p
>> .....
>> Type Target TPort Source Interface Att
>> Comps
>> DSCP
>> echo 213.79.125.122 N 188.75.64.21 PO8/1/0 1
>> 1
>> 0
>> echo 213.79.125.122 N 188.75.64.21 Tu108 1
>> 0
>> 0
>> echo 213.79.125.122 N 188.75.64.21 Tu101 1
>> 1
>> 0
>> echo 213.79.125.122 N 188.75.64.21 Gi8/0/0 1
>> 1
>> 0
>> echo 213.79.125.122 N 188.75.64.21 Tu105 1
>> 1
>> 0
>> ......
>> !
>>
>> show pfr master traffi
>> ....
>>
>> --------------------------------------------------------------------------------
>> 37.32.34.0/24 N N N N N N
>>
>> # INPOLICY @5 172.31.255.14 PO8/1/0
>> STATIC
>> U U 0 0 10420 10557 11
>> 9
>> 13 11 0 0 N N N
>> N
>>
>> 94.101.185.0/24 N N N N N N
>>
>> # INPOLICY @21 172.31.255.14 Gi8/0/0
>> STATIC
>> U U 0 0 4077 5430 17
>> 15
>> 12 13 0 0 N N N
>> N
>>
>> 94.201.94.128/30 N N N N N N
>>
>> # DEFAULT* @25 172.31.255.14 Tu105
>> U
>> 313 313 0 0 102311 96658 57
>> 0
>> U U 1000000 1000000 N N N
>> N
>>
>> 176.9.63.104/30 N N N N N N
>>
>> # INPOLICY @42 172.31.255.14 PO8/1/0
>> STATIC
>> U U 0 0 0 0 0
>> 0
>> 132 132 0 0 N N N
>> N
>> 178.32.55.52/30 N N N N N N
>>
>> # HOLDDOWN @155 172.31.255.14 Gi8/0/0
>> STATIC
>> U U 0 0 0 0 1
>> 1
>> 131 131 0 0 N N N
>> N
>> .....
>> !
>>
>> show pfr master traffi inside
>> ....
>>
>> --------------------------------------------------------------------------------
>> x.x.64.0/18 N N N N N N
>>
>> DEFAULT* 0 U
>> U
>>
>> x.x.112.0/23 N N N N N N
>>
>> DEFAULT* 0 U
>> U
>>
>> x.x.114.0/23 N N N N N N
>>
>> DEFAULT* 0 U
>> U
>>
>> x.x.76.0/23 N N N N N N
>>
>> DEFAULT* 0 U
>> U
>>
>> Best Regards,
>> *Mohammad Moghaddas*
>>
>>
>> On Fri, Feb 14, 2014 at 4:13 AM, Tony Singh <mothafungla_at_gmail.com>wrote:
>>
>>>
>>>
>>> Border Status UP/DOWN AuthFail Version DOWN
>>> Reason
>>> 172.31.255.14 INACTIVE DOWN 0 3.1
>>>
>>> That's not good for a start, second why are your customer routes in the
>>> same routing table sounds like you have no security policies tut tut
>>>
>>> can you post
>>>
>>> show pfr master
>>> show pfr master traffic-class
>>> sh run | s key-chain
>>>
>>> On both BR's
>>>
>>> Is the GRE tunnel up/up between the BR's
>>>
>>> The major 3. number must match between your MC and BR the minor .1 on MC
>>> must be greater or equal to the BR's minor version
>>>
>>> For echo probe you don't need ip sla responder for the other tcp-connect
>>> operations you do on the remote side
>>>
>>> --
>>> BR
>>>
>>> Tony
>>>
>>> > On 13 Feb 2014, at 13:46, Mohammad Moghaddas <moghaddas.it_at_gmail.com>
>>> wrote:
>>> >
>>> > Hi.
>>> >
>>> > I hope you are all doing well, and I'm sorry for posting such a long
>>> OT.
>>> > Straight to the issue, we have one 7609S which its IOS is 15.1(3)S. I
>>> > should note that this an ISP environment and this router has 15
>>> private IX
>>> > peers, and 5 Exit links.
>>> > I've configured the router being MC and BR the same time, 1 Internal
>>> > interface, and 5 External interface.
>>> > Each exit link has specific customers, we have separated each link's
>>> > customers using ACL. When customer's TX traffic reaches the Internal
>>> > interface, they are routed using PBR (default next-hop) to their
>>> specific
>>> > exit link. Also these ACLs are referenced in a route-map assigned to
>>> each
>>> > exit BGP peer, so we only advertise the customers to their specific
>>> exit
>>> > BGP peer.
>>> > We have categorized our BGP peers in 3 template peer-policy.
>>> >
>>> > *The issue is that, I see PFR configuring /30 STATIC routes to exit
>>> links
>>> > (it should be /24), and much more important for me, no inbound
>>> optimization
>>> > is happening.*
>>> >
>>> > Below you will find some partial logging plus the configurations.
>>> > And I'm again sorry for such long post.
>>> >
>>> > Feb 13 16:41:43: %OER_MC-5-NOTICE: Uncontrol Prefix 85.133.140.168/30,
>>> > Couldn't find the best exit
>>> > Feb 13 16:41:43: %OER_MC-5-NOTICE: Uncontrol Prefix 85.133.140.168/30,
>>> > Couldn't choose exit in prefix timeout
>>> > Feb 13 16:41:43: %OER_MC-5-NOTICE: Range Entrance OOP BR
>>> 172.31.255.14, i/f
>>> > Tu108, percent 100. Other BR 172.31.255.14, i/f Gi8/0/0 percent 15
>>> > Feb 13 16:41:43: %OER_MC-5-NOTICE: Load Entrance OOP BR 172.31.255.14,
>>> i/f
>>> > Tu108, load 33000 policy 31350
>>> > Feb 13 16:41:43: %OER_MC-5-NOTICE: Entrance 172.31.255.14 intf Tu108
>>> OOP,
>>> > Tx BW 24, Rx BW 33000, Tx Load 0, Rx Load 100
>>> > Feb 13 16:41:43: %OER_MC-5-NOTICE: Uncontrol Prefix 220.98.114.8/30,
>>> > Couldn't find the best exit
>>> > Feb 13 16:41:43: %OER_MC-5-NOTICE: Uncontrol Prefix 220.98.114.8/30,
>>> > Couldn't choose exit in prefix timeout
>>> > Feb 13 16:41:46: %OER_MC-5-NOTICE: Uncontrol Prefix 217.169.166.40/30,
>>> > Couldn't choose exit in prefix timeout
>>> > Feb 13 16:41:48: %OER_MC-5-NOTICE: Route changed Prefix
>>> 188.253.53.96/30,
>>> > BR 172.31.255.14, i/f Gi8/0/0, Reason Utilization, OOP Reason Timer
>>> Expired
>>> >
>>> > route-map CHNG_GW permit 10
>>> > description ***CUST1 through EXIT1***
>>> > match ip address CUST1
>>> > set ip default next-hop 10.30.148.169
>>> > route-map CHNG_GW permit 11
>>> > description ****CUST2 through EXIT2****
>>> > match ip address CUST2
>>> > set ip default next-hop 172.16.108.2
>>> > route-map CHNG_GW permit 12
>>> > description ****CUST3 through EXIT3****
>>> > match ip address CUST3
>>> > set ip default next-hop 172.16.101.2
>>> > route-map CHNG_GW permit 13
>>> > description ****CUST4 through EXIT2****
>>> > match ip address CUST4
>>> >
>>> > !! All other customers are routed using the PRIMARY default route. !!
>>> >
>>> > ip route 0.0.0.0 0.0.0.0 192.168.64.1 name PRIMARY
>>> > ip route 0.0.0.0 0.0.0.0 10.30.148.169 5 name PFR
>>> > ip route 0.0.0.0 0.0.0.0 172.16.101.2 6 name PFR
>>> > ip route 0.0.0.0 0.0.0.0 172.16.105.2 7 name PFR
>>> > ip route 0.0.0.0 0.0.0.0 172.16.108.2 8 name PFR
>>> >
>>> > template peer-policy CUST_BGP
>>> > route-map BGP_CUST_NO-OUT out
>>> > default-originate
>>> > soft-reconfiguration inbound
>>> > send-community both
>>> > exit-peer-policy
>>> > !
>>> > template peer-policy BW_UPLINKS
>>> > prefix-list ISP_IX-in in
>>> > next-hop-self all
>>> > soft-reconfiguration inbound
>>> > send-community both
>>> > exit-peer-policy
>>> > !
>>> > template peer-policy IX
>>> > route-map IX_BGP-OUT out
>>> > prefix-list ISP_IX-in in
>>> > next-hop-self all
>>> > soft-reconfiguration inbound
>>> > send-community both
>>> >
>>> > pfr master
>>> > policy-rules PFR_BGP
>>> > max-range-utilization percent 80
>>> > logging
>>> > !
>>> > border 172.31.255.14 key-chain OER
>>> > interface GigabitEthernet8/0/0 external
>>> > max-xmit-utilization percentage 95
>>> > maximum utilization receive percentage 95
>>> > interface Tunnel101 external
>>> > max-xmit-utilization percentage 95
>>> > maximum utilization receive percentage 95
>>> > interface Tunnel108 external
>>> > max-xmit-utilization percentage 95
>>> > maximum utilization receive percentage 95
>>> > interface Tunnel105 external
>>> > max-xmit-utilization percentage 95
>>> > maximum utilization receive percentage 95
>>> > interface POS8/1/0 external
>>> > max-xmit-utilization percentage 95
>>> > maximum utilization receive percentage 95
>>> > interface GigabitEthernet5/1 internal
>>> > !
>>> > learn
>>> > throughput
>>> > inside bgp
>>> > periodic-interval 0
>>> > monitor-period 1
>>> > prefixes 200 applications 200
>>> > expire after time 30
>>> > max range receive percent 80
>>> > backoff 150 150
>>> > mode route control
>>> > mode monitor fast
>>> > periodic 150
>>> > no resolve delay
>>> > no resolve range
>>> > !
>>> > active-probe tcp-conn 216.239.32.20 target-port 80
>>> > active-probe tcp-conn 216.239.32.20 target-port 443
>>> > active-probe echo 4.2.2.4
>>> > active-probe echo 8.8.8.8
>>> > active-probe tcp-conn 173.194.34.53 target-port 443
>>> > active-probe tcp-conn 46.228.47.114 target-port 80
>>> > active-probe echo 4.2.2.1
>>> > active-probe echo 8.8.4.4
>>> > active-probe echo 4.2.2.2
>>> > pfr border
>>> > local Loopback17231255
>>> > master 172.31.255.14 key-chain OER
>>> > active-probe address source interface GigabitEthernet5/1
>>> > pfr-map PFR_BGP 10
>>> > match pfr learn inside
>>> > set mode route control
>>> > set mode monitor passive
>>> > set resolve utilization priority 1 variance 10
>>> > no set resolve delay
>>> > no set resolve range
>>> >
>>> > show pfr master:
>>> > OER state: ENABLED and INACTIVE
>>> > Conn Status: SUCCESS, PORT: 3949
>>> > Version: 3.1
>>> > Number of Border routers: 1
>>> > Number of Exits: 5
>>> > Number of monitored prefixes: 0 (max 5000)
>>> > Max prefixes: total 5000 learn 2500
>>> > Prefix count: total 0, learn 0, cfg 0
>>> > PBR Requirements met
>>> > Nbar Status: Inactive
>>> >
>>> > Border Status UP/DOWN AuthFail Version DOWN
>>> Reason
>>> > 172.31.255.14 INACTIVE DOWN 0 3.1
>>> >
>>> > OER master in special monitor mode
>>> >
>>> > Global Settings:
>>> > max-range-utilization percent 80 recv 80
>>> > rsvp post-dial-delay 0 signaling-retries 1
>>> > mode route metric bgp local-pref 5000
>>> > mode route metric static tag 5000
>>> > trace probe delay 1000
>>> > logging
>>> > exit holddown time 60 secs, time remaining 0
>>> >
>>> > Default Policy Settings:
>>> > backoff 150 150 150
>>> > delay relative 50
>>> > holddown 300
>>> > periodic 150
>>> > probe frequency 56
>>> > number of jitter probe packets 100
>>> > mode route control
>>> > mode monitor fast
>>> > mode select-exit good
>>> > loss relative 10
>>> > jitter threshold 20
>>> > mos threshold 3.60 percent 30
>>> > unreachable relative 50
>>> > resolve utilization priority 13 variance 20
>>> >
>>> > Learn Settings:
>>> > current state : DISABLED
>>> > time remaining in current state : 0 seconds
>>> > throughput
>>> > no delay
>>> > inside bgp
>>> > monitor-period 5
>>> > periodic-interval 5
>>> > aggregation-type prefix-length 24
>>> > prefixes 200 appls 200
>>> > expire after time 30
>>> >
>>> >
>>> > show pfr master policy:
>>> > HT-CoreRT(config-pfr-mc)#do s pfr mas pol
>>> > Default Policy Settings:
>>> > backoff 150 150 150
>>> > delay relative 50
>>> > holddown 300
>>> > periodic 150
>>> > probe frequency 56
>>> > number of jitter probe packets 100
>>> > mode route control
>>> > mode monitor fast
>>> > mode select-exit good
>>> > loss relative 10
>>> > jitter threshold 20
>>> > mos threshold 3.60 percent 30
>>> > unreachable relative 50
>>> > resolve utilization priority 13 variance 20
>>> > oer-map PFR_BGP 10
>>> > sequence no. 8444249301975040, provider id 1, provider priority 30
>>> > host priority 0, policy priority 10, Session id 0
>>> > match oer learn inside
>>> > backoff 150 150 150
>>> > delay relative 50
>>> > holddown 300
>>> > periodic 150
>>> > probe frequency 56
>>> > number of jitter probe packets 100
>>> > *mode route control
>>> > *mode monitor passive
>>> > mode select-exit good
>>> > loss relative 10
>>> > jitter threshold 20
>>> > mos threshold 3.60 percent 30
>>> > unreachable relative 50
>>> > next-hop not set
>>> > forwarding interface not set
>>> > *resolve utilization priority 1 variance 10
>>> >
>>> > Best Regards,
>>> > *Mohammad Moghaddas*
>>> >
>>> >
>>> > Blogs and organic groups at http://www.ccie.net
>>> >
>>> > _______________________________________________________________________
>>> > Subscription information may be found at:
>>> > http://www.groupstudy.com/list/CCIELab.html

Blogs and organic groups at http://www.ccie.net
Received on Mon Feb 17 2014 - 12:57:51 ART