Date created: Thursday, April 21, 2016 10:18:19 AM. Last modified: Wednesday, June 29, 2022 8:51:57 AM
LAG, ECMP, MPLS-TE: Tech Overview
References:
https://supportforums.cisco.com/document/69496/etherchannel-loadbalancing-catalyst-switches
http://www.cisco.com/c/en/us/td/docs/switches/metro/me3600x_3800x/software/release/15-3_1_S/configuration/guide/3800x3600xscg/swethchl.html#pgfId-1116754
http://www.cisco.com/c/en/us/td/docs/switches/metro/me3600x_3800x/software/release/15-3_3_S/configuration/guide/3800x3600xscg/swmplsloadbalancing.html
BRKSPG-2904 - ASR9000 Selected Topics and Troubleshooting (2016)
https://drive.google.com/drive/folders/0B5Q6qCRMe89_ZThNbWdDUWpyR2c?usp=sharing
https://www.cisco.com/c/en/us/td/docs/routers/asr920/configuration/guide/mpls/mp-l3-vpns-xe-3s-asr920-book/mp-l3-vpns-xe-3s-asr920-book_chapter_0100.html
Contents:
- LAGs, Ether-Channels, Bundles
- IP & MPLS Un/Equal-Cost Multi-Path Routing
- MPLS FAT
- MPLS-TE
See also: ASR9000 Load-Balancing.
LAGs, Ether-Channels, Bundles:
Different devices support different load balancing algorithms so they must be checked before a specific hashing function can be chosen.
! 7606-S, RSP7203XCL on 15.2(4)S4
7606-S-RSP720-3XCL#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-ip vlan included
mpls label-ip
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination IP address
IPv6: Source XOR Destination IP address
MPLS: Label or IP
7606-S-RSP720-3XCL#remote command module 6 show platform hardware superman config | i Hash|HASH
Hash 0: x^12+x^8+x^6+x^5+1
Hash 1: x^12+x^9+x^8+x^6+x^5+x^4+1
NEW_MIX_HASH_EN: Enable
VLAN_IP_HASH_EN: Enable
BNDL_HASH: Source XOR Destination
RBH_HASH_SEL: x^17 + x^4 + 1
HASH_CNFG: 0
HASH_INPUT_SEL: Lyra mode
7606-S-RSP720-3XCL(config)#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
dst-mixed-ip-port Dst IP Addr and TCP/UDP Port
dst-port Dst TCP/UDP Port
mpls Load Balancing for MPLS packets
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-dst-mixed-ip-port Src XOR Dst IP Addr and TCP/UDP Port
src-dst-port Src XOR Dst TCP/UDP Port
src-ip Src IP Addr
src-mac Src Mac Addr
src-mixed-ip-port Src IP Addr and TCP/UDP Port
src-port Src TCP/UDP Port
! Cat 6500 with Sup720
Cat6500#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-ip
mpls label-ip
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination IP address
Cat6500(config)#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
dst-mixed-ip-port Dst IP Addr and TCP/UDP Port
dst-port Dst TCP/UDP Port
mpls Load Balancing for MPLS packets
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-dst-mixed-ip-port Src XOR Dst IP Addr and TCP/UDP Port
src-dst-port Src XOR Dst TCP/UDP Port
src-ip Src IP Addr
src-mac Src Mac Addr
src-mixed-ip-port Src IP Addr and TCP/UDP Port
src-port Src TCP/UDP Port
! 6509-E with VS-SUP2T-10G on 15.1(1)SY
6509-E#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-ip enhanced
mpls label-ip
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination IP address
IPv6: Source XOR Destination IP address
MPLS: Label or IP
6509-E(config)#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
dst-mixed-ip-port Dst IP Addr and TCP/UDP Port
dst-port Dst TCP/UDP Port
mpls Load Balancing for MPLS packets
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-dst-mixed-ip-port Src XOR Dst IP Addr and TCP/UDP Port
src-dst-port Src XOR Dst TCP/UDP Port
src-ip Src IP Addr
src-mac Src Mac Addr
src-mixed-ip-port Src IP Addr and TCP/UDP Port
src-port Src TCP/UDP Port
vlan-dst-ip Vlan, Dst IP Addr
vlan-dst-mixed-ip-port Vlan, Dst IP Addr and TCP/UDP Port
vlan-src-dst-ip Vlan, Src XOR Dst IP Addr
vlan-src-dst-mixed-ip-port Vlan, Src XOR Dst IP Addr and TCP/UDP Port
vlan-src-ip Vlan, Src IP Addr
vlan-src-mixed-ip-port Vlan, Src IP Addr and TCP/UDP Port
! 4500X on 03.07.02.E
4500X#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-port
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination TCP/UDP (layer-4) port number
IPv6: Source XOR Destination TCP/UDP (layer-4) port number
4500X(config)#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
dst-port Dst TCP/UDP Port
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-dst-port Src XOR Dst TCP/UDP Port
src-ip Src IP Addr
src-mac Src Mac Addr
src-port Src TCP/UDP Port
! ME3600X running 15.3(3)S6
ME3600X#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-mac
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source MAC address
IPv4: Source MAC address
IPv6: Source MAC address (routed packets)
Source IP address (bridged packets)
ME3600X(config)#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-ip Src IP Addr
src-mac Src Mac Addr
Commands to check hash results for ether-channel/port-channel load balancing are "test etherchannel load-balance interface port-channel 1 ..." on 6500/7600, "test etherchannel load-balance interface po 1..." on ME3600X/ME3800X, "show platform software etherchannel port-channel 11 map..." on 4500X and "test etherchannel load-balance interface port-channel 1..." on 2k and 3k switches (2960/3560/3650 etc).
Looking at the 7600 output below it can bee seen in the output from “show etherchannel 1 port-channel” that Gi4/11 has load-balancing result 55 assigned and Gi4/12 has AA assigned. 0x55 in binary is 0101 0101 and 0xAA is 1010 1010. With just two links in the bundle a 50% traffic split is preferred so those two hex values match 50% of hash results to each link.
When XOR’ing two values together (such as MAC addresses) to provide the RBH (Result Bundle Hash) value, the RBH value produced is always in the range of 0-7 (the formula for doing this is very complicated). The binary for 0, 2, 4, and 6 (0x55) all XOR with the RBH value and equate to Gi4/11. If the RBH value XOR’s with 1, 3, 5 or 7 (0xAA) the result is Gi4/12.
7600#remote login switch
Trying Switch ...
Entering CONSOLE for Switch
Type "^C^C^C" to end this session
7600-sp#show etherchannel 1 summary
...
Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
1 Po1(SU) LACP Gi4/11(P) Gi4/12(P)
7600-sp#test etherchannel load-balance interface port-channel 1 mac 0000.0000.0000 0000.0000.0000
Computed RBH: 0x0
Would select Gi4/11 of Po1
7600-sp#test etherchannel load-balance interface port-channel 1 mac 0000.0000.0000 0000.0000.0001
Computed RBH: 0x1
Would select Gi4/12 of Po1
...
7600-sp#test etherchannel load-balance interface port-channel 1 mac 0000.0000.0000 0000.0000.0007
Computed RBH: 0x7
Would select Gi4/12 of Po1
7600-sp#test etherchannel load-balance interface port-channel 1 mac 0000.0000.0000 0000.0000.0008
Computed RBH: 0x8
Would select Gi4/11 of Po1
7600-sp#show etherchannel 1 port-channel
Port-channels in the group:
----------------------
Port-channel: Po1 (Primary Aggregator)
------------
Age of the Port-channel = 3d:14h:44m:43s
Logical slot/port = 14/1 Number of ports = 2
HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = LACP
Fast-switchover = disabled
Direct Load Swap = disabled
Ports in the Port-channel:
Index Load Port EC state No of bits
------+------+--------+------------------+-----------
1 55 Gi4/11 Active 4
0 AA Gi4/12 Active 4
Time since last port bundled: 3d:14h:40m:08s Gi4/12
Last applied Hash Distribution Algorithm: Fixed
In the case that 3 links are in a bundle (for example) the below output may be seen in "show interface port-channel x etherchannel". Here the bit sharing ratio is 3:3:2 which means two links have a higher probability of being utilized when compared to the third member link. Further down the table shows the bit distributions for an etherchannel with different numbers of member links:
Index Load Port EC state No of bits ------+------+------+------------------+----------- 0 49 Gi3/1 Active 3 1 92 Gi3/2 Active 3 2 24 Gi3/3 Active 2 No. of link in bundle | RBH bit distribution 8 1:1:1:1:1:1:1:1 7 2:1:1:1:1:1:1 6 2:2:1:1:1:1 5 2:2:2:1:1 4 2:2:2:2 3 3:3:2 2 4:4
6500s/7600s support having a different hashing method per line card by using the "port-channel load-balance per-module" command followed by "port-channel load-balance src-dst-port module 2" for example.
On 6500/7600s the traffic is hashed on ingress. This means that when using DFCs and per module hashing methods, traffic entering a port on module 1 then egresses module 2 will egress based on the hashing method chosen for module 1 not module 2. When mixing DFCs and CFCs then the CFC modules will use the global PFC method which can be different from the DFC enable modules.
Cat 4500 & 4500-X LAG Specifics:
4500X's use an Agg2PhyPort array to store port-channel members and create the hashing results for load-balancing. The Cat 4500 and 4500-X support up to 8 member interfaces in a port-channel. Each interface in the port-channel using one entry in the Agg2PhyPort array. For example a 4 interface bond using interfaces 2, 4, 6, and 8 would be stored in array indexes 0, 1, 2 and 3. In this case the Agg2PhyPort array is { 0:2, 1:4, 2:6, 3:8 }. Whatever load-balancing algorithm is chosen, the Cisco proprietary algorithm will crunch the tuples down to a number, for example '6' which is then modulus against the number of elements in the array (4) to give the array index, 6%4=2, array index 2 is port 6.
Cat 4500s use the same approach as the 6500s/7600s above. However for newer Cat 4500s using the newer K5 ASICs, the traditional modulus approach is altered. Normally in the case of 3 member links (for example) the traffic distribution ratio becomes 3:3:2 to each member link. On the newer ASICs only the last three bits of the load-balancing hash are used so only a value of 1-3 is produced by the hash giving equal traffic balancing to a port-channel with 3 links. The same is true for 5 member links, only the last 5 bits of the hash are used, and so on for all possible combinations of member links.
# Outputs from two WS-C4500X-16's with C4KX-NM-8 module installed in a stack...
#show platform mapping ports
Executing the command on VSS member switch role = VSS Active, id = 1
Interface Superport Subport CompactSubportId PortSet Phyport Aggport PimPhyport
Te1/2/1 4 0 48 1 108 Po31(863) 0
Te1/2/2 5 0 50 1 110 Po32(864) 1
Te1/2/3 0 0 52 0 112 Po33(865) 2
Te1/2/4 1 0 54 0 114 Po34(866) 3
Te1/2/5 12 0 56 3 116 Po4(836) 4
Te1/2/6 13 0 58 3 118 Po3(835) 5
Te1/2/7 8 0 60 2 120 Po1(833) 6
Te1/2/8 9 0 62 2 122 Po1(833) 7
Te1/1/1 60 0 16 15 12 Po11(843) 48
Te1/1/2 61 0 18 15 14 Po12(844) 49
Te1/1/3 66 0 20 17 16 Po13(845) 50
Te1/1/4 67 0 22 17 18 59 51
Te1/1/5 68 0 24 18 20 60 52
Te1/1/6 69 0 26 18 22 61 53
Te1/1/7 70 0 28 19 24 62 54
Te1/1/8 71 0 30 19 26 63 55
Te1/1/9 44 0 32 11 28 64 56
Te1/1/10 45 0 34 11 30 65 57
Te1/1/11 56 0 36 14 32 66 58
Te1/1/12 57 0 38 14 34 67 59
Te1/1/13 52 0 40 13 36 68 60
Te1/1/14 53 0 42 13 38 69 61
Te1/1/15 64 0 44 16 40 Po1(833) 62
Te1/1/16 65 0 46 16 42 Po1(833) 63
#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-port
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination TCP/UDP (layer-4) port number
IPv6: Source XOR Destination TCP/UDP (layer-4) port number
#show platform software etherchannel port-channel 1 map l4-port 1.1.1.1 100 2.2.2.2 200 | i is Te
Map port for l4-port 1.1.1.1:100, 2.2.2.2:200 is Te1/1/16(Po1)
#show platform software etherchannel port-channel 1 map l4-port 1.1.1.1 100 2.2.2.2 201 | i is Te
Map port for l4-port 1.1.1.1:100, 2.2.2.2:201 is Te1/1/15(Po1)
#show platform software etherchannel port-channel 1 map l4-port 1.1.1.1 100 2.2.2.2 202 | i is Te
Map port for l4-port 1.1.1.1:100, 2.2.2.2:202 is Te1/2/7(Po1)
...
#show platform software etherchannel port-channel 1 map l4-port 1.1.1.1 100 2.2.2.2 210 | i is Te
Map port for l4-port 1.1.1.1:100, 2.2.2.2:210 is Te1/2/8(Po1)
ME3600X/ME3800X LAG Specifics:
When using the destination MAC method these switches support load-based hashing for the destination MAC however this limits the destination member port in a bundle to be only one of four. In an eight port bundle, 4 links will never be used for any given destination MAC.
MX3600X/ME3800X(config)#port-channel load-balance ? dst-ip Dst IP Addr dst-mac Dst Mac Addr src-dst-ip Src XOR Dst IP Addr src-dst-mac Src XOR Dst Mac Addr src-ip Src IP Addr src-mac Src Mac Addr
When routing vanilla IP traffic, acting as an MPLS P node for an IP VPN or as an ingress/egress LER for a L3 MPLS VPN, and a layer 2 LAG is in the next-hop path, the LAG hashes based on the global config "port-channel load-balance" option above. If using the IP hash options IPv4 and IPv6 are both supported for src-ip/dst-ip-/src-dst-ip hashing. When using the IP hash options and MPLS labelled traffic is being transported the IP details are extracted from the L3 VPN (there is a max label depth of 3). If there are no IP details behind the MPLS label stack the EOS label value is used. The MAC options for the LAG load-balancing use the outer MAC details (so the next-hop P/PE node in an MPLS network) which are usually the same, this can cause problems.
When ingressing or egressing a L2 VPN (LER) or acting as a L2 VPN P node, all of the global options above can be used for MAC or IP hashing. When using the MAC options and when there is a control-word after the end/bottom of stack label in the L2 VPN, the inner MAC details from the L2 VPN are extracted (again, there is a max label depth of 3 excluding control-word) otherwise the outer MAC details are used (the same caveat applied about likely having the same hash result for all traffic). When using the IP hashing options the inner IP details are extracted from within the L2 VPN traffic (max label depth limit of 3 excluding the control-word). If the IP details can't be extracted the last MPLS label value is used.
Note: 10G and 1G ports can't be mixed into a LAG group.
Note: ASR920 supports a max of 64 LAG groups, each group supports a max of 8 active links and 8 standby links.
The ASR920 can hash based upon various fields when the next-hop path is over a LAG:
! IOS-XE 3.16.4S on ASR-920-4SZ-A
ASR920(config)#port-channel load-balance-hash-algo ? dst-ip Destination IP dst-mac Destination MAC src-dst-ip Source XOR Destination IP Addr src-dst-mac Source XOR Destination MAC src-dst-mixed-ip-port Source XOR Destination Port, IP addr src-ip Source IP src-mac Source MAC
The Cylon ASIC supports "per-flow" load balancing. The ASIC supports 16 hash buckets and links are mapped sequentially to these buckets based on a CRC32 hash. Non-labelled flows are hashed using destination IP, destination MAC, source XOR destination IP address, source XOR destination MAC, source IP or Source MAC as inputs.
Labelled flows can be hashed based on IP source and destination address from within the L3 VPN and inner labels for L2 VPN (P node). For PEs it depends on the device global configuration.
For L3 VPN transit traffic (P node) the default action is hash based on the outer source MAC (which can cause poor traffic distribution). For L2 VPN transit traffic (P node) the default action is to hash on the inner VC label.
For a L3 VPN LER/PE the globally configured port-channel load-balancing mode is used, the default is outer source MAC (which can cause poor traffic distribution). For a L2 VPN LER/PE the globally configured port-channel load-balancing mode is used, the default is the outer source MAC.
IP & MPLS Un/Equal-Cost Multi-Path Routing
mls ip cef load-sharing full
http://www.cisco.com/c/en/us/support/docs/ip/express-forwarding-cef/116376-technote-cef-00.html
The maximum number of load balancing paths is 8. The PFC forwards MPLS labeled packets without explicit configuration. If the packet has three labels or less and the underlying packet is IPv4, then the PFC uses the source and destination IPv4 address. If the underlying packet is not IPv4 or more than three labels are present, the PFC parses down as deep as the fifth or lowest label and uses it for hashing.
MPLS Layer 2 VPN Load Balancing: Load balancing is based on the VC label in the MPLS core if the first nibble of the MAC address in the customer Ethernet frame is not 4.
Note: Load balancing is not supported at the ingress PE for Layer 2 VPNs. Load balancing is done based on the VC label and it is pre-selected.
MPLS Layer 3 VPN Load Balancing: MPLS Layer 3 VPN load balancing is similar to basic MPLS load balancing.
As of IOS-XR 5.3.0 Typhoon and Tomahawk NPs support 64-way ECMP for OSPF/OSPFv3 and ISIS. This is tunable on the CLI by using the following commands:
-OSPF/OSPFv3:
-router ospf 100 maximum paths <1-128>
-router ospfv3 100 maximum paths <1-128>
ISIS:
-router isis 1111 address-family ipv4 unicast/mcast maximum-paths <1-64>
-router isis 1111 address-family ipv6 unicast/mcast maximum-paths <1-64>
RP/0/RSP0/CPU0:ASR9001-XR-5.3.3#show cef misc ... Platform capabilities: -------------------------- L3 loadbalancing levels: 2 L3 Hash buckets: 64 L3 recursive Hash buckets: 32 L3 Unequal cost hash buckets: 0 Forwarding stages: 2 Local label split: 3 Statistics support: No Path per Tunnel class: 8 V4 update disabled: No V6 update disabled: No LBA tuples: 1 MRAPS support: No Tunnel default class support: Yes Loadinfo choices are platform filtered Prefix filter support: host prefix only LISP ipv4/v4 decapsulation support: Yes LISP ipv4/v6 decapsulation support: Yes LISP unequal cost loadbalance support for ipv4 payload: No Fallback support for ipv4: Yes LISP ipv6/v4 decapsulation support: Yes LISP ipv6/v6 decapsulation support: Yes LISP unequal cost loadbalance support for ipv6 payload: No Fallback support for ipv6: Yes NHID (Next Hop ID) support: No Backwalk (Dependents) required: No Link MPLS IPv6 nexthop support: Yes Exclude Deag backup paths: No
Slides 53 onwards: https://clnv.s3.amazonaws.com/2017/usa/pdf/BRKSPG-2904.pdf
Test CEF load-balancing result:
# Using Layer 3 and optionally layer 4 header info:
RP/0/RSP0/CPU0:ASR9001#show cef exact-route 1.1.1.1 2.2.2.2 protocol tcp source-port 10 destination-port 20 ingress-interface te0/0/0/0
RP/0/RSP0/CPU0:ASR9001#show cef exact-route 1.1.1.1 2.2.2.2 protocol ?
Enter protocol Number or name below
AHP authentication header
EGP exterior gateway protocol
EIGP EIGRP protocol
ENCAP Encapsulation header
EON IOS CNLP
ESP IPSEC Encapsulating Security Payload
GRE GRE encapsulation
ICMP control message protocol
IDP XNS IDP
IGMP group managment protocol
IGRP IGRP protocol
IPINIP IP in IP encapsulation
L2TV3 L2TPv3 tunneling encapsulation
NOSIP KA9Q/NOS compatible IP over IP
OSPF OSPF protocols
PCP porotocol
PUP PUP protocol
RAW raw IP packet
RSVP resource reservation protocol
SCTP Stream Control Transmission
TCP TCP protocol
TP tp-4 with class negotiation
UDP UDP protocol
UTI UTI tunneling encapsulation
# Using MPLS labels and optionally layer 3 VPN payload IPs
RP/0/RSP0/CPU0:ASR9001#show mpls forwarding exact-route label 12345 ipv4 1.1.1.1 2.2.2.1
RP/0/RSP0/CPU0:ASR9001#show mpls forwarding exact-route label 12345 ?
bottom-label Bottom label
entropy-label Entropy label
ipv4 Exact path for IPv4 payload
ipv6 Exact path for IPv6 payload
ME3600X/ME3800X MPLS Specifics
When ECMP paths exist for forwarding vanilla IPv4 and IPv6 traffic and MPLS L3 VPNs the source and destination IPs are used for hashing. If the IPs can't be found (label stack too deep, 3 is the max label depth) the last label value is used. When ECMP paths existing for MPLS L2 VPNs (if the control-word is beneath the end of stack label) the inner IP source and destination IPs are used, if there aren't any (such non-IP traffic over Ethernet) then the inner source and destination MACs are used.
Layer 2 VPN Load Balancing: To get an evenly-distributed load balancing experience over virtual circuits (VC), it is recommended to have more number of VCs.
Layer 3 VPN Load Balancing: Load balancing is supported only on homogeneous paths, that is, all MPLS adjacencies or all IP adjacencies. This is because the MPLS SWAP/POP action in the LFIB also contains the next-hop rewrite details.
For L3 VPN traffic where the ASR920 is either a P node or PE node, it hashes based on the L3 VPN source and destination IP addresses (max label depth of 3). For L2 VPN traffic where the ASR920 is a P node it hashes based on the VC label and where it is a PE node it hashes based on the local label.
On the ASR920 either 4 or 8 ECMPs are supported:
ASR920(config)#platform loadbalance max-paths ? 4 4 paths, default 8 8 paths ASR920(config)#ip cef load-sharing algorithm ? include-ports Algorithm that includes layer 4 ports original Original algorithm tunnel Algorithm for use in tunnel only environments universal Algorithm for use in most environments
The "original" option hashes using only the source and destination IP address. This produces a 4 bit value as ECMP routing is only supported over 16 links. For example, a device with 4 ECMP links to another device might show the following output (the same four CEF adjacency entries are repeated 4 times to fill the 16 hash buckets):
#show ip cef 1.1.1.1/32 internal
...
16 hash buckets
< 0 > IP adj out of GigabitEthernet1/0, addr 12.0.0.2 68D03140
< 1 > IP adj out of GigabitEthernet2/0, addr 13.0.0.3 6A9B1220
< 2 > IP adj out of GigabitEthernet3/0, addr 14.0.0.4 68D02FE0
< 3 > IP adj out of GigabitEthernet4/0, addr 15.0.0.5 6A9B1380
< 4 > IP adj out of GigabitEthernet1/0, addr 12.0.0.2 68D03140
< 5 > IP adj out of GigabitEthernet2/0, addr 13.0.0.3 6A9B1220
< 6 > IP adj out of GigabitEthernet3/0, addr 14.0.0.4 68D02FE0
< 7 > IP adj out of GigabitEthernet4/0, addr 15.0.0.5 6A9B1380
< 8 > IP adj out of GigabitEthernet1/0, addr 12.0.0.2 68D03140
< 9 > IP adj out of GigabitEthernet2/0, addr 13.0.0.3 6A9B1220
< 10 > IP adj out of GigabitEthernet3/0, addr 14.0.0.4 68D02FE0
< 11 > IP adj out of GigabitEthernet4/0, addr 15.0.0.5 6A9B1380
< 12 > IP adj out of GigabitEthernet1/0, addr 12.0.0.2 68D03140
< 13 > IP adj out of GigabitEthernet2/0, addr 13.0.0.3 6A9B1220
< 14 > IP adj out of GigabitEthernet3/0, addr 14.0.0.4 68D02FE0
< 15 > IP adj out of GigabitEthernet4/0, addr 15.0.0.5 6A9B1380
In the case that there are say 5 ECMP paths they would be repeated 3 times to fill 15 hash buckets and the 16th would be empty/unused.
One can check ECMP and CEF details using the following commands:
show ip cef x.x.x.x/x internal
show ip cef exact-route y.y.y.y x.x.x.x
show platform hardware pp active feature cef database ipv4 x.x.x.x/x
show platform hardware pp active feature cef database ipv6 x::x/x
By default OSPF or ISIS might only support 4 ECMPs, to increase this configure "maximum-paths <1-32>" under "router ospf X" or "router isis X".
The "ip cef load-sharing algorithm universal" features adds a value to the hash function that is unique to this local device so that multiple devices which all have ECMP configure will hash differently to reduce link polarisation across multiple devices.
The "ip cef load-sharing algorithm tunnel" feature is supposed to provide better hashing in environments where there are just a few source and destination IPs as most traffic is tunneled (inside L2TP or GTP tunnels for example). The documentation is unclear as to how this works but it looks like it's meant for GTP and will look at the sessions inside to get additional hashing entropy.
MPLS & FAT
MPLS-TE
https://www.cisco.com/c/en/us/td/docs/switches/metro/me3600x_3800x/software/release/15-3_3_S/configuration/guide/3800x3600xscg/swmpls.html#22717
Previous page: IGP/LDP/BGP Convergence Tuning (IOS/IOS-XE)
Next page: OSPFv2 Inter-Area Filtering & Prefix-Suppression