Date created: 04/21/16 10:18:19. Last modified: 08/27/18 10:49:14

LAG, ECMP, MPLS-TE: Tech Overview

References:
https://supportforums.cisco.com/document/69496/etherchannel-loadbalancing-catalyst-switches
http://www.cisco.com/c/en/us/td/docs/switches/metro/me3600x_3800x/software/release/15-3_1_S/configuration/guide/3800x3600xscg/swethchl.html#pgfId-1116754
http://www.cisco.com/c/en/us/td/docs/switches/metro/me3600x_3800x/software/release/15-3_3_S/configuration/guide/3800x3600xscg/swmplsloadbalancing.html
BRKSPG-2904 - ASR9000 Selected Topics and Troubleshooting (2016)
https://drive.google.com/drive/folders/0B5Q6qCRMe89_ZThNbWdDUWpyR2c?usp=sharing
https://www.cisco.com/c/en/us/td/docs/routers/asr920/configuration/guide/mpls/mp-l3-vpns-xe-3s-asr920-book/mp-l3-vpns-xe-3s-asr920-book_chapter_0100.html

Contents:

 

See also: ASR9000 Load-Balancing.

LAGs, Ether-Channels, Bundles:

Different devices support different load balancing algorithms so they must be checked before a specific hashing function can be chosen.

! 7606-S, RSP7203XCL on 15.2(4)S4

7606-S-RSP720-3XCL#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-ip vlan included
        mpls label-ip

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination IP address
  IPv6: Source XOR Destination IP address
  MPLS: Label or IP


7606-S-RSP720-3XCL#remote command module 6 show platform hardware superman config | i Hash|HASH
Hash 0: x^12+x^8+x^6+x^5+1
Hash 1: x^12+x^9+x^8+x^6+x^5+x^4+1
 NEW_MIX_HASH_EN:          Enable
 VLAN_IP_HASH_EN:            Enable
 BNDL_HASH:                Source XOR Destination
 RBH_HASH_SEL:             x^17 + x^4 + 1
 HASH_CNFG:                0
 HASH_INPUT_SEL:           Lyra mode


7606-S-RSP720-3XCL(config)#port-channel load-balance ?
  dst-ip                 Dst IP Addr
  dst-mac                Dst Mac Addr
  dst-mixed-ip-port      Dst IP Addr and TCP/UDP Port
  dst-port               Dst TCP/UDP Port
  mpls                   Load Balancing for MPLS packets
  src-dst-ip             Src XOR Dst IP Addr
  src-dst-mac            Src XOR Dst Mac Addr
  src-dst-mixed-ip-port  Src XOR Dst IP Addr and TCP/UDP Port
  src-dst-port           Src XOR Dst TCP/UDP Port
  src-ip                 Src IP Addr
  src-mac                Src Mac Addr
  src-mixed-ip-port      Src IP Addr and TCP/UDP Port
  src-port               Src TCP/UDP Port



! Cat 6500 with Sup720
Cat6500#show etherchannel load-balance
 EtherChannel Load-Balancing Configuration:
 src-dst-ip
 mpls label-ip

EtherChannel Load-Balancing Addresses Used Per-Protocol:
 Non-IP: Source XOR Destination MAC address
 IPv4: Source XOR Destination IP address


Cat6500(config)#port-channel load-balance ?
 dst-ip                 Dst IP Addr
 dst-mac                Dst Mac Addr
 dst-mixed-ip-port      Dst IP Addr and TCP/UDP Port
 dst-port               Dst TCP/UDP Port
 mpls                   Load Balancing for MPLS packets
 src-dst-ip             Src XOR Dst IP Addr
 src-dst-mac            Src XOR Dst Mac Addr
 src-dst-mixed-ip-port  Src XOR Dst IP Addr and TCP/UDP Port
 src-dst-port           Src XOR Dst TCP/UDP Port
 src-ip                 Src IP Addr
 src-mac                Src Mac Addr
 src-mixed-ip-port      Src IP Addr and TCP/UDP Port
 src-port               Src TCP/UDP Port



! 6509-E with VS-SUP2T-10G on 15.1(1)SY
6509-E#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-ip enhanced
        mpls label-ip

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination IP address
  IPv6: Source XOR Destination IP address
  MPLS: Label or IP


6509-E(config)#port-channel load-balance ?
  dst-ip                      Dst IP Addr
  dst-mac                     Dst Mac Addr
  dst-mixed-ip-port           Dst IP Addr and TCP/UDP Port
  dst-port                    Dst TCP/UDP Port
  mpls                        Load Balancing for MPLS packets
  src-dst-ip                  Src XOR Dst IP Addr
  src-dst-mac                 Src XOR Dst Mac Addr
  src-dst-mixed-ip-port       Src XOR Dst IP Addr and TCP/UDP Port
  src-dst-port                Src XOR Dst TCP/UDP Port
  src-ip                      Src IP Addr
  src-mac                     Src Mac Addr
  src-mixed-ip-port           Src IP Addr and TCP/UDP Port
  src-port                    Src TCP/UDP Port
  vlan-dst-ip                 Vlan, Dst IP Addr
  vlan-dst-mixed-ip-port      Vlan, Dst IP Addr and TCP/UDP Port
  vlan-src-dst-ip             Vlan, Src XOR Dst IP Addr
  vlan-src-dst-mixed-ip-port  Vlan, Src XOR Dst IP Addr and TCP/UDP Port
  vlan-src-ip                 Vlan, Src IP Addr
  vlan-src-mixed-ip-port      Vlan, Src IP Addr and TCP/UDP Port



! 4500X on 03.07.02.E
4500X#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-port

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination TCP/UDP (layer-4) port number
  IPv6: Source XOR Destination TCP/UDP (layer-4) port number


4500X(config)#port-channel load-balance ?
  dst-ip        Dst IP Addr
  dst-mac       Dst Mac Addr
  dst-port      Dst TCP/UDP Port
  src-dst-ip    Src XOR Dst IP Addr
  src-dst-mac   Src XOR Dst Mac Addr
  src-dst-port  Src XOR Dst TCP/UDP Port
  src-ip        Src IP Addr
  src-mac       Src Mac Addr
  src-port      Src TCP/UDP Port



! ME3600X running 15.3(3)S6
ME3600X#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-mac

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source MAC address
  IPv4: Source MAC address
  IPv6: Source MAC address (routed packets)
        Source IP address (bridged packets)

ME3600X(config)#port-channel load-balance ?
  dst-ip       Dst IP Addr
  dst-mac      Dst Mac Addr
  src-dst-ip   Src XOR Dst IP Addr
  src-dst-mac  Src XOR Dst Mac Addr
  src-ip       Src IP Addr
  src-mac      Src Mac Addr

Commands to check hash results for ether-channel/port-channel load balancing are "test etherchannel load-balance interface port-channel 1 ..." on 6500/7600, "test etherchannel load-balance interface po 1..." on ME3600X/ME3800X, "show platform software etherchannel port-channel 11 map..." on 4500X and "test etherchannel load-balance interface port-channel 1..." on 2k and 3k switches (2960/3560/3650 etc).

 

6500/7600 LAG Specifics:

Looking at the 7600 output below it can bee seen in the output from “show etherchannel 1 port-channel” that Gi4/11 has load-balancing result 55 assigned and Gi4/12 has AA assigned. 0x55 in binary is 0101 0101 and 0xAA is 1010 1010. With just two links in the bundle a 50% traffic split is preferred so those two hex values match 50% of hash results to each link.

When XOR’ing two values together (such as MAC addresses) to provide the RBH (Result Bundle Hash) value, the RBH value produced is always in the range of 0-7 (the formula for doing this is very complicated). The binary for 0, 2, 4, and 6 (0x55) all XOR with the RBH value and equate to Gi4/11. If the RBH value XOR’s with 1, 3, 5 or 7 (0xAA) the result is Gi4/12.

7600#remote login switch
Trying Switch ...
Entering CONSOLE for Switch
Type "^C^C^C" to end this session


7600-sp#show etherchannel 1 summary
...
Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         LACP      Gi4/11(P)   Gi4/12(P)


7600-sp#test etherchannel load-balance interface port-channel 1 mac 0000.0000.0000 0000.0000.0000
Computed RBH: 0x0
Would select Gi4/11 of Po1

7600-sp#test etherchannel load-balance interface port-channel 1 mac 0000.0000.0000 0000.0000.0001
Computed RBH: 0x1
Would select Gi4/12 of Po1

...

7600-sp#test etherchannel load-balance interface port-channel 1 mac 0000.0000.0000 0000.0000.0007
Computed RBH: 0x7
Would select Gi4/12 of Po1

7600-sp#test etherchannel load-balance interface port-channel 1 mac 0000.0000.0000 0000.0000.0008
Computed RBH: 0x8
Would select Gi4/11 of Po1

7600-sp#show etherchannel 1 port-channel
                Port-channels in the group:
                ----------------------

Port-channel: Po1    (Primary Aggregator)

------------

Age of the Port-channel   = 3d:14h:44m:43s
Logical slot/port   = 14/1          Number of ports = 2
HotStandBy port = null
Port state          = Port-channel Ag-Inuse
Protocol            =   LACP
Fast-switchover     = disabled
Direct Load Swap    = disabled

Ports in the Port-channel:

Index   Load    Port      EC state        No of bits
------+------+--------+------------------+-----------
  1     55     Gi4/11     Active             4
  0     AA     Gi4/12     Active             4

Time since last port bundled:    3d:14h:40m:08s    Gi4/12

Last applied Hash Distribution Algorithm: Fixed

In the case that 3 links are in a bundle (for example) the below output may be seen in "show interface port-channel x etherchannel". Here the bit sharing ratio is 3:3:2 which means two links have a higher probability of being utilized when compared to the third member link. Further down the table shows the bit distributions for an etherchannel with different numbers of member links:

Index   Load   Port     EC state        No of bits

------+------+------+------------------+-----------

  0     49     Gi3/1    Active    3

  1     92     Gi3/2    Active    3

  2     24     Gi3/3    Active    2


No. of link in bundle | RBH bit distribution 
8                       1:1:1:1:1:1:1:1
7                       2:1:1:1:1:1:1
6                       2:2:1:1:1:1
5                       2:2:2:1:1
4                       2:2:2:2
3                       3:3:2
2                       4:4

6500s/7600s support having a different hashing method per line card by using the "port-channel load-balance per-module" command followed by "port-channel load-balance src-dst-port module 2" for example.

On 6500/7600s the traffic is hashed on ingress. This means that when using DFCs and per module hashing methods, traffic entering a port on module 1 then egresses module 2 will egress based on the hashing method chosen for module 1 not module 2. When mixing DFCs and CFCs then the CFC modules will use the global PFC method which can be different from the DFC enable modules.

 

Cat 4500 & 4500-X LAG Specifics:

4500X's use an Agg2PhyPort array to store port-channel members and create the hashing results for load-balancing. The Cat 4500 and 4500-X support up to 8 member interfaces in a port-channel. Each interface in the port-channel using one entry in the Agg2PhyPort array. For example a 4 interface bond using interfaces 2, 4, 6, and 8 would be stored in array indexes 0, 1, 2 and 3. In this case the Agg2PhyPort array is { 0:2, 1:4, 2:6, 3:8 }. Whatever load-balancing algorithm is chosen, the Cisco proprietary algorithm will crunch the tuples down to a number, for example '6' which is then modulus against the number of elements in the array (4) to give the array index, 6%4=2, array index 2 is port 6.


Cat 4500s use the same approach as the 6500s/7600s above. However for newer Cat 4500s using the newer K5 ASICs, the traditional modulus approach is altered. Normally in the case of 3 member links (for example) the traffic distribution ratio becomes 3:3:2 to each member link. On the newer ASICs only the last three bits of the load-balancing hash are used so only a value of 1-3 is produced by the hash giving equal traffic balancing to a port-channel with 3 links. The same is true for 5 member links, only the last 5 bits of the hash are used, and so on for all possible combinations of member links.

# Outputs from two WS-C4500X-16's with C4KX-NM-8 module installed in a stack...

#show platform mapping ports

Executing the command on VSS member switch role = VSS Active, id = 1

Interface  Superport  Subport  CompactSubportId  PortSet  Phyport  Aggport    PimPhyport
Te1/2/1    4          0        48                1        108      Po31(863)  0
Te1/2/2    5          0        50                1        110      Po32(864)  1
Te1/2/3    0          0        52                0        112      Po33(865)  2
Te1/2/4    1          0        54                0        114      Po34(866)  3
Te1/2/5    12         0        56                3        116      Po4(836)   4
Te1/2/6    13         0        58                3        118      Po3(835)   5
Te1/2/7    8          0        60                2        120      Po1(833)   6
Te1/2/8    9          0        62                2        122      Po1(833)   7
Te1/1/1    60         0        16                15       12       Po11(843)  48
Te1/1/2    61         0        18                15       14       Po12(844)  49
Te1/1/3    66         0        20                17       16       Po13(845)  50
Te1/1/4    67         0        22                17       18       59         51
Te1/1/5    68         0        24                18       20       60         52
Te1/1/6    69         0        26                18       22       61         53
Te1/1/7    70         0        28                19       24       62         54
Te1/1/8    71         0        30                19       26       63         55
Te1/1/9    44         0        32                11       28       64         56
Te1/1/10   45         0        34                11       30       65         57
Te1/1/11   56         0        36                14       32       66         58
Te1/1/12   57         0        38                14       34       67         59
Te1/1/13   52         0        40                13       36       68         60
Te1/1/14   53         0        42                13       38       69         61
Te1/1/15   64         0        44                16       40       Po1(833)   62
Te1/1/16   65         0        46                16       42       Po1(833)   63

#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-port

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination TCP/UDP (layer-4) port number
  IPv6: Source XOR Destination TCP/UDP (layer-4) port number


#show platform software etherchannel port-channel 1 map l4-port 1.1.1.1 100 2.2.2.2 200 | i is Te
Map port for l4-port 1.1.1.1:100, 2.2.2.2:200 is Te1/1/16(Po1)

#show platform software etherchannel port-channel 1 map l4-port 1.1.1.1 100 2.2.2.2 201 | i is Te
Map port for l4-port 1.1.1.1:100, 2.2.2.2:201 is Te1/1/15(Po1)

#show platform software etherchannel port-channel 1 map l4-port 1.1.1.1 100 2.2.2.2 202 | i is Te
Map port for l4-port 1.1.1.1:100, 2.2.2.2:202 is Te1/2/7(Po1)

...
#show platform software etherchannel port-channel 1 map l4-port 1.1.1.1 100 2.2.2.2 210 | i is Te
Map port for l4-port 1.1.1.1:100, 2.2.2.2:210 is Te1/2/8(Po1)

 

ME3600X/ME3800X LAG Specifics:

When using the destination MAC method these switches support load-based hashing for the destination MAC however this limits the destination member port in a bundle to be only one of four. In an eight port bundle, 4 links will never be used for any given destination MAC.

MX3600X/ME3800X(config)#port-channel load-balance ?
  dst-ip       Dst IP Addr
  dst-mac      Dst Mac Addr
  src-dst-ip   Src XOR Dst IP Addr
  src-dst-mac  Src XOR Dst Mac Addr
  src-ip       Src IP Addr
  src-mac      Src Mac Addr

When routing vanilla IP traffic, acting as an MPLS P node for an IP VPN or as an ingress/egress LER for a L3 MPLS VPN, and a layer 2 LAG is in the next-hop path, the LAG hashes based on the global config "port-channel load-balance" option above. If using the IP hash options IPv4 and IPv6 are both supported for src-ip/dst-ip-/src-dst-ip hashing. When using the IP hash options and MPLS labelled traffic is being transported the IP details are extracted from the L3 VPN (there is a max label depth of 3). If there are no IP details behind the MPLS label stack the EOS label value is used. The MAC options for the LAG load-balancing use the outer MAC details (so the next-hop P/PE node in an MPLS network) which are usually the same, this can cause problems.

When ingressing or egressing a L2 VPN (LER) or acting as a L2 VPN P node, all of the global options above can be used for MAC or IP hashing. When using the MAC options and when there is a control-word after the end/bottom of stack label in the L2 VPN, the inner MAC details from the L2 VPN are extracted (again, there is a max label depth of 3 excluding control-word) otherwise the outer MAC details are used (the same caveat applied about likely having the same hash result for all traffic). When using the IP hashing options the inner IP details are extracted from within the L2 VPN traffic (max label depth limit of 3 excluding the control-word). If the IP details can't be extracted the last MPLS label value is used.

 

ASR920 LAG specifics:

Note: 10G and 1G ports can't be mixed into a LAG group.
Note: ASR920 supports a max of 64 LAG groups, each group supports a max of 8 active links and 8 standby links.

The ASR920 can hash based upon various fields when the next-hop path is over a LAG:

! IOS-XE 3.16.4S on ASR-920-4SZ-A
ASR920(config)#port-channel load-balance-hash-algo ? dst-ip Destination IP dst-mac Destination MAC src-dst-ip Source XOR Destination IP Addr src-dst-mac Source XOR Destination MAC src-dst-mixed-ip-port Source XOR Destination Port, IP addr src-ip Source IP src-mac Source MAC

The Cylon ASIC supports "per-flow" load balancing. The ASIC supports 16 hash buckets and links are mapped sequentially to these buckets based on a CRC32 hash. Non-labelled flows are hashed using destination IP, destination MAC, source XOR destination IP address, source XOR destination MAC, source IP or Source MAC as inputs.

Labelled flows can be hashed based on IP source and destination address from within the L3 VPN and inner labels for L2 VPN (P node). For PEs it depends on the device global configuration.

For L3 VPN transit traffic (P node) the default action is hash based on the outer source MAC (which can cause poor traffic distribution). For L2 VPN transit traffic (P node) the default action is to hash on the inner VC label.

For a L3 VPN LER/PE the globally configured port-channel load-balancing mode is used, the default is outer source MAC (which can cause poor traffic distribution). For a L2 VPN LER/PE the globally configured port-channel load-balancing mode is used, the default is the outer source MAC.

 

IP & MPLS Un/Equal-Cost Multi-Path Routing

6500/7600 Specifics:

mls ip cef load-sharing full
http://www.cisco.com/c/en/us/support/docs/ip/express-forwarding-cef/116376-technote-cef-00.html

The maximum number of load balancing paths is 8. The PFC forwards MPLS labeled packets without explicit configuration. If the packet has three labels or less and the underlying packet is IPv4, then the PFC uses the source and destination IPv4 address. If the underlying packet is not IPv4 or more than three labels are present, the PFC parses down as deep as the fifth or lowest label and uses it for hashing.

MPLS Layer 2 VPN Load Balancing: Load balancing is based on the VC label in the MPLS core if the first nibble of the MAC address in the customer Ethernet frame is not 4.
Note: Load balancing is not supported at the ingress PE for Layer 2 VPNs. Load balancing is done based on the VC label and it is pre-selected.

MPLS Layer 3 VPN Load Balancing: MPLS Layer 3 VPN load balancing is similar to basic MPLS load balancing.

 

ASR9000 Specifics:

As of IOS-XR 5.3.0 Typhoon and Tomahawk NPs support 64-way ECMP for OSPF/OSPFv3 and ISIS. This is tunable on the CLI by using the following commands:

-OSPF/OSPFv3:
-router ospf 100 maximum paths <1-128>
-router ospfv3 100 maximum paths <1-128>
ISIS:
-router isis 1111 address-family ipv4 unicast/mcast maximum-paths <1-64>
-router isis 1111 address-family ipv6 unicast/mcast maximum-paths <1-64>

RP/0/RSP0/CPU0:ASR9001-XR-5.3.3#show cef misc
...
Platform capabilities:
--------------------------

L3 loadbalancing levels: 2
L3 Hash buckets: 64
L3 recursive Hash buckets: 32
L3 Unequal cost hash buckets: 0
Forwarding stages: 2
Local label split: 3
Statistics support: No
Path per Tunnel class: 8
V4 update disabled: No
V6 update disabled: No
LBA tuples: 1
MRAPS support: No
Tunnel default class support: Yes
Loadinfo choices are platform filtered
Prefix filter support: host prefix only
LISP ipv4/v4 decapsulation support: Yes
LISP ipv4/v6 decapsulation support: Yes
LISP unequal cost loadbalance support for ipv4 payload: No
Fallback support for ipv4: Yes
LISP ipv6/v4 decapsulation support: Yes
LISP ipv6/v6 decapsulation support: Yes
LISP unequal cost loadbalance support for ipv6 payload: No
Fallback support for ipv6: Yes
NHID (Next Hop ID) support: No
Backwalk (Dependents) required: No
Link MPLS IPv6 nexthop support: Yes
Exclude Deag backup paths: No

Slides 53 onwards: https://clnv.s3.amazonaws.com/2017/usa/pdf/BRKSPG-2904.pdf

Test CEF load-balancing result:

# Using Layer 3 and optionally layer 4 header info:

RP/0/RSP0/CPU0:ASR9001#show cef exact-route 1.1.1.1 2.2.2.2 protocol tcp source-port 10 destination-port 20 ingress-interface te0/0/0/0

RP/0/RSP0/CPU0:ASR9001#show cef exact-route 1.1.1.1 2.2.2.2 protocol ?
    Enter protocol Number or name below
  AHP      authentication header
  EGP      exterior gateway protocol
  EIGP     EIGRP protocol
  ENCAP    Encapsulation header
  EON      IOS CNLP
  ESP      IPSEC Encapsulating Security Payload
  GRE      GRE encapsulation
  ICMP     control message protocol
  IDP      XNS IDP
  IGMP     group managment protocol
  IGRP     IGRP protocol
  IPINIP   IP in IP encapsulation
  L2TV3     L2TPv3 tunneling encapsulation
  NOSIP    KA9Q/NOS compatible IP over IP
  OSPF     OSPF protocols
  PCP      porotocol
  PUP      PUP protocol
  RAW      raw IP packet
  RSVP     resource reservation protocol
  SCTP     Stream Control Transmission
  TCP      TCP protocol
  TP       tp-4 with class negotiation
  UDP      UDP protocol
  UTI      UTI tunneling encapsulation


# Using MPLS labels and optionally layer 3 VPN payload IPs
RP/0/RSP0/CPU0:ASR9001#show mpls forwarding exact-route label 12345 ipv4 1.1.1.1 2.2.2.1

RP/0/RSP0/CPU0:ASR9001#show mpls forwarding exact-route label 12345 ?
  bottom-label   Bottom label
  entropy-label  Entropy label
  ipv4           Exact path for IPv4 payload
  ipv6           Exact path for IPv6 payload

 

ME3600X/ME3800X MPLS Specifics

When ECMP paths exist for forwarding vanilla IPv4 and IPv6 traffic and MPLS L3 VPNs the source and destination IPs are used for hashing. If the IPs can't be found (label stack too deep, 3 is the max label depth) the last label value is used. When ECMP paths existing for MPLS L2 VPNs (if the control-word is beneath the end of stack label) the inner IP source and destination IPs are used, if there aren't any (such non-IP traffic over Ethernet) then the inner source and destination MACs are used.

Layer 2 VPN Load Balancing: To get an evenly-distributed load balancing experience over virtual circuits (VC), it is recommended to have more number of VCs.

Layer 3 VPN Load Balancing: Load balancing is supported only on homogeneous paths, that is, all MPLS adjacencies or all IP adjacencies. This is because the MPLS SWAP/POP action in the LFIB also contains the next-hop rewrite details.

 

ASR920 MPLS Specifics:

For L3 VPN traffic where the ASR920 is either a P node or PE node, it hashes based on the L3 VPN source and destination IP addresses (max label depth of 3). For L2 VPN traffic where the ASR920 is a P node it hashes based on the VC label and where it is a PE node it hashes based on the local label.

On the ASR920 either 4 or 8 ECMPs are supported:

ASR920(config)#platform loadbalance max-paths ?
  4  4 paths, default
  8  8 paths

ASR920(config)#ip cef load-sharing algorithm ?
  include-ports  Algorithm that includes layer 4 ports
  original       Original algorithm
  tunnel         Algorithm for use in tunnel only environments
  universal      Algorithm for use in most environments

The "original" option hashes using only the source and destination IP address. This produces a 4 bit value as ECMP routing is only supported over 16 links. For example, a device with 4 ECMP links to another device might show the following output (the same four CEF adjacency entries are repeated 4 times to fill the 16 hash buckets):

#show ip cef 1.1.1.1/32 internal
...
    16 hash buckets
      < 0 > IP adj out of GigabitEthernet1/0, addr 12.0.0.2 68D03140
      < 1 > IP adj out of GigabitEthernet2/0, addr 13.0.0.3 6A9B1220
      < 2 > IP adj out of GigabitEthernet3/0, addr 14.0.0.4 68D02FE0
      < 3 > IP adj out of GigabitEthernet4/0, addr 15.0.0.5 6A9B1380
      < 4 > IP adj out of GigabitEthernet1/0, addr 12.0.0.2 68D03140
      < 5 > IP adj out of GigabitEthernet2/0, addr 13.0.0.3 6A9B1220
      < 6 > IP adj out of GigabitEthernet3/0, addr 14.0.0.4 68D02FE0
      < 7 > IP adj out of GigabitEthernet4/0, addr 15.0.0.5 6A9B1380
      < 8 > IP adj out of GigabitEthernet1/0, addr 12.0.0.2 68D03140
      < 9 > IP adj out of GigabitEthernet2/0, addr 13.0.0.3 6A9B1220
      < 10 > IP adj out of GigabitEthernet3/0, addr 14.0.0.4 68D02FE0
      < 11 > IP adj out of GigabitEthernet4/0, addr 15.0.0.5 6A9B1380
      < 12 > IP adj out of GigabitEthernet1/0, addr 12.0.0.2 68D03140
      < 13 > IP adj out of GigabitEthernet2/0, addr 13.0.0.3 6A9B1220
      < 14 > IP adj out of GigabitEthernet3/0, addr 14.0.0.4 68D02FE0
      < 15 > IP adj out of GigabitEthernet4/0, addr 15.0.0.5 6A9B1380

In the case that there are say 5 ECMP paths they would be repeated 3 times to fill 15 hash buckets and the 16th would be empty/unused.

One can check ECMP and CEF details using the following commands:
show ip cef x.x.x.x/x internal
show ip cef exact-route y.y.y.y x.x.x.x
show platform hardware pp active feature cef database ipv4 x.x.x.x/x
show platform hardware pp active feature cef database ipv6 x::x/x

By default OSPF or ISIS might only support 4 ECMPs, to increase this configure "maximum-paths <1-32>" under "router ospf X" or "router isis X".

The "ip cef load-sharing algorithm universal" features adds a value to the hash function that is unique to this local device so that multiple devices which all have ECMP configure will hash differently to reduce link polarisation across multiple devices.

The "ip cef load-sharing algorithm tunnel" feature is supposed to provide better hashing in environments where there are just a few source and destination IPs as most traffic is tunneled (inside L2TP or GTP tunnels for example). The documentation is unclear as to how this works but it looks like it's meant for GTP and will look at the sessions inside to get additional hashing entropy.

 

MPLS & FAT

 

 

MPLS-TE


https://www.cisco.com/c/en/us/td/docs/switches/metro/me3600x_3800x/software/release/15-3_3_S/configuration/guide/3800x3600xscg/swmpls.html#22717