Offload routes from Kernel to hardware

References:
https://www.arista.com/en/support/toi/eos-4-29-1f/16705-routing-control-functions-kernelfib-agent-support
https://www.arista.com/en/support/toi/eos-4-28-1f/15683-explicit-source-address-for-routes-with-fwd-next-hop-device
https://www.arista.com/en/support/toi/eos-4-28-1f/15859-software-forwarding-offload-to-fwd0-hardware-using-rcf-policy

 

By default, all routes in all VRFs (including the default table) are installed in the Kernel (in the respective VRF) so that punted traffic such as traceroutes work (the Kernel needs a return route).

In the following example, the global BGP DFZ is in a VRF, and the routes in the DFZ VRF will be prevented from being installed into the Kernel (to save on CPU cycles constantly programming DFZ updates into the Kernel).

interface Loopback0
   description Underlay Loopback
   ip address 10.0.0.1/32
   ipv6 address fd::1/128
!
interface Loopback1
   description DFZ Loopback
   vrf DFZ
   ip address 10.0.0.1/32
   ipv6 address fd::1/128
   hardware forwarding id ! this makes the interface usable by the kernel
!
! Tell the kernel to use Loopback1 as the interface for local lookups in the DFZ, in order to off-load these lookups to hardware.
! We also tell the kernel to off-load non-local lookups to BGP learned routes in hardware
!
router kernel vrf DFZ
   address-family ipv4
      software forwarding hardware offload route local interface Loopback1
      software forwarding hardware offload route lookup bgp
   !
   address-family ipv6
      software forwarding hardware offload route local interface Loopback1
      software forwarding hardware offload route lookup bgp
!
! Restart the KernelFib agent to apply:
!
agent KernelFib terminate

! At this point, the kernel can only respond to packets destined for the local device, which means it doesn't respond if it's a mid-point in a traceroute for example.
! This is because by default two blackhole routes are installed which cover the entire IPv4 address space, and only local interface routes are copied to the Kernel FIB:
!
r1#show kernel ip route vrf DFZ
VRF: DFZ
...
blackhole 0.0.0.0/8 proto gated scope nowhere 
blackhole 127.0.0.0/8 proto gated scope nowhere
...

! As a result, when a packet is punted to the Kernel, such as with traceroute, and the Kernel does an IP lookup for the destination IP, it see's no route so drops the packet.
! The Kernel doesn't know which routes to send to hardware for lookup.
! Here we define an RCF to allow static routes to be added to the Kernel FIB, and define a static route for all loopback IPs in the DFZ across the network:
!
router general
   control-functions
      code unit KERNELFIB
         function KERNELFIB_OFFLOAD_STATIC() {
           return source_protocol is STATIC;
         }
      EOF
!
ip route vrf DFZ 10.0.0.0/24 Null0
!
! Change the kernel off-load rules for the DFZ, to program in static routes into the Kernel instead of using the interface lookup
router kernel vrf DFZ
   address-family ipv4
      software forwarding hardware offload route rcf KERNELFIB_OFFLOAD_STATIC()
      software forwarding hardware offload route lookup bgp
!
! Now the kernel has the more specific static route, which points to the hardware offload interface fwd1 (as well as local interface subnets which are not shown below for brevity).
! The router is now able to respond to traceroutes for non-local IPs (ICMP TTL exceeded).
!
r1#show kernel ip route vrf DFZ
VRF: DFZ
...
blackhole 0.0.0.0/8 proto gated scope nowhere 
blackhole 127.0.0.0/8 proto gated scope nowhere
127.254.254.1 dev fwd1 proto gated scope link metric 1024 
10.0.0.0/24 via 127.254.254.1 dev fwd1 proto gated 
...

 

Routes in the kernel which point to hardware fardwaring use the interface "fwdX" such as "fwd0" or "fwd1" etc.

Routes can be seen in the kernel via either:

show kernel ip route vrf DFZ
show kernel ipv6 route vrf DFZ

or:

bash sudo ip netns exec ns-DFZ ip r
bash sudo ip netns exec ns-DFZ ip -6 r

Check RCF is applied to kernel using:

show kernelfib rcf

 

As a side note, it is possible to tell the Kernel to offload all underlay / IGP routes to hardware and enable ECMP in the kernel:

router kernel
   address-family ipv4
      software forwarding hardware offload route rcf KERNEL_OFFLOAD_POLICY()
   !
   address-family ipv6
      software forwarding hardware offload route rcf KERNEL_OFFLOAD_POLICY()
!
router general
   hardware next-hop fast-failover
   control-functions
      code unit kernel
      function KERNEL_OFFLOAD_POLICY() {
         return source_protocol is ISIS;
      }
      EOF
!
agent KernelFib environment KERNELFIB_PROGRAM_ALL_ECMP='true
!
agent KernelFib terminate

 

---