CoPP & CPU Protection (IOS on 7600)

References:
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12-2SX/best/practices/recommendations.html#wp1214688
http://www.cisco.com/c/en/us/about/security-center/copp-best-practices.html#8
http://www.cisco.com/web/about/security/intelligence/coppwp_gs.html
http://www.cisco.com/c/en/us/td/docs/routers/7600/ios/12-2SR/configuration/guide/swcg/dos.html
http://www.cisco.com/c/en/us/support/docs/quality-of-service-qos/qos-congestion-management-queueing/18664-rtgupdates.html
http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-6500-series-switches/prod_white_paper0900aecd802ca5d6.pdf
http://www.cisco.com/c/en/us/support/docs/ip/access-lists/43920-iacl.html
http://www.cisco.com/c/dam/en/us/products/collateral/security/ios-network-foundation-protection-nfp/prod_white_paper0900aecd804ac831.pdf
http://www.cisco.com/c/en/us/td/docs/routers/7600/troubleshoot/guide/7600_Trouble_Shooting.pdf
https://tools.ietf.org/html/rfc6192
https://www.nanog.org/meetings/nanog42/presentations/Bhaiji_Network_Core.pdf
http://mailman.nanog.org/pipermail/nanog/2010-June/022649.html
https://puck.nether.net/pipermail/cisco-nsp/2015-June/099836.html

 

CoPP Overview:

To remove and reduce unwanted traffic from the CPU path "CoPP" alone is not enough. On platforms like the 7600/6500 they also have MLS hardware rate-limiters which must be configured to work alongside CoPP. The MLS HWRL are acting against all trafic transiting though the router as well as traffic destined for the router, were as CoPP is specifically on the punt path only so care must be taken with MLS HWRL not to negatively affect transit traffic.

Some traffic will hit the mls policers if for example no adjacency is found. When traffic hits the MLS rate limites it bypasses CoPP. Traffic passes through one or the other only, not both.Cards with DFCs perform CoPP in the line card. With CFCs the PFC provides the hardware CoPP processing.

Also features that cause traffic to originate from the CPU can be tuned like ICMP uncreachables.

Note Before: CoPP doesn't just cover traffic that is destined for the CPU, all traffic that is passed to the CPU will be affected by a CoPP policy, for example CPU punted traffic.

Note Before: Some devices can have issues when MPLS Explicit Null is used. The CoPP policies won't match traffic coming into the control plane if it has label 0 applied. 7600s are one platform that can handle MPLS explicit null though, it will look beyond the null label to match a traffic class.

Some bugs to be aware off on 7600s:
CSCsf25709 - "vpn-cam gets disable if class-default is in use in an applied policy-map (or a MATCH ANY clause is used within another class)".

CSCsi25255 - "policers using "class-default" reduce vpn forwarding performance".
Because the VPN-CAM is disable all L3 VPN traffic is passed through the policer!

CSCsf96383 - "After reload Control plane policing is not applied in hw anymore"
Check with "show mls qos ip | i CPP" that the QoS policies are applied to the control plane.

CSCsg85740 - "mls rate-limit mtu-failure drops >MTU-18 packets when CoPP configured"

 

Hardware Rate Limiter Example:

! There can be no CoPP in hardware or MLS rate-limtiters without first globally enabling MLS QoS
mls qos
! ***********Interrupt/Process Scheduler***********

! scheduler allocate  
! default: scheduler allocate 4000 800

! First integer, maximum number of microseconds to spend on fast switching within any one
! network interrupt context

! Second integer, guarantees the minimum number of microseconds to spend at the process
! level when network interrupts are disabled

scheduler allocate 1000 2000

! Decrease the maximum amount of continuous time the CPU may spend on any one process
! Default: process-max-time 200 (ms)
process-max-time 100
! ***********ICMP Unreachables***********

! Stop sending ICMP unreachable messages for traffic we black hole, default is to send 
! ICMP unreachables

interface null0
 ! default: ip unreachables
 ! default: ipv6 unreachables
 no ipv6 unreachables
 no ip unreachables


! Rate-limit icmp unreachables code 0, network unreachable and code 4, fragmentation 
! needed when DF bit set, default is 500ms for both, check before and after with 
! "show ip icmp rate-limit"

ip icmp rate-limit unreachable 1000
ip icmp rate-limit unreachable df 1000
! ***********Directred Broadcasts***********

! Move directed broadcasts into hardware and not don't forward them to the RP
! Interface level command
! default: no mls ip directed-broadcast

(config-if)# mls ip directed-broadcast exclude-router

! As of IOS 11, "no ip directed-broadcast" is the configured by default on all interfaces so the above mls command is not needed, only if directed broadcasts have been explicitly enabled on for an interface.
! ***********CoPP & MLS***********

! CoPP is supported in software for multicast and broadcast traffic.

! Hardware support for multicast and broadcast traffic is provided by special-cases
! hardware-based rate-limiters.

! ARP traffic is not covered by CoPP (neither hardware nor software) at all on 6500/7600
! platforms. However, an ARP special-cases hardware-based rate-limiter is available.

! The special-cases hardware-based rate-limiters will override the hardware CoPP policy for
! packets matching the rate-limiters criteria.

! CoPP uses hardware QoS TCAM resources, check TCAM usage before and after configuring CoPP with
! "show tcam counts" or "show tcam utilization"
! "show tcam interface fa5/2 acl in ip detail"
! "show mls statistics"
! CoPP does not support ACL entries with log keyword. ! Only the “input” direction is supported for CoPP.
! ***********MLS Limiters/Policers***********

! Check MLS rate limiter usage before and after configuring with
! "show mls rate-limit"
! "show mls rate-limit usage"
! "clear icmp rate-limit"

! This command is basically useless and shouldn't be used:
! mls rate-limit unicast cef receive XXX
! "Rate limits all packets that contain any route processor IP address as the destination address"
! HWRLs take precedence over CoPP so this will rate-limit RP traffic without allowing a CoPP policy to properly filter it.
! mls qos protocol xxx mls qos protocol arp police 2000000 62000 mls qos protocol neigh-discover police 2000000 62000
! Layer 3 HWRLs:
! Disable this rate-limit which is used by default (unless you need it)
! because it uses one of the HWRL spaces in CEF
no mls rate-limit unicast acl vacl-log
! mls rate-limit multicast ipv4 xxx mls rate-limit multicast ipv4 fib-miss 2000 10 !Note that this rate-limiter uses a special register that is not accounted for in the available ten hardware registers and it is applied globally, not on a per-forwarding-engine basis. mls rate-limit multicast ipv4 non-rpf 10 10 mls rate-limit multicast ipv4 partial 2000 10
mls rate-limit multicast ipv4 ip-options 10 10

! mls rate-limit multicast ipv6 xxx
! Several HWRL for Multicast don't work on 6500s:
! https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuo37358/?referring_site=bugquickviewredir
! broken: mls rate-limit multicast ipv6 connected 1500 20
! broken: mls rate-limit multicast ipv6 default-drop 1000 20
! broken: mls rate-limit multicast ipv6 mld 10 1
! broken: mls rate-limit multicast ipv6 route-cntl 10 1
! mls rate-limit unicast ip xxx
mls rate-limit unicast ip features 10 10 mls rate-limit unicast ip icmp redirect 0 ! These next four all share a single HWRL mls rate-limit unicast ip icmp unreachable no-route 10 10 mls rate-limit unicast ip icmp unreachable acl-drop 10 10 mls rate-limit unicast ip rpf-failure 10 10 mls rate-limit unicast ip errors 10 10
! mls rate-limit unicast ip options 10 10
! This one is not enabled because it is covered in the CoPP polocy
! mls rate-limit unicast cef xxx mls rate-limit unicast cef glean 200 50 ! "all" applied to both Unicast and Multicast mls rate-limit all ttl-failure 200 50 mls rate-limit all mtu-failure 10 10 ! Was affected by CSCsg85740 but fixed now
! Layer 2 HWRLs:
! mls rate-limit layer2 xxx mls rate-limit layer2 pdu 20 20
mls rate-limit multicast ipv4 igmp 2000 10
! Before:
abr1#show mls rate-limit
 Sharing Codes: S - static, D - dynamic
 Codes dynamic sharing: H - owner (head) of the group, g - guest of the group 

   Rate Limiter Type       Status     Packets/s   Burst  Sharing
 ---------------------   ----------   ---------   -----  -------
         MCAST NON RPF   Off                  -       -     -
        MCAST DFLT ADJ   On              100000     100  Not sharing
      MCAST DIRECT CON   Off                  -       -     -
        ACL BRIDGED IN   Off                  -       -     -
       ACL BRIDGED OUT   Off                  -       -     -
           IP FEATURES   Off                  -       -     -
          ACL VACL LOG   On                2000       1  Not sharing
           CEF RECEIVE   Off                  -       -     -
             CEF GLEAN   Off                  -       -     -
      MCAST PARTIAL SC   On              100000     100  Not sharing
        IP RPF FAILURE   On                 100      10  Group:0 S
           TTL FAILURE   On                  97      10  Not sharing
 ICMP UNREAC. NO-ROUTE   On                 100      10  Group:0 S
 ICMP UNREAC. ACL-DROP   On                 100      10  Group:0 S
         ICMP REDIRECT   Off                  -       -     -
           MTU FAILURE   On                 997      10  Not sharing
       MCAST IP OPTION   Off                  -       -     -
       UCAST IP OPTION   Off                  -       -     -
           LAYER_2 PDU   Off                  -       -     -
            LAYER_2 PT   Off                  -       -     -
      DHCP Snooping IN   Off                  -       -     -
     DHCP Snooping OUT   Off                  -       -     -
        ARP Inspection   Off                  -       -     -
       LAYER_2 PORTSEC   Off                  -       -     -
     LAYER_2 MiniProto   Off                  -       -     -
             IP ERRORS   On                 100      10  Group:0 S
           CAPTURE PKT   Off                  -       -     -
            MCAST IGMP   Off                  -       -     -
 MCAST IPv6 DIRECT CON   Off                  -       -     -
 MCAST IPv6 ROUTE CNTL   Off                  -       -     -
 MCAST IPv6 *G M BRIDG   Off                  -       -     -
  MCAST IPv6 SG BRIDGE   Off                  -       -     -
  MCAST IPv6 DFLT DROP   Off                  -       -     -
 MCAST IPv6 SECOND. DR   Off                  -       -     -
  MCAST IPv6 *G BRIDGE   Off                  -       -     -
        MCAST IPv6 MLD   Off                  -       -     -
  IP ADMIS. ON L2 PORT   Off                  -       -     -
 IPV6 FIRST HOP SECURI   Off                  -       -     -
    UCAST IP TINY FRAG   Off                  -       -     -
    MCAST IP TINY FRAG   Off                  -       -     -
        LAYER_2 MACSEC   Off                  -       -     -
        MCAST IPv4 PIM   Off                  -       -     -
           IPV6 BRIDGE   Off                  -       -     -


abr1#show mls rate-limit usage
                             Rate Limiter Type     Packets/s   Burst
                           ---------------------   ---------   -----
Layer3 Rate Limiters:
             RL# 0: Free                       -           -       -
             RL# 1: Free                       -           -       -
             RL# 2: Free                       -           -       -
             RL# 3: Used
                                     MTU FAILURE         997      10
             RL# 4: Used
                                     TTL FAILURE          97      10
             RL# 5: Used
                                  IP RPF FAILURE         100      10
                           ICMP UNREAC. NO-ROUTE         100      10
                           ICMP UNREAC. ACL-DROP         100      10
                                       IP ERRORS         100      10
             RL# 6: Used
                                    ACL VACL LOG        2000       1
             RL# 7: Used
                                  MCAST DFLT ADJ      100000     100
             RL# 8: Rsvd for capture           -           -       -

Layer2 Rate Limiters:
             RL# 9: Reserved
             RL#10: Reserved
             RL#11: Free                       -           -       -
             RL#12: Free                       -           -       -



! After

abr1#show mls rate-limit
 Sharing Codes: S - static, D - dynamic
 Codes dynamic sharing: H - owner (head) of the group, g - guest of the group

   Rate Limiter Type       Status     Packets/s   Burst  Sharing
 ---------------------   ----------   ---------   -----  -------
         MCAST NON RPF   On                  10      10  Not sharing
        MCAST DFLT ADJ   On                2000      10  Not sharing
      MCAST DIRECT CON   Off                  -       -     -
        ACL BRIDGED IN   Off                  -       -     -
       ACL BRIDGED OUT   Off                  -       -     -
           IP FEATURES   On                  10      10  Not sharing
          ACL VACL LOG   Off                  -       -     -
           CEF RECEIVE   Off                  -       -     -
             CEF GLEAN   On                 200      50  Not sharing
      MCAST PARTIAL SC   On                2000      10  Not sharing
        IP RPF FAILURE   On                  10      10  Group:0 S
           TTL FAILURE   On                 200      50  Not sharing
 ICMP UNREAC. NO-ROUTE   On                  10      10  Group:0 S
 ICMP UNREAC. ACL-DROP   On                  10      10  Group:0 S
         ICMP REDIRECT   On                   0       0    -
           MTU FAILURE   On                  10      10  Not sharing
       MCAST IP OPTION   On                  10      10  Group:3 S
       UCAST IP OPTION   Off                  -       -     -
           LAYER_2 PDU   On                  20      20  Not sharing
            LAYER_2 PT   Off                  -       -     -
      DHCP Snooping IN   Off                  -       -     -
     DHCP Snooping OUT   Off                  -       -     -
        ARP Inspection   Off                  -       -     -
       LAYER_2 PORTSEC   Off                  -       -     -
     LAYER_2 MiniProto   Off                  -       -     -
             IP ERRORS   On                  10      10  Group:0 S
           CAPTURE PKT   Off                  -       -     -
            MCAST IGMP   On                2000      10  Not sharing
 MCAST IPv6 DIRECT CON   Off                  -       -     -
 MCAST IPv6 ROUTE CNTL   Off                  -       -     -
 MCAST IPv6 *G M BRIDG   Off                  -       -     -
  MCAST IPv6 SG BRIDGE   Off                  -       -     -
  MCAST IPv6 DFLT DROP   Off                  -       -     -
 MCAST IPv6 SECOND. DR   Off                  -       -     -
  MCAST IPv6 *G BRIDGE   Off                  -       -     -
        MCAST IPv6 MLD   Off                  -       -     -
  IP ADMIS. ON L2 PORT   Off                  -       -     -
 IPV6 FIRST HOP SECURI   Off                  -       -     -
    UCAST IP TINY FRAG   Off                  -       -     -
    MCAST IP TINY FRAG   On/Sharing          10      10     -
        LAYER_2 MACSEC   Off                  -       -     -
        MCAST IPv4 PIM   Off                  -       -     -
           IPV6 BRIDGE   Off                  -       -     -

abr1#show mls rate-limit usage
                             Rate Limiter Type     Packets/s   Burst
                           ---------------------   ---------   -----
Layer3 Rate Limiters:
             RL# 0: Used
                                 MCAST IP OPTION          10      10
             RL# 1: Used
                                       CEF GLEAN         200      50
             RL# 2: Used
                                   MCAST NON RPF          10      10
             RL# 3: Used
                                     MTU FAILURE          10      10
             RL# 4: Used
                                     TTL FAILURE         200      50
             RL# 5: Used
                                  IP RPF FAILURE          10      10
                           ICMP UNREAC. NO-ROUTE          10      10
                           ICMP UNREAC. ACL-DROP          10      10
                                       IP ERRORS          10      10
             RL# 6: Used
                                     IP FEATURES          10      10
             RL# 7: Used
                                  MCAST DFLT ADJ        2000      10
             RL# 8: Rsvd for capture           -           -       -

Layer2 Rate Limiters:
             RL# 9: Reserved
             RL#10: Reserved
             RL#11: Used
                                     LAYER_2 PDU          20      20
             RL#12: Used
                                      MCAST IGMP        2000      10

Example CoPP config below. Use "show tcp brief all | i LIST" and "show udp" to check that all TCP and UDP sessions to the RP would be sovered by the policy being written.

! ***********Access-groups***********

!!!!! Control plane traffic (such as routing protocols)

ip access-list extended CoPP-Limit-and-Permit-BGP
 permit tcp any eq 179 any
 permit tcp any any eq 179

ipv6 access-list CoPP-Limit-and-Permit-BGPv6
 permit tcp any eq 179 any
 permit tcp any any eq 179

ip access-list extended CoPP-Limit-and-Permit-RSVP
 permit 46 any any
 
ip access-list extended CoPP-Limit-and-Permit-LDP
 permit tcp any any eq 646
 permit tcp any eq 646 any
 permit udp any any eq 646
 permit udp any eq 646 any
 
ip access-list extended CoPP-Limit-and-Permit-OSPF
 permit ospf any any

ipv6 access-list CoPP-Limit-and-Permit-OSPFv3
 permit 89 any any

ip access-list extended CoPP-Limit-and-Permit-HSRP
 permit udp host 224.0.0.2 eq 1985 any
 permit udp any host 224.0.0.2 eq 1985
 permit udp host 224.0.0.102 eq 1985 any
 permit udp any host 224.0.0.102 eq 1985

ip access-list extended CoPP-Limit-and-Permit-BFD
 permit udp any any eq 3784
 permit udp any eq 3784 any



!!!!! Control plane SYN traffic

ip access-list extended CoPP-Limit-and-Permit-BGP-SYN
 permit tcp any eq 179 any syn
 permit tcp any any eq 179 syn

ipv6 access-list CoPP-Limit-and-Permit-BGPv6-SYN
 permit tcp any eq 179 any syn
 permit tcp any any eq 179 syn

ip access-list extended CoPP-Limit-and-Permit-LDP-SYN
 permit tcp any any eq 646 syn
 permit tcp any eq 646 any syn



!!!!! Management plane traffic (such as SSH,SNMP)

ip access-list extended CoPP-Limit-and-Permit-TACACS
 permit tcp host 192.168.30.50 eq 49 any 
 permit tcp any host 192.168.30.50 eq 49
 permit tcp host 192.168.30.51 eq 49 any
 permit tcp any host 192.168.30.51 eq 49
 permit udp host 192.168.30.50 eq 49 any
 permit udp any host 192.168.30.50 eq 49
 permit udp host 192.168.30.51 eq 49 any
 permit udp any host 192.168.30.51 eq 49

ip access-list extended CoPP-Limit-and-Permit-RADIUS
 permit udp host 192.168.30.50 eq 1812 1813 any
 permit udp any host 192.168.30.50 eq 1812 1813
 permit udp host 192.168.30.51 eq 1812 1813 any
 permit udp any host 192.168.30.51 eq 1812 1813

ip access-list extended CoPP-Limit-and-Permit-TELNET-SSH
 permit tcp 192.168.30.0 0.0.1.255 any eq 22
 permit tcp any eq 22 192.168.30.0 0.0.1.255
 permit tcp 192.168.30.0 0.0.1.255 any eq 21
 permit tcp any eq 21 192.168.30.0 0.0.1.255

ip access-list extended CoPP-Limit-and-Permit-SNMP
 permit udp 192.168.30.0 0.0.1.255 any eq snmp snmptrap
 permit udp any eq 161 161 192.168.30.0 0.0.1.255

ip access-list standard CoPP-Limit-and-Permit-NTP
 permit udp any host 192.168.30.20 eq 123
 permit udp host 192.168.30.20 eq 123 any
 permit udp any host 192.168.30.22 eq 213
 permit udp host 192.168.30.22 eq 123 any

ip access-list standard CoPP-Limit-and-Permit-SNMP-Pollers
 permit udp host 192.168.30.34 any eq 161
 permit udp host 192.168.30.35 any eq 161
 permit udp host 192.168.30.33 any eq 161
 permit udp host 192.168.30.40 any eq 161
 permit udp any eq 161 host 192.168.30.34
 permit udp any eq 161 host 192.168.30.35
 permit udp any eq 161 host 192.168.30.33
 permit udp any eq 161 host 192.168.30.40



!!!!! Management SYN plane traffic

ip access-list extended CoPP-Limit-and-Permit-TACACS-SYN
 permit tcp host 192.168.30.50 eq 49 any syn
 permit tcp any host 192.168.30.50 eq 49 syn
 permit tcp host 192.168.30.51 eq 49 any syn
 permit tcp any host 192.168.30.51 eq 49 syn

ip access-list extended CoPP-Limit-and-Permit-TELNET-SSH-SYN
 permit tcp 192.168.30.0 0.0.1.255 any eq 22
 permit tcp any eq 22 192.168.30.0 0.0.1.255
 permit tcp 192.168.30.0 0.0.1.255 any eq 23
 permit tcp any eq 23 192.168.30.0 0.0.1.255



!!!!! Forwarding traffic that is CPU punted
 
ip access-list extended CoPP-Limit-and-Permit-IP-Options
 permit ip any any option record-route

ip access-list extended CoPP-Limit-and-Permit-ICMP
 permit icmp any any echo
 permit icmp any any echo-request
 permit icmp any any unreachable
 permit icmp any any ttl-exceeded
 permit icmp any any packet-too-big

ipv6 access-list CoPP-Limit-and-Permit-ICMPv6
 permit icmp any any echo-reply 
 permit icmp any any echo-request
 permit icmp any any unreachable
 permit icmp any any ttl-exceeded
 permit icmp any any packet-too-big

ip access-list extended CoPP-Limit-and-Permit-UDP-Traceroute
 permit udp any any range 33434 33689
 permit udp any range 33434 33689 any

ip access-list extended CoPP-Limit-and-Permit-TCP-syn-fin-rst
 permit tcp any any fin syn rst

ipv6 access-list CoPP-Limit-and-Permit-TCP-syn-fin-rst-IPv6
 permit tcp any any fin syn rst

ip access-list extended CoPP-Limit-and-Permit-TCP-established
 permit tcp any any established

ipv6 access-list CoPP-Limit-and-Permit-TCP-established-IPv6
 permit tcp any any established



!!!!! Traffic we want to always drop

ip access-list extended CoPP-Deny-IP-Fragments
 permit ip any any fragments
 permit icmp any any fragments

ip access-list CoPP-Deny-IPv6-Fragments
 permit ip any any fragments
 permit icmp any any fragments

ip access-list extended CoPP-Deny-ICMP
  permit icmp any any

ipv6 access-list CoPP-Deny-ICMPv6
 permit icmp any any

ip access-list extended CoPP-Deny-IP-Options
 permit ip any any option any 

ip access-list extended CoPP-Deny-IGMP
 permit igmp any any


!!!!! Catch-all for traffic that doesn't match any of the above ACLS

ip access-list extended CoPP-Limit-and-Permit-Catch-All
 permit icmp any any
 permit ip any any

ipv6 access-list CoPP-Limit-and-Permit-Catch-All-IPv6
 permit icmp any any
 permit ipv6 any any
! ***********Class Definitions***********

!!!!! Control plane traffic (such as routing protocols)

class-map match-any CoPP-Limit-and-Permit-Critical
 match access-group name CoPP-Limit-and-Permit-BGP
 match access-group name CoPP-Limit-and-Permit-BGPv6
 match access-group name CoPP-Limit-and-Permit-RSVP
 match access-group name CoPP-Limit-and-Permit-LDP
 match access-group name CoPP-Limit-and-Permit-LDP
 match access-group name CoPP-Limit-and-Permit-OSPF
 match access-group name CoPP-Limit-and-Permit-OSPFv3
 match access-group name CoPP-Limit-and-Permit-HSRP
 match access-group name CoPP-Limit-and-Permit-BFD



!!!!! Control plane SYN traffic

class-map match-any CoPP-Limit-and-Permit-Critical-SYN
 match access-group name CoPP-Limit-and-Permit-BGP-SYN
 match access-group name CoPP-Limit-and-Permit-BGPv6-SYN
 match access-group name CoPP-Limit-and-Permit-LDP-SYN



!!!!! Management plane traffic (such as SSH,SNMP)

class-map match-any CoPP-Limit-and-Permit-Management-Plane
 match access-group name CoPP-Limit-and-Permit-TACACS
 match access-group name CoPP-Limit-and-Permit-RADIUS
 match access-group name CoPP-Limit-and-Permit-TELNET-SSH
 match access-group name CoPP-Limit-and-Permit-SNMP
 match access-group name CoPP-Limit-and-Permit-NTP
 match access-group name CoPP-Limit-and-Permit-SNMP-Pollers



!!!!! Management plane SYN traffic

class-map match-any CoPP-Limit-and-Permit-Management-Plane-SYN
 match access-group name CoPP-Limit-and-Permit-TACACS-SYN
 match access-group name CoPP-Limit-and-Permit-TELNET-SSH-SYN



!!!!! Forwarding traffic that is CPU punted

class-map match-any CoPP-Limit-and-Permit-Forwarding-Plane
 match access-group name CoPP-Limit-and-Permit-IP-Options
 match access-group name CoPP-Limit-and-Permit-ICMP
 match access-group name CoPP-Limit-and-Permit-ICMPv6
 match access-group name CoPP-Limit-and-Permit-UDP-Traceroute
 match access-group name CoPP-Limit-and-Permit-TCP-syn-fin-rst
 match access-group name CoPP-Limit-and-Permit-TCP-syn-fin-rst-IPv6
 match access-group name CoPP-Limit-and-Permit-TCP-established
 match access-group name CoPP-Limit-and-Permit-TCP-established-IPv6



!!!!! Traffic we want to always drop

class-map match-any CoPP-Deny-Always
 match access-group name CoPP-Deny-IGMP
 match access-group name CoPP-Deny-IP-Fragments
 match access-group name CoPP-Deny-IPv6-Fragments
 match access-group name CoPP-Deny-ICMP
 match access-group name CoPP-Deny-ICMPv6



!!!!! Catch-all for traffic that doesn't match any of the above ACLS

class-map match-any CoPP-Catch-All
 match access-group name CoPP-Limit-and-Permit-Catch-All
 match access-group name CoPP-Limit-and-Permit-Catch-All-IPv6
! ***********Class Limiting***********

policy-map Control-Plane-Filter-In

 !!!!! Control plane traffic (such as routing protocols)
 class CoPP-Limit-and-Permit-Critical
  police cir 10000000 bc 312500 be 312500 conform-action transmit exceed-action drop violate-action drop


 !!!!! Control plane SYN traffic
 class CoPP-Limit-and-Permit-Critical-SYN
  police cir 250000 bc 7812 be 7812 conform-action transmit exceed-action drop violate-action drop


 !!!!! Management plane traffic (such as SSH,SNMP)
 class CoPP-Limit-and-Permit-Management-Plane
  police cir 1000000 bc 312500 be 312500 conform-action transmit exceed-action drop violate-action drop


 !!!!! Management plane SYN traffic
 class CoPP-Limit-and-Permit-Management-Plane-SYN
  police cir 250000 bc 3125 be 3125 conform-action transmit exceed-action drop violate-action drop


 !!!!! Forwarding traffic that is CPU punted

 class CoPP-Limit-and-Permit-Forwarding-Plane
   police cir 1000000 bc 31250 be 31250 conform-action transmit exceed-action drop violate-action drop


 !!!!! Traffic we want to always drop

 class CoPP-Deny-Always
  drop


 !!!!! Catch-all for traffic that doesn't match any of the above ACLS

 class CoPP-Catch-All
  police cir 500000 bc 15625 be 15625 conform-action transmit exceed-action drop violate-action drop


 !!!!! Any other non-IP traffic such as IS-IS (not being used here) or L2 keepalives could be cought with a class-default
 ! however for many paltforms this breaks a lot of stuff so it's not recommended....

! class class-default
!  police cir 1000000 bc 31250 be 31250 conform-action transmit exceed-action drop violate-action drop


control-plane
 service-policy input Control-Plane-Filter-In

 

These are the outputs from applying CoPP

abr1#show mls qos ip
 QoS Summary [IPv4]:      (* - shared aggregates, Mod - switch module)

      Int Mod Dir  Class-map DSCP  Agg  Trust Fl   AgForward-By   AgPoliced-By
                                   Id         Id
-------------------------------------------------------------------------------
       CPP  6  In CoPP-Limit    0   14   dscp  0      594716644              0
       CPP  6  In CoPP-Limit    0   15   dscp  0      102965249              0
       CPP  6  In CoPP-Limit    0   16   dscp  0      158932401              0
       CPP  6  In CoPP-Deny-    0   17   dscp  0          28080              0
       CPP  6  In CoPP-Catch    0   18   dscp  0              0              0
       CPP  6  In CoPP-Limit    0   19   dscp  0              0              0
       CPP  6  In CoPP-Limit    0   20   dscp  0              0              0
    Vl3002  6  In class-defa   46    7     No  0              0              0
     Gi1/2  6 Out PSN-Real-T    0    8     --  0              0              0
     Gi1/2  6 Out PSN-Applic    0    9     --  0              0              0
     Gi1/2  6 Out PSN-Applic    0   10     --  0         125693          15818
     Gi1/2  6 Out PSN-Applic    0   11     --  0              0              0
     Gi1/2  6 Out PSN-Applic    0   12     --  0           6088              0
     Gi1/2  6 Out class-defa    0   13     --  0    27365727626     1969338894
     Vl104  6 Out PSN-Real-T    0    1     --  0      120806022              0
     Vl104  6 Out PSN-Applic    0    2     --  0              0              0
     Vl104  6 Out PSN-Applic    0    3     --  0         930976              0
     Vl104  6 Out PSN-Applic    0    4     --  0              0              0
     Vl104  6 Out PSN-Applic    0    5     --  0           1856              0
     Vl104  6 Out class-defa    0    6     --  0     2259866025        4922990

       All  6   -    Default    0    0*    No  0   262775558861              0


abr1#show vlan internal usage | i Control
4087 Control Plane Protection


abr1#remote command switch show tcam interface vlan 4087 qos type2 ip
* Global Defaults shared

------------------------------------------------------
QOS Results:
A - Aggregate Policing       F - Microflow Policing
M - Mark                     T - Trust
U - Untrust
------------------------------------------------------
    MAU     any any
    MAU    ospf any any
    MAU    tcp any eq bgp any
    MAU    tcp any eq 646 any
    MAU    udp any eq 646 any
    MAU    udp any eq 3784 any
    MAU    tcp any any eq bgp
    MAU    tcp any any eq 646
    MAU    udp any any eq 646
    MAU    udp any any eq 3784
    MAU    udp host 224.0.0.2 eq 1985 any
    MAU    udp host 224.0.0.102 eq 1985 any
    MAU    udp any host 224.0.0.2 eq 1985
    MAU    udp any host 224.0.0.102 eq 1985
    MAU    tcp 192.168.30.0 0.0.1.255 any eq 22
    MAU    tcp 192.168.30.0 0.0.1.255 any eq ftp
    MAU    udp 192.168.30.0 0.0.1.255 any eq snmp
    MAU    udp 192.168.30.0 0.0.1.255 any eq snmptrap
    MAU    tcp host 192.168.30.50 eq tacacs any
    MAU    tcp host 192.168.30.51 eq tacacs any
    MAU    udp host 192.168.30.50 eq tacacs any
    MAU    udp host 192.168.30.51 eq tacacs any
    MAU    udp host 192.168.30.50 eq 1812 any
    MAU    udp host 192.168.30.50 eq 1813 any
    MAU    udp host 192.168.30.51 eq 1812 any
    MAU    udp host 192.168.30.51 eq 1813 any
    MAU    udp host 192.168.30.20 eq ntp any
    MAU    udp host 192.168.30.22 eq ntp any
    MAU    tcp any eq 22 192.168.30.0 0.0.1.255
    MAU    tcp any eq ftp 192.168.30.0 0.0.1.255
    MAU    udp any eq snmp 192.168.30.0 0.0.1.255
    MAU    tcp any host 192.168.30.50 eq tacacs
    MAU    tcp any host 192.168.30.51 eq tacacs
    MAU    udp any host 192.168.30.50 eq tacacs
    MAU    udp any host 192.168.30.51 eq tacacs
    MAU    udp any host 192.168.30.50 eq 1812
    MAU    udp any host 192.168.30.50 eq 1813
    MAU    udp any host 192.168.30.51 eq 1812
    MAU    udp any host 192.168.30.51 eq 1813
    MAU    udp any host 192.168.30.20 eq ntp
    MAU    udp any host 192.168.30.22 eq 213
    MAU    ip any any
    AT     ip any any

At first the idea that the Software Counters are increasing might seem like a mistake in the ACLs that has caused the CPU destined traffic to also be rate-limited in software (or similar config mistake) however that is not the case. Expected traffic coming into the CPU (such as BGP updates for example) causes the software counters to increase because that traffic is supposed to be processed in software. What we are looking for here is traffic classes where the hardware counters are increasing much faster than the software counters to indicate they are either set too low, or policing a burst of traffic during an attack or network issue for example.

As long as the violations are not increasing at a high rate no traffic that should be sent to the CPU is being restricted. The hardware rate limiters are just that, programmed into the line cards so if during an attack the hardware policers are being hammered then they are doing their job and that’s fine, as long as for example BGP traffic isn’t being completely starved.

abr1#show policy-map control-plane
 Control Plane

  Service-policy input: Control-Plane-Filter-In

  Hardware Counters:

    class-map: CoPP-Limit-and-Permit-Critical (match-any)
      Match: access-group name CoPP-Limit-and-Permit-BGP
      Match: access-group name CoPP-Limit-and-Permit-BGPv6
      Match: access-group name CoPP-Limit-and-Permit-RSVP
      Match: access-group name CoPP-Limit-and-Permit-LDP
      Match: access-group name CoPP-Limit-and-Permit-OSPF
      Match: access-group name CoPP-Limit-and-Permit-OSPFv3
      Match: access-group name CoPP-Limit-and-Permit-HSRP
      Match: access-group name CoPP-Limit-and-Permit-BFD
      police :
        10000000 bps 312000 limit 312000 extended limit
      Earl in slot 6 :
        51302250422 bytes
        5 minute offered rate 85976 bps
        aggregate-forwarded 51302250422 bytes action: transmit
        exceeded 0 bytes action: transmit
        aggregate-forward 74832 bps exceed 0 bps

  Software Counters:

    Class-map: CoPP-Limit-and-Permit-Critical (match-any)
      376908484 packets, 32494165377 bytes
      5 minute offered rate 55000 bps, drop rate 0000 bps
      Match: access-group name CoPP-Limit-and-Permit-BGP
        324057526 packets, 28695202303 bytes
        5 minute rate 49000 bps
      Match: access-group name CoPP-Limit-and-Permit-BGPv6
        181642 packets, 15090592 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-RSVP
        8072 packets, 1276824 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-LDP
        15000928 packets, 1138418002 bytes
        5 minute rate 1000 bps
      Match: access-group name CoPP-Limit-and-Permit-OSPF
        6731250 packets, 731170868 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-OSPFv3
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-HSRP
        30929060 packets, 1913006504 bytes
        5 minute rate 2000 bps
      Match: access-group name CoPP-Limit-and-Permit-BFD
        4 packets, 284 bytes
        5 minute rate 0 bps
      police:
          cir 10000000 bps, bc 312500 bytes, be 312500 bytes
        conformed 376908486 packets, 32494165377 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          transmit
        violated 0 packets, 0 bytes; actions:
          drop
        conformed 55000 bps, exceeded 0000 bps, violated 0000 bps

  Hardware Counters:

    class-map: CoPP-Limit-and-Permit-Management-Plane (match-any)
      Match: access-group name CoPP-Limit-and-Permit-TACACS
      Match: access-group name CoPP-Limit-and-Permit-RADIUS
      Match: access-group name CoPP-Limit-and-Permit-TELNET-SSH
      Match: access-group name CoPP-Limit-and-Permit-SNMP
      Match: access-group name CoPP-Limit-and-Permit-NTP
      Match: access-group name CoPP-Limit-and-Permit-SNMP-Pollers
      police :
        1000000 bps 312000 limit 312000 extended limit
      Earl in slot 6 :
        9305000717 bytes
        5 minute offered rate 9896 bps
        aggregate-forwarded 9305000717 bytes action: transmit
        exceeded 0 bytes action: transmit
        aggregate-forward 2240 bps exceed 0 bps

  Software Counters:

    Class-map: CoPP-Limit-and-Permit-Management-Plane (match-any)
      77496404 packets, 8995276492 bytes
      5 minute offered rate 5000 bps, drop rate 0000 bps
      Match: access-group name CoPP-Limit-and-Permit-TACACS
        30642 packets, 1863750 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-RADIUS
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-TELNET-SSH
        480489 packets, 36072767 bytes
        5 minute rate 2000 bps
      Match: access-group name CoPP-Limit-and-Permit-SNMP
        76917195 packets, 8951213045 bytes
        5 minute rate 4000 bps
      Match: access-group name CoPP-Limit-and-Permit-NTP
        68077 packets, 6126930 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-SNMP-Pollers
        0 packets, 0 bytes
        5 minute rate 0 bps
      police:
          cir 1000000 bps, bc 312500 bytes, be 312500 bytes
        conformed 77496404 packets, 8995276492 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          transmit
        violated 0 packets, 0 bytes; actions:
          drop
        conformed 5000 bps, exceeded 0000 bps, violated 0000 bps

  Hardware Counters:

    class-map: CoPP-Limit-and-Permit-Forwarding-Plane (match-any)
      Match: access-group name CoPP-Limit-and-Permit-IP-Options
      Match: access-group name CoPP-Limit-and-Permit-ICMP
      Match: access-group name CoPP-Limit-and-Permit-ICMPv6
      Match: access-group name CoPP-Limit-and-Permit-UDP-Traceroute
      Match: access-group name CoPP-Limit-and-Permit-TCP-syn-fin-rst
      Match: access-group name CoPP-Limit-and-Permit-TCP-syn-fin-rst-IPv6
      Match: access-group name CoPP-Limit-and-Permit-TCP-established
      Match: access-group name CoPP-Limit-and-Permit-TCP-established-IPv6
      police :
        1000000 bps 31000 limit 31000 extended limit
      Earl in slot 6 :
        16948565405 bytes
        5 minute offered rate 23264 bps
        aggregate-forwarded 16948565405 bytes action: transmit
        exceeded 0 bytes action: transmit
        aggregate-forward 23320 bps exceed 0 bps

  Software Counters:

    Class-map: CoPP-Limit-and-Permit-Forwarding-Plane (match-any)
      85359674 packets, 6650725463 bytes
      5 minute offered rate 7000 bps, drop rate 0000 bps
      Match: access-group name CoPP-Limit-and-Permit-IP-Options
        36055 packets, 3245310 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-ICMP
        79493442 packets, 6241343837 bytes
        5 minute rate 7000 bps
      Match: access-group name CoPP-Limit-and-Permit-ICMPv6
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-UDP-Traceroute
        4408893 packets, 311453581 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-TCP-syn-fin-rst
        1390938 packets, 89258079 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-TCP-syn-fin-rst-IPv6
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-TCP-established
        30346 packets, 5424734 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-TCP-established-IPv6
        0 packets, 0 bytes
        5 minute rate 0 bps
      police:
          cir 1000000 bps, bc 31250 bytes, be 31250 bytes
        conformed 85346535 packets, 6631803600 bytes; actions:
          transmit
        exceeded 12489 packets, 17964959 bytes; actions:
          transmit
        violated 651 packets, 956982 bytes; actions:
          drop
        conformed 7000 bps, exceeded 0000 bps, violated 0000 bps

  Hardware Counters:

    class-map: CoPP-Deny-Always (match-any)
      Match: access-group name CoPP-Deny-IGMP
      Match: access-group name CoPP-Deny-IP-Fragments
      Match: access-group name CoPP-Deny-IPv6-Fragments
      Match: access-group name CoPP-Deny-ICMP
      Match: access-group name CoPP-Deny-ICMPv6
      police :
        8000 bps 1000 limit 1000 extended limit
      Earl in slot 6 :
        2585914 bytes
        5 minute offered rate 0 bps
        aggregate-forwarded 2585914 bytes action: transmit
        exceeded 0 bytes action: transmit
        aggregate-forward 0 bps exceed 0 bps

  Software Counters:

    Class-map: CoPP-Deny-Always (match-any)
      1285120 packets, 115082280 bytes
      5 minute offered rate 0000 bps, drop rate 0000 bps
      Match: access-group name CoPP-Deny-IGMP
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Deny-IP-Fragments
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Deny-IPv6-Fragments
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Deny-ICMP
        103 packets, 9706 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Deny-ICMPv6
        1285017 packets, 115072574 bytes
        5 minute rate 0 bps
      police:
          cir 8000 bps, bc 1500 bytes, be 1500 bytes
        conformed 1285120 packets, 115082280 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          transmit
        violated 0 packets, 0 bytes; actions:
          drop
        conformed 0000 bps, exceeded 0000 bps, violated 0000 bps

  Hardware Counters:

    class-map: CoPP-Catch-All (match-any)
      Match: access-group name CoPP-Limit-and-Permit-Catch-All
      Match: access-group name CoPP-Limit-and-Permit-Catch-All-IPv6
      police :
        496000 bps 15000 limit 15000 extended limit
      Earl in slot 6 :
        0 bytes
        5 minute offered rate 0 bps
        aggregate-forwarded 0 bytes action: transmit
        exceeded 0 bytes action: transmit
        aggregate-forward 0 bps exceed 0 bps

  Software Counters:

    Class-map: CoPP-Catch-All (match-any)
      25991884 packets, 2564942791 bytes
      5 minute offered rate 3000 bps, drop rate 0000 bps
      Match: access-group name CoPP-Limit-and-Permit-Catch-All
        25706499 packets, 2531503884 bytes
        5 minute rate 3000 bps
      Match: access-group name CoPP-Limit-and-Permit-Catch-All-IPv6
        285385 packets, 33438907 bytes
        5 minute rate 0 bps
      police:
          cir 500000 bps, bc 15625 bytes, be 15625 bytes
        conformed 25990453 packets, 2564284549 bytes; actions:
          transmit
        exceeded 1375 packets, 578866 bytes; actions:
          transmit
        violated 56 packets, 79376 bytes; actions:
          drop
        conformed 3000 bps, exceeded 0000 bps, violated 0000 bps

  Hardware Counters:

    class-map: CoPP-Limit-and-Permit-Management-Plane-SYN (match-any)
      Match: access-group name CoPP-Limit-and-Permit-TACACS-SYN
      Match: access-group name CoPP-Limit-and-Permit-TELNET-SSH-SYN
      police :
        248000 bps 3000 limit 3000 extended limit
      Earl in slot 6 :
        0 bytes
        5 minute offered rate 0 bps
        aggregate-forwarded 0 bytes action: transmit
        exceeded 0 bytes action: transmit
        aggregate-forward 0 bps exceed 0 bps

  Software Counters:

    Class-map: CoPP-Limit-and-Permit-Management-Plane-SYN (match-any)
      0 packets, 0 bytes
      5 minute offered rate 0000 bps, drop rate 0000 bps
      Match: access-group name CoPP-Limit-and-Permit-TACACS-SYN
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-TELNET-SSH-SYN
        0 packets, 0 bytes
        5 minute rate 0 bps
      police:
          cir 250000 bps, bc 3125 bytes, be 3125 bytes
        conformed 0 packets, 0 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          transmit
        violated 0 packets, 0 bytes; actions:
          drop
        conformed 0000 bps, exceeded 0000 bps, violated 0000 bps

  Hardware Counters:

    class-map: CoPP-Limit-and-Permit-Critical-SYN (match-any)
      Match: access-group name CoPP-Limit-and-Permit-BGP-SYN
      Match: access-group name CoPP-Limit-and-Permit-BGPv6-SYN
      Match: access-group name CoPP-Limit-and-Permit-LDP-SYN
      police :
        248000 bps 7000 limit 7000 extended limit
      Earl in slot 6 :
        0 bytes
        5 minute offered rate 0 bps
        aggregate-forwarded 0 bytes action: transmit
        exceeded 0 bytes action: transmit
        aggregate-forward 0 bps exceed 0 bps

  Software Counters:

    Class-map: CoPP-Limit-and-Permit-Critical-SYN (match-any)
      0 packets, 0 bytes
      5 minute offered rate 0000 bps, drop rate 0000 bps
      Match: access-group name CoPP-Limit-and-Permit-BGP-SYN
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-BGPv6-SYN
        0 packets, 0 bytes
        5 minute rate 0 bps
      Match: access-group name CoPP-Limit-and-Permit-LDP-SYN
        0 packets, 0 bytes
        5 minute rate 0 bps
      police:
          cir 250000 bps, bc 7812 bytes, be 7812 bytes
        conformed 0 packets, 0 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          transmit
        violated 0 packets, 0 bytes; actions:
          drop
        conformed 0000 bps, exceeded 0000 bps, violated 0000 bps

Below we can see the internal gigabit connection to the RSP and how much traffic we are sending/receiving to/from it, and dropping. When a PFC or DFC decides to punt a packet it instructs the ingress line card to send the packet through the switch fabric to the fabric or bus interface on the supervisor/RSP. The fabric or bus interface forwards the packet to the packet ASIC on the RSP module. The packet ASIC forwards the packet to either the SP or RP CPU.

Each CPU has a separate In-band Interface Channel (IBC) with two input queues. The high priority queue (queue 0) receives packets with the data bus CoS value from 4 to 7 and low priority queue (queue 1) receives packets with the databus CoS value from 0 to 3. See the two RX queues below:

abr1#show ibc
Interface information:
        Interface IBC0/0(idb 0x1D1CBF88)
        5 minute rx rate 147000 bits/sec, 217 packets/sec
        5 minute tx rate 281000 bits/sec, 202 packets/sec
        1121195907 packets input, 92510091102 bytes
        108882982 broadcasts received
        1050940422 packets output, 173920811941 bytes
        117764347 broadcasts sent
        0 Bridge Packet loopback drops
        511989670 Packets CEF Switched, 21 Packets Fast Switched
        0 Packets SLB Switched, 0 Packets CWAN Switched
        Label switched pkts dropped: 10    Pkts dropped during dma: 130
        Invalid pkts dropped: 0    Pkts dropped(not cwan consumed): 0
        Pkts marked to drop by VLAN clients: 0
        IPSEC pkts: 1543
        Xconnect pkts processed: 0, dropped: 0
        Xconnect pkt reflection drops: 0
        Total paks copied for process level 0
        Total short paks sent in route cache 161251443
        Total throttle drops 46    Input queue drops 5316
        total spd packets classified (198955545 low, 345036684 medium, 52655209 high)
        total spd packets dropped (129 low, 1 medium, 0 high)
        spd prio pkts allowed in due to selective throttling (0 med, 0 high)
        IBC resets   = 1; last at 00:07:55.471 BST Fri Jun 6 2014

Driver Level Counters: (Cumulative, Zeroed only at Reset)
          Frames          Bytes
  Rx(0)   55849743        2464316430
  Rx(1)   1065352741      2142727136
  Tx(0)   1050946996      1544836869



abr1#show mls statistics module 6

Statistics for Earl in Module 6

L2 Forwarding Engine
  Total packets Switched                : 2205950654529

L3 Forwarding Engine
  Total packets Processed               : 2205866081585 @ 1260305 pps
  Total packets L3 Switched             : 1974895372912 @ 1167585 pps

  Total Packets Bridged                 : 21411016783
  Total Packets FIB Switched            : 1974895372912
  Total Packets ACL Routed              : 0
  Total Packets Netflow Switched        : 0
  Total Mcast Packets Switched/Routed   : 172466214
  Total ip packets with TOS changed     : 28805093757
  Total ip packets with COS changed     : 33136415156
  Total non ip packets COS changed      : 73930452703
  Total packets dropped by ACL          : 16567242
  Total packets dropped by Policing     : 126602138
  Total packets exceeding CIR           : 0
  Total packets exceeding PIR           : 0

Errors
  MAC/IP length inconsistencies         : 0
  Short IP packets received             : 0
  IP header checksum errors             : 0
  No-route packet drops                 : 991111396
  TTL failures                          : 23942682
  MTU failures                          : 654618

When the IBC controller receives the packet, it copies the packet into IOS input/output memory and raises a Network Input/Output (NetIO) interrupt to the relivant CPU (RP or SP). More info here.