Date created: Thursday, February 14, 2013 9:38:47 AM. Last modified: Monday, October 9, 2017 8:10:38 PM
6500/7600 Chassis Architecture
6500 Architecture
http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd80673385.html
http://www.cisco.com/en/US/products/hw/switches/ps708/products_qanda_item09186a00809a7673.shtml
https://supportforums.cisco.com/docs/DOC-18210
https://supportforums.cisco.com/document/71201/switch-fabric-troubleshooting-tips
7600 Architecture
http://www.cisco.com/web/YU/events/expo_08/pdfs/Arhitektura_C7600_Aleksandar_Vidakovic.pdf
http://www.cisco.com/warp/public/cc/pd/rt/7600osr/prodlit/76osr_ov.pdf
http://211.79.59.247/Course/QoS20071122/slide/Day2-03-7600QoSArchitecture.pdf
http://www.cisco.com/c/en/us/products/collateral/routers/7606-router/product_data_sheet0900aecd8057f3c8.html
www.cisco.com/c/en/us/td/docs/routers/7600/Hardware/Chassis_Installation/7600_Series_Router_Installation_Guide/cis_76xx/osr_over.html
MSFC (multilayer switch feature card)
Multilayer Switch Feature Card is the Layer 3 switching engine that sites on the Catalyst Supervisor as a daughter card. The MSFC is an integral part of the Supervisor Engine, providing high performance, multilayer switching and routing intelligence. On the MSFC daughter card, the route processor (RP) is located on the MSFC itself. Equipped with a high performance processor, the MSFC runs layer 2 protocols on one CPU and layer 3 protocols on the second CPU. These include routing protocol support, layer 2 protocols (Spanning Tree Protocol and VLAN Trunking Protocol for example), and security services.
The control plane functions in the Cisco Catalyst 6500 are processed by the MSFC and include handling Layer 3 routing protocols, maintaining the routing table, some access control, flow initiation, and other services not found in hardware. Performance of the control plane is dependent on the type and number of processes running on the MSFC. The MSFC3 can support forwarding rates up to 500Kpps. The MSFC provide a means to perform Multilayer Switching (MLS) and interVLAN routing.
The MSFC builds the Cisco Express Forwarding information Base (FIB) table in software and then downloads this table to the hardware Application-specific-integrated circuits (ASICs) on the PFC and DFC (if present) that make the forwarding decisions for IP unicast and multicast traffic.
PFC (policy feature card)
The PFC3 is the ASIC-based forwarding engine daughtercard for the Sup720; the DFC3 is the ASIC-based forwarding engine daughtercard for various fabric-enabled linecards (CEF256, CEF720). Contains the ASICs that are used to accelerate Layer 2 and Layer 3 switching, store and process QoS and security ACLs, and maintain NetFlow statistics.
The PFC3/DFC3 generation is built upon a forwarding architecture known as EARL7. Within this generation, there are three different versions - 'A', 'B', and 'BXL' - that are all based on the same fundamental technologies but that each have incremental functionality. 'A' is the standard offering; 'B' is the intermediate option, and 'BXL' is the high-end option.
The PFC contains a Layer 2 and a Layer 3 forwarding engine.
DFC (distributed feature card)
The Catalyst 6500 & 7600 architectures support the use of Distributed Forwarding Cards (DFC). Distributed Forwarding Card is a combo daughter card comprising a MSFC and PFC used by a fabric enabled linecards to perform distributed switching. DFCs are located in linecards, not in Supervisors.
A DFC is used to hold a local copy of the forwarding tables (constructed by the MSFC) along with Security and QoS policies to facilitate local switching on the linecard. The DFC3A is available as an option on CEF256 and CEF720 based linecards. The DFC3B and DFC3BXL were introduced for linecards to operate with the Supervisor 720 equipped with PFC3B and PFC3BXL. The last generation of DFC, the DFC3C, is available as an option on the CEF720 based linecards but are integrated on the latest generation linecards, the WS-X6708 and WS-X6716. Update: DFC4-A, DFC4-AXL, DFC4-E, or
DFC4-EXL were released for use with the Sup2T supervisor. Modules that are equipped with DFC4 daughter cards are only supported by the Supervisor Engine 2T with Cisco IOS software release 12.2(50)SY or later. The DFC4 daughter card is shipped factory-installed on the WS-X69xx and the WS-X68xx series line cards (except for the WS-X6816-GBIC Ethernet module).
It is important to note that there are some operational considerations that can impact the ability of the Catalyst 6500 system to provide specific QoS features. This can happen when you mix different generations of PFC's and DFC's together. The rule is that the system will operate at the lowest common feature denominator.
Some example DFCs;
|
WS-F6K-DFC4-A |
Distributed Forwarding Card-4-A |
|
WS-F6K-DFC4-AXL |
Distributed Forwarding Card-4-AXL |
|
WS-F6K-DFC4-E |
Distributed Forwarding Card-4-E |
|
WS-F6K-DFC4-EXL |
Distributed Forwarding Card-4-EXL |
|
WS-F6700-DFC3CXL |
Distributed Forwarding Card-3CXL |
|
WS-F6700-DFC3C |
Distributed Forwarding Card-3C |
|
WS-F6700-DFC3BXL |
Distributed Forwarding Card-3BXL |
|
WS-F6700-DFC3B |
Distributed Forwarding Card-3B |
|
WS-F6700-DFC3A |
Distributed Forwarding Card-3A |
The following is a list of 67xx series line cards are upgradeable using DFC4s so that they may be used with a Sup2T:
- With a DFC4 or DFC4XL upgrade (WS-F6k-DFC4-A, WS-F6k-DFC4-AXL): WS-X6704-10GE, WS-X6724-SFP, WS-X6748-SFP, WS-X6748-GE-TX
- With a DFC4 or DFC4XL upgrade (WS-F6k-DFC4-E, WS-F6k-DFC4-EXL): WS-X6716-10G-3C, WS-X6716-10G-3CXL, WS-X6716-10T-3C, WS-X6716-10T-3CXL
6708-10G is not upgradable with a DFC4 so not usable with a Sup2T.
All 69xx series line cards use 40Gbps fabric channel connections (or faster). 67xx series cards that are upgraded with DFC4s still only have 20Gbps fabric channel connections.
References:
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/hardware/Config_Notes/OL_24918.pdf
http://www.cisco.com/c/en/us/products/collateral/interfaces-modules/catalyst-6500-series-supervisor-engine-2t/data_sheet_c78-648214.html
Centralized Forwarding Card (CFC)
CFC is a centralized forwarding card for the switching modules which makes IPv4 routing lookups over the PFC. A CFC does not support local forwarding lookups, the forwarding lookup is done by the PFC in the Supervisor. As the forwarding decisions are centralized, the PFC performance, FIB entries, ACL and lables are shared among the line cards that uses the Supervisor PFC for forwrding. WS-F6700-CFC is the CFC card used on WS-X67xx Ethernet Modules. This daughter card is supported only by the Supervisor Engine 720.
For clarity; the same line card with a DFC or CFC will forward traffic at the same PPS rate as long as the CFC card can make enough lookups to the PFC on the supervisor (where as a DFC enabled line card stores the forwarding tables locally). With CFC or DFC the traffic is forwarded across the crossbar fabric as long as the card is fabric enabled (such as CEF256 or CEF720 line cards). Mixing DFC and CFC cards has no impact, the CFC cards will make lookups to the supervisor PFC and DFC cards will hold the forwarding information locally. However when mixing different DFC line cards, they will all drop to the lowest shared type/version (this includes the PFC, the PFC will slow to the slowest DFC present).
Reference: https://supportforums.cisco.com/docs/DOC-21377
Bus and Fabric Connections
The Catalyst 6500 & 7600 supports two different switch backplanes, the crossbar switch fabric and bus backplanes. The crossbar switch fabric is the high-capacity backplane that is used by the CEF720 and CEF2T generation of line cards to optimize switching performance. A second backplane (referred to as the “bus” backplane) is also present to support older line cards like the WS-X61xx series cards, supported service modules, and line cards that do not utilize a local CFC or DFC for forwarding lookups.
The crossbar switch fabric provides a set of fabric channels (or data paths) that are assigned to the slots in the chassis where line cards are inserted. This is referred to collectively as the crossbar switch backplane. This array of fabric channels provides an any-to-any (full-mesh) connection option for the attached line card to forward data over a dedicated path to any other line card installed in the chassis. When using an RSP720 the total aggregate chassis throughput for a 7600 is 720Gbps using CEF720 line cards all with DFCs. When using a SUP2T the total aggregate chassis throughput for a 6500 is 2Tbps when using only cards with dual 40Gbps fabric channels and DFCs.
The bus backplane is a 16 Gbps (full duplex) shared data bus that is used to provide a connection between attached “classic” line cards.
With an RSP720 the DBUS (data bus) is clocked at 62.5Mhz and can transfer 32 bytes per cycle. CFC line cards will send lookup data over the DBUS to the supervisor PFC and receive a response on the RBUS (result/return bus, which is actually sent to all cards but they will all ignore it except for the card that transmitted the original lookup request on the DBUS). IPv4 lookups use 64 bytes so require 2 clock cycles, MPLS and IPv6 use 96 bytes and thus require 3 clock cycles. This means 62.5MHz / 2 cycles = 31.25Mpps forwarding lookup rate for IPv4 traffic and 62.5MHz / 3 cycles = 20.83Mpps lookup rate for IPv6 and MPLS traffic. If traffic requires recirculation (such as certain QoS on MPLS features or H-FIB/FIB indirection) then the Pps rate for that traffic is halved. In the SUP2T the data bus operates at 62.5Mhz and is 256bits wide. Also with the SUP2T, the bridge ASIC provides the interface through which those classic line cards can communicate with the PFC4 and MSFC5 for data processing services.
6500 Chassis:
Standard EOL 6500 chassis are the 6503/6506/6509/6513 chassis. Only the E series chassis support the Sup2T/2T-EX modules such as 6513-E. The E-chassis ranges are 6503-E/6504-E/6506-E/6509-E/6509-V-E/6513-E.
7600 Chassis:
Hardware: Cisco 7609-S Chassis
Backplane: 720 Gbits
Notes: 21U, 2x Supervisor Slots, 7xLine Card Slots, Dual Fan Trays, Dual PSUs. Shared switching bus total aggregate bandwidth is 32Gbps / 15M PPS or with RSP720 using CEF720 line cards = 720 Gbits / 400M PPS crossbar switching fabric. Each line card slot has 1x 8Gbps shared switching fabric bus connection or when using RSP720 each slot has 2x full duplex 20Gbps crossbar fabric channels, any cards over 40Gbps will be oversubscribed.
Hardware: Cisco 7606-S Chassis
Backplane: 480 Gbits / upto 240Mpps (using DFCs)
Notes: 7U, 2x Supervisor Slots, 4x Line Card Slots, Singel Fan Tray, Dual PSUs. Shared switching bus total aggregate bandwidth is 32Gbps with SUP32 or with either SUP720 or RSP720 it is 720Gbps (when using CEF720 line cards) of crossbar switching fabric bandwidth. Each line card slot has 1x 8Gbps shared switching fabric bus connection or when using RSP720 each slot has 2x full duplex 20Gbps crossbar fabric channels, any cards over 40Gbps will be oversubscribed.
1Gbps Linecards:
Hardware: WS-X6348-RJ-45Backplane: 1x Shared bus connection
Notes: 48xFE, 128k Per port buffers, 32k MAC addresses per port, supports 1000 VLANs overall, upgradeable to PoE with WS-F6K-VPWR
Hardare: WS-X6548-RJ-45
Backplane: 1x Shared bus connection, 1x 8 Gbits switch fabric connection (requires DFC)
Notes: 48xFE, 1MB Per port buffer, upgradable to DFC
Hardware: WS-X6548-GE-TX
Backplane: 2x 8 Gbits switch fabric connection (CEF256)
Notes: 48xGig, 1MB port buffer per 8 ports, oversubscription is 8:1, upgradeable to PoE with WS-F6K-VPWR-GE, upgradable to DFC
Hardware: WS-X6516A-GBIC
Backplane: 2x 8 Gbits switch fabric connection (CEF256)
Notes: 16xGig GBIC, 1MB Per port buffer, upgradable to DFC
Hardware: WS-X6748-GE-TX
Backplane:48xGig, 40Gbps full duplex (CEF720), 1.3MB buffer per port
Notes: 48 port 10/100/1000 Ethernet
Hardware: WS-X6724-SFP
Backplane: 24x 1Gig SFP slots, port buffers: Rx-166KB Tx-1.17MB, 20Gbps full duplex (CEF720), requires DFC3 card for 26Mpps sustained
Notes: 48 SFP slots
Hardware: WS-X6748-SFP
Backplane: 48x 1Gig SFP slots, port buffers: Rx-166KB Tx-1.17MB, 40Gbps full duplex (CEF720)
Notes: 48 SFP slots
Linecard references: http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-6500-series-switches/product_data_sheet0900aecd8017376e.html
http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_bulletin0900aecd800f6e27.html
http://www.cisco.com/c/en/us/products/collateral/routers/7600-series-routers/product_data_sheet0900aecd8072579f.html
10Gig Linecards:
Hardware: WS-X6908-10G-2T / WS-X6908-10G-2TXL
Backplane: 80Gbps full duplex (requires Sup2t), oversubscription is 1:1 with 256MBs of buffers per port
Notes: 8x X2 10Gbps ports supporting 60Mpps L2, IPv4 and MPLS, 30Mpps IPv6. Uses DFC4or DFC4XL.
Hardware: WS-X6816-10G-2T / WS-X6816-10G-2TXL
Backplane: 40Gbps (2x10Gbps), oversubscription is 4:1 using 16 ports (oversubscription mode) or 1:1 using 4 ports (performance mode), oversubscription mode gives 90 MB per port group, performance mode gives 256 MB per port (128 MB for ingress and 128 MB for egress)
Notes: 16x 10G X2 ports supporting 60Mpps L2, IPv4 and MPLS, 30Mpps IPv6. Uses DFC4or DFC4XL.
Hardware: WS-X6816-10T-2T / WS-X6816-10T-2TXL
Backplane: 40Gbps (2x10Gbps), oversubscription is 4:1 using 16 ports (oversubscription mode) or 1:1 using 4 ports (performance mode), oversubscription mode gives 90 MB per port group, performance mode gives 256 MB per port (128 MB for ingress and 128 MB for egress)
Notes: 16x 10G RJ45 ports supporting 60Mpps L2, IPv4 and MPLS, 30Mpps IPv6. Uses DFC4or DFC4XL.
Hardware: WS-X6716-10T-3C
Backplane: 16xRJ45 10Gig ports (Cat6a/7), 256k Routes, 90MB buffer per port, 80Gbps full duplex, Oversub 4:1
Notes: 16-Port 10 Gigabit Ethernet Copper Module with DFC3C
Hardware: WS-X6716-10T-3CXL
Backplane: 16xRJ45 10Gig ports (Cat6a/7), 1M Routes, 200MB buffer per port, 80Gbps full duplex, Oversub 4:1
Notes: 16-Port 10 Gigabit Ethernet Copper Module with DFC3CXL
Hardware: WS-X6716-10G-3C
Backplane: 16x X2 10Gig optics ports, 256k Routes, 90MB buffer per port, 80Gbps full duplex, Oversub 4:1
Notes: 16-Port 10 Gigabit Ethernet Module with DFC3C, requires X2
Hardware: WS-X6716-10G-3CXL
Backplane: 16x X2 10Gig optics ports, 1M Routes, 200MB buffer per port, 80Gbps full duplex, Oversub 4:1
Notes: 16-Port 10 Gigabit Ethernet Module with DFC3CXL, requires X2
Hardware: WS-X6708-10G-3C
Backplane: 8x X2 10Gig optics ports, 256k Routes, 200MB buffer per port, 80Gbps full duplex, Oversub 2:1
Notes: 8-Port 10 Gigabit Ethernet Module with DFC3C, requires X2
Hardware: WS-X6708-10G-3CXL
Backplane: 8x X2 10Gig optics ports, 1M Routes, 200MB buffer per port, 80Gbps full duplex, Oversub 2:1 (using ports 1, 2, 5 and 6 only gives 1:1 subscription)
Notes: 8-Port 10 Gigabit Ethernet Module with DFC3CXL, requires X2
Hardware: WS-X6704-10GE
Backplane: 4x XENPACK 10Gig optic ports, CFC dependant route count, 16MB buffer per port, 80Gbps full duplex, Oversub 1:1
Notes: 4-Port 10 Gigabit Ethernet Module, requires XENPAK
10 Gig line cards reference:
http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/data-center-switching/net_qanda0900aecd80534905.html
http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-6500-series-switches/data_sheet_c78-648212.html
http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-6500-series-switches/data_sheet_c78-451794.html
http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/product_data_sheet09186a00801dce34.html
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns668/net_business_benefit0900aecd805348f3.html
Sups/RSPs:
Supervisor: Sup32
Backplane: 32Gbps shared bus, 15Mpps for IPv4, MSFC2A onboard, default 512MB DRAM upgradeable to 1GB, hardware forwarding support from PFC3B onboard
Supervisor: Sup720
Backplane: 720Gbps, 400Mpps with CEF, MSFC3 onboard, default 512MB DRAM upgradeable to 1GB on 720 and 720-3B, 1GB default on 720-3BXL, hardware forwarding support from either PFC3A, PFC3B, or PFC3BXL onboard
Supervisor: RSP720-3B-?
Backplane: Memory route processor/switch processor: 512 MB/512 MB,
Supervisor: RSP720-3BXL-?
Backplane: Memory route processor/switch processor: 1 GB/1 GB,
Supervisor: RSP720-3C-GE
Backplane: 720Gbps, Memory route processor/switch processor: 1 GB/1 GB, routes: 256k IPv4 128,000k IPv6k, PFC3C with MSFC4
Supervisor: RSP720-3CXL-GE
Backplane: 720Gbps, Memory route processor/switch processor: 2 GB/1 GB, routes: 1M IPv4 512K IPv6, PFC3CXL with MSFC4
Notes: 1x1Gig (Wired), 1x1Gig/SFP Combo, PFC-3CXL, MSFC4
Supervisor: RSP720-3CXL-10GE
Backplane: 720 Gbps, Memory route processor/switch processor: 2 GB/1 GB, routes: 1M IPv4 512K IPv6, PFC3CXL with MSFC4
Notes: 1x1Gig (Wired), 2x1Gig SFP, 2x10G X2, PFC-3CXL, MSFC4
Supervisor: 2T-XL
Backplane, 2080Gbps (2Tbps), uses MSFC5, support PFC4 and PFC4XL, supports DFC4 and DFC4XL, up to 60 Mpps for L2, IPv4, and MPLS traffic, up to 30 Mpps for IPv6 traffic. Sup2T supports 80Gbps per slot full-duplex (80Gbps in each direction simultaneously, but not 160Gbps in one direction only), it uses 2x40Gbps connections).
References: http://www.cisco.com/en/US/prod/collateral/modules/ps2797/ps5972/product_data_sheet0900aecd801c5cab_ps708_Products_Data_Sheet.html
http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/white_paper_c11-676346.html
http://www.cisco.com/en/US/products/hw/switches/ps708/prod_models_comparison.html#~tab-b
http://www.cisco.com/en/US/prod/collateral/routers/ps368/product_data_sheet0900aecd8057f3b6.html
Previous page: 2960
Next page: 6500/7600 Conf Register