Continuous progress in data center technology is propelling a transition to 40 Gigabit Ethernet. A decade ago, servers became powerful enough to saturate Gigabit Ethernet interfaces, prompting a move to 10 Gigabit Ethernet. Today, the power of Intel® Xeon® processors combined with massive amounts of memory allow servers to saturate 10 Gigabit Ethernet links, increasing the demand for more network bandwidth and for more headroom to absorb spikes in workload demands.
Data center networks are transitioning to 40 Gigabit Ethernet to accommodate the increased network and storage traffic imposed by an increasing number of application workloads. With virtualization now the norm, massive east-west scalability is required, as well as the ability to handle more bandwidth between servers in virtualization clusters. Many organizations are accommodating this shift by adopting the leaf-and-spine network topology provided by Cisco® Application Centric Networking (Cisco ACI™). This approach delivers more consistent latency and greater east-west bandwidth while helping maintain security and Quality of Service (QoS) through policy-based network infrastructure deployment and management.
The problem is how to join servers with a need for 40 Gbps of bandwidth with a 40-Gbps-ready data center network so that the transition can be graceful, with minimal cost and disruption. Existing servers need to be accommodated, and a plan needs to be in place to support new servers. With traditional infrastructure, each server would need new network interfaces and cabling to top-of-rack switches, and then new fiber from top-of-rack switches to aggregation-layer switches—requiring tasks taking hours of administrator time per server and significant recabling time and expense.
The third generation of Cisco Unified Fabric for the Cisco Unified Computing System™ (Cisco UCS®) makes the transition to 40 Gigabit Ethernet smooth and cost effective. Your upgrade concerns are addressed with a new family of 40-Gbps fabric components that can be swapped into existing systems or provisioned with new Cisco UCS instances:
● Cisco UCS 6300 Series Fabric Interconnects provide 40 Gigabit Ethernet connectivity upstream to access-layer switches and downstream to Cisco UCS servers.
● Cisco UCS 2304 Fabric Extenders provides up to 160 Gbps of bandwidth to each of the two fabrics in a Cisco UCS 5108 Blade Server Chassis, for up to 320 Gbps of bandwidth using two fabric extenders and only eight cables.
● Cisco Nexus® 2348UPQ 10GE Fabric Extenders bring the 40-Gbps unified fabric to rack servers
● Cisco UCS Virtual Interface Card (VIC) 1300 platform adapters adjust automatically to support the new fabric, eliminating the need to physically modify servers to increase connectivity.
The approach to connectivity used in Cisco UCS continues a long history of technology improvements along with investment protection. The first generation of fabric interconnects supported up to 80 Gbps of bandwidth to each blade server chassis, and the next generation supported up to 160 Gbps of bandwidth. The third generation now supports up to 320 Gbps of bandwidth per chassis with only a swap of fabric components. This approach makes the best networking technology available while protecting your investment in servers, chassis, and racks. Benefits include:
● 40 Gbps of bandwidth upstream and downstream: Fewer cables are needed to provide more bandwidth, and each 40-Gbps cable is more efficient. More traffic can pass on a single 40-Gbps interface than on four 10-Gbps interfaces configured as a port channel.
● Fewer cables needed to increase server connectivity, with up to 320 Gbps of network connectivity per Cisco UCS 5108 Blade Server Chassis: This is the same chassis that was introduced in 2009—designed with the vision to support multiple generations of networking capabilities in the same simplified housing.
● No new upstream cabling needed if you have fiber connecting your current fabric interconnects to your data center network: Using Cisco bidirectional (BiDi) transceivers, you can use your existing 10-Gbps fiber to carry 40 Gbps of network bandwidth.
● The same unified fabric, only faster: The fabric now supports 40 Gigabit Ethernet and Fibre Channel over Ethernet (FCoE), with the capability to connect to blade and rack servers and virtual machines with equivalent visibility and control.
● The same storage connectivity: Cisco UCS allows your servers to access shared IP-based storage anywhere on your IP networks, and native Fibre Channel connectivity to connect to Fibre Channel-based enterprise shared storage devices.
● Deterministic latency: This feature is one of the fabric interconnect’s design criteria. All east-west traffic within a single Cisco UCS instance requires only a single network hop, simplifying virtual machine placement.
● No change in policy or operations: The same familiar role- and policy-based automation of Cisco UCS management continues to help keep your operating costs down and make your data center more agile than your competitors’.
● Ready for a 40-Gbps data center: Whether you use Cisco Nexus 7000 or 9000 Series Switches, this fabric upgrade prepares you for 40-Gbps speeds. If you have deployed Cisco ACI in your data center, Cisco UCS is ready to interconnect servers and virtual machines and appliances through the Cisco ACI fabric’s encapsulation normalization mechanisms.
Cisco UCS Fabric for the Future
The Cisco UCS network fabric is implemented with an end-to-end architecture that uses Cisco Unified Fabric and Cisco Fabric Extender Technology (FEX Technology) to connect every Cisco UCS component with a single network and a single network layer. The result is a single, large, virtual blade server chassis that can accommodate both blade and rack servers and connect any two entities, physical or virtual, with a single network hop.
Single Unified Network
The same network brings LAN, SAN, and management connectivity to each rack and blade server in Cisco UCS using Cisco Unified Fabric. Now every server—rack or blade—has equal access to all network resources, eliminating the need to support three physical networks, each with its own Network Interface Cards (NICs), Host Bus Adapters (HBAs), transceivers, cables, and top-of-rack switches. Infrastructure silos are eliminated because software—not cabling—determines the way that each server connects to the network, making every server ready to support any workload at a moment’s notice through automated configuration.
This “wire once” capability is a core feature of Cisco UCS: you can wire your system once for the bandwidth you require, and add features (such as access to shared storage) through software, not more cabling. If you need more bandwidth for any I/O modality, you add more cables, not more infrastructure. Consistent forwarding policies using technologies such as VLANs and QoS controls align your networking features and performance with business needs.
Single Network Layer
A single network layer brings the unified fabric to every blade chassis and server rack. FEX Technology reduces three network layers to one, eliminating hypervisor switches and blade-chassis-resident switches and replacing them with a single point of management and connectivity (Figure 1). In Cisco UCS, low-cost, lowpower, and zero-management Cisco UCS and Cisco Nexus fabric extenders pass all traffic from servers and virtual machines to the system’s fabric interconnects. This approach reduces infrastructure cost per server and interconnects servers with a single hop, in contrast to the multiple hops and higher latency inherent in traditional environments. The combination of the system’s fabric interconnects, fabric extenders, and Cisco UCS VICs establishes a centrally managed yet physically distributed system that can contain both blade and rack servers.
Virtual Interfaces Allow the Network Fabric to Treat Every Link as if It Were on a Separate Cable
Connecting to Physical Servers and Virtual Machines
Cisco VICs directly connect the network to physical rack and blade servers and virtual machines. Static PCI Express (PCIe) interfaces are configured on demand to adapt servers to meet the best-practice needs of any operating system or hypervisor, providing the NICs and HBAs they need without requiring any special driver software. Dynamic interfaces are configured and attached to virtual machines, giving them direct access to the network. After they are attached, a virtual machine’s network interfaces migrate from server to server along with the virtual machine, simplifying virtual network management and providing air-gap security to virtual environments. Every network link has the same security as if it were a physical cable. This Cisco Data Center Virtual Machine Fabric Extender (VM-FEX) technology coordinates the movement of virtual network connectivity in concert with the leading hypervisors.
Within a Cisco UCS instance, every interface on a Cisco VIC (or on a third-party converged network adapter [CNA]) is terminated at a virtual interface within the fabric interconnect. Virtual interfaces are not permanently bound to a specific physical interface on the fabric interconnect. Simply by changing the virtual-to-physical mapping in the fabric interconnect, you can move virtual NICs (vNICs) from server to server. Terminating every connection at a virtual interface allows physical and virtual machines to be treated equivalently, with the same visibility and control as a physical network. This approach securely isolates each link and subjects each link to QoS controls as if the traffic were traversing its own physical cable (Figure 1).
Based on Open Standards and Accelerated by Silicon
Cisco has a tradition of using existing open standards (such as FCoE) and promoting new ones as necessary (such as the IEEE 802.1BR Bridge Port Extension standard). Cisco uses off-the-shelf silicon for common functions and designs custom silicon to incorporate Cisco innovations and to increase performance. Organizations gain the functional benefits of open-standard innovations with the improved performance that hardware implementations deliver (Figure 2).
Cisco Adopts and Creates New Open Standards, Accelerating Performance Through Custom Silicon
The implications of the Cisco UCS I/O architecture are profound. The system is logically centralized with a single point of management for all I/O in the system: network, storage, and management. The system is physically distributed so that a single management domain can span both blade server chassis and racks, bringing up to 160 servers into what can be viewed as a single logical blade server chassis. The result is dramatically reduced cost and complexity compared to traditional rack and blade server deployments.
Advanced technology such as multihop FCoE and Cisco Direct Connect technology gives organizations the flexibility to integrate servers with shared storage as business needs dictate, without the limits of traditional network. Open-standards–based separation between Ethernet traffic lanes helps ensure lossless handling of FCoE storage traffic and secure separation from other traffic types.
The new fabric components are essential to the Cisco UCS architecture because they provide a single point of connectivity and management for the entire system.
Figure 3 illustrates the Cisco UCS product family and connectivity options.
● Fabric interconnects: Cisco UCS 6332 or 6332-16UP Fabric Interconnects are typically deployed in an active-active high-availability configuration. If you want to upgrade an existing system with 40-Gbps connectivity to upstream data center switches, you can replace your exiting fabric interconnects, and they will support 10-Gbps of connectivity to existing downstream servers and fabric extenders.
● Blade-chassis-resident fabric extenders: To bring the 40-Gbps fabric to blade server chassis, Cisco UCS 2304 Fabric Extenders provide 40 to 160 Gbps of connectivity between one of the chassis’ two fabrics and one of the fabric interconnects. With a pair of these devices supporting two fabrics, 320 Gbps of connectivity can be established to each chassis. To preserve any existing investment, the Cisco UCS 6300 Series Fabric Interconnects support the preceding generation of 10-Gbps fabric extenders: the Cisco UCS 2204XP (up to 80 Gbps per chassis) and 2208 (up to 160 Gbps per chassis) Fabric Extenders. This support allows you to upgrade your fabric interconnects and your blade server chassis as your connectivity demands require.
● Top-of-rack fabric extenders: Cisco Nexus fabric extenders bring the unified fabric to the top of every rack, supporting greater scale through oversubscription. For applications requiring dedicated 40-Gbps connectivity, Cisco UCS C-Series Rack Servers can be connected directly to the fabric interconnects; otherwise, you can use the fabric extenders to integrate more servers into the system.
Cisco UCS Architecture and Connectivity Options
To bring the 40-Gbps fabric to the top of the rack, the Cisco Nexus 2348UPQ 40GE Fabric Extender connects up to six 40 Gigabit Ethernet links to the fabric interconnects and supports 1 and 10 Gigabit Ethernet connections downstream to rack servers. The Cisco UCS 6300 Series also can support existing Cisco Nexus 2232PP 10GE Fabric Extenders but with multiple 10 Gigabit Ethernet links between the fabric extenders and the fabric interconnects.
● Cisco UCS VIC 1300 platform: The unified fabric terminates with Cisco or third-party CNAs. The most popular form of CNA is a Cisco VIC, which supports more than 256 (more or less, depending on the model) PCIe-compliant devices that can be created on demand through Cisco UCS Manager. Applications with complex requirements such as VMware vSphere, with best practices dictating dual Fibre Channel connectivity to shared storage, a separate management network, and a separate redundant VMware vMotion network in addition to production IP networks, can be accommodated with a single adapter. Virtual machines can be connected directly to network interfaces that avoid hypervisor intervention and which move from server to server whenever the virtual machine migrates. As described later in this document, the Cisco UCS VIC 1300 platform automatically adapts to the presence of a 40-Gbps unified fabric and makes 40 Gigabit Ethernet NICs available to be programmed on demand.
Upgrade Path with Investment Protection
Simply by swapping out fabric components and cabling within racks, you can upgrade your existing blade chassis and rack servers to 40 Gigabit Ethernet connectivity without changing the network adapters in your servers. (This capability assumes that blade servers are equipped with Cisco UCS VIC 1340 adapters with port expander cards, and that rack servers are equipped with the Cisco UCS VIC 1285, 1385, or 1387.) Cabling to upstream 40-Gbps access-layer switches is simple: with Cisco BiDi technology, you can replace transceivers and upgrade to 40 Gbps without having to lay new fiber. Downstream to blade servers, the increase in capacity is impressive: with only eight cables, each blade server chassis can support up to 320 Gbps of network connectivity, with 80 Gbps of connectivity to each halfwidth blade server. Rack servers can be connected with less oversubscription using Cisco Nexus 2348UPQ fabric extenders and can still be connected using existing 10 Gigabit Ethernet cabling if desired.
The third generation of Cisco Unified Fabric is designed to fit easily into a Cisco UCS environment comprising Cisco UCS fabric extenders, VICs, and Cisco Nexus 9300 platform switches.
Cisco UCS 6300 Series Fabric Interconnects have been carefully designed to combine the cost-saving advantages of merchant silicon with custom Cisco innovations. They provide an intelligent progression from previous-generation fabric interconnects (Table 1).
Two Cisco UCS 6332 models are offered to provide deployment flexibility (Table 2). The Cisco UCS 6300 Series requires Cisco UCS Manager Release 3.1 operating software.
Table 1. Cisco UCS Fabric Interconnect Configuration Limits
Unicast MAC addresses per fabric interconnect
Multicast MAC addresses per fabric interconnect
Active VLANs per Cisco UCS domain
Spanning Tree Protocol logical interfaces and VLAN port count
Internet Group Management Protocol (IGMP) groups
Virtual interfaces (VIFs)
Virtual HBAs (vHBAs)
Fibre Channel login (FLOGI) instances
Chassis per Cisco UCS domain
IP storage appliances per fabric interconnect
Table 2. Cisco UCS 6300 Series Fabric Interconnects
Cisco UCS 6332
Cisco UCS 6332-16UP
32-port fabric interconnect
40-port fabric interconnect
32 fixed ports:
● 26 x 40-Gbps Quad Enhanced Small Form-Factor Pluggable (QSFP+) modules or 96 x 10-Gbps SFP+ modules with a QSFP–to–4 SFP breakout cable or Cisco QSFP to SFP or SFP+ Adapter (QSA) module
● 6 x 40-Gbps uplinks
40 fixed ports:
● 16 unified ports (1/10-Gbps SFP+ or 4/8/16-Gbps Fibre Channel)
● 18 x 40-Gbps QSFP+ modules or 72 x 10-Gbps SFP+ modules with a QSFP–to–4 SFP breakout cable or QSA module
● 6 x 40-Gbps uplinks
2 (1+1 redundant, hot swappable)
2 (1+1 redundant, hot swappable)
3+1 (port-side exhaust, hot swappable)
4000 3+1 (port-side exhaust, hot swappable)
Cisco UCS 6332 Fabric Interconnect
The Cisco UCS 6332 32-Port Fabric Interconnect is a 1-Rack-Unit (1RU) 40 Gigabit Ethernet and FCoE interconnect offering up to 2.56-Gbps throughput and up to 32 ports (Figure 4).
Cisco UCS 6332 Fabric Interconnect Block-Level Architecture
● The switch offers two high-availability ports for interconnection with a paired Cisco UCS 6300 Series Fabric Interconnect, and it supports a total of 32 x 40-Gbps fixed Ethernet and FCoE ports.
● You can use 26 of the 40-Gbps QSFP+ ports at the native 40-Gbps speed. You can convert 24 of these ports to four 10-Gbps ports by inserting a QSFP–to–4 SFP breakout cable, yielding a total of up to 96 x 10-Gbps SFP+ ports. The remaining two 40-Gbps ports can be converted to 10-Gbps though a QSA module, to bring the total to 98 x 10-Gbps SFP+ ports.
● Six additional 40-Gps QSFP+ ports operate at 40 Gbps.
Two power supplies (hot swappable, 1+1) and four fans (hot swappable, 3+1) insert into the front of the chassis with front-to-back cooling. Server-facing ports are at the rear of the chassis to simplify connections to servers and fabric extenders. Serial and management ports are provided at the front of the chassis for easy access at the front of the rack.
Cisco UCS 6332-16UP Fabric Interconnect
The Cisco UCS 6332-16UP 40-Port Fabric Interconnect is a 1RU 40 Gigabit Ethernet and FCoE interconnect offering up to 2.43-Gbps throughput with 40 ports (Figure 5).
The switch has 24 x 40 Gigabit Ethernet and FCoE ports and 16 unified ports.
● The switch offers two high-availability ports for interconnection with a paired Cisco UCS 6300 Series Fabric Interconnect, and it supports a total of 24 x 40- Gbps fixed Ethernet and FCoE ports.
● Eighteen of the 40-Gps QSFP+ ports operate at 40 Gbps. These ports also can be converted to four 10-Gbps ports by inserting a QSFP–to–4 x SFP breakout cable or QSA module, yielding a total of up to 72 x 10-Gbps SFP+ ports.
Cisco UCS 6332-16UP Fabric Interconnect Block-Level Arc
● Sixteen Cisco UCS Unified Ports (UP) offer 10- and 40-Gbps unified fabric through SFP+ connections or 4-, 8-, or 16-Gbps Fibre Channel connections.
● Six additional 40-Gps QSFP+ ports operate at 40-Gbps speed. You can convert 24 of these ports to four 10-Gbps ports by inserting a QSFP–to–4 SFP breakout cable, yielding a total of up to 96 x 10-Gbps SFP+ ports
Two power supplies (hot swappable, 1+1) and four fans (hot swappable, 2+2) insert into the front of the chassis with front-to-back cooling. Server-facing ports are at the rear of the chassis to simplify connections to servers and fabric extenders. Serial and management ports are provided at the front of the chassis for easy access at the front of the rack.
Several Cisco UCS fabric extenders can be connected to Cisco UCS 6300 Series Fabric Interconnects.
● Cisco UCS 2304 Fabric Extender: For blade chassis, the Cisco UCS 2304 provides 160 Gbps of bandwidth to each of the two fabrics in a Cisco UCS 5108 Blade Server Chassis, for up to 320 Gbps of bandwidth. Each fabric extender provides eight 40-Gbps server links and four 40-Gbps SFP+ uplinks to the fabric interconnects (Figure 6). Server links are supported by multiple 10GBASEKR connections across the blade chassis midplane. These connections are combined into 40-Gbps links when Cisco UCS VIC 1340s are used with the optional Port Expander Card, or left as 10-Gbps links otherwise. In addition to extending the network to blade servers, the fabric extender contains logic that extends the management network from Cisco UCS Manager into each server’s Cisco integrated management console.
Cisco UCS 6332-16UP Fabric Interconnect Block-Level Architecture
● Cisco Nexus 2348UPQ 10GE Fabric Extender: For Cisco UCS rack servers and other appliances, the Cisco Nexus 2348UPQ 10GE connects to the fabric interconnects with up to 240 Gbps of bandwidth and 1- and 10-Gbps unified fabric downstream to Cisco UCS C-Series Rack Servers. Each fabric extender provides forty-eight 1- and 10-Gbps unified port interfaces (SFP+) and six 10-and 40-Gbps unified fabric interfaces (QSFP+).
In addition, previous-generation Cisco UCS 2200 Series Fabric Extenders are also supported, as described in the sections that follow.
Cisco UCS Virtual Interface Cards
The Cisco UCS VIC 1340 is a 2-port, 40 Gigabit Ethernet, FCoE-capable modular LAN on motherboard (mLOM) mezzanine adapter. Designed exclusively for Cisco UCS B200 M3, M4, and M5 Blade Servers, the Cisco UCS VIC 1340 provides 10-, 20-, or 40-Gbps unified fabric access directly to the blade server, depending on how the VIC is configured and the fabric extender to which it is connected.
The personality of the card is determined dynamically at boot time using the service profile associated with the server. The number, type (NIC or HBA), identity (MAC address and World Wide Name [WWN]), failover policy, bandwidth, and QoS policies of the PCIe interfaces are all determined using the service profile. The capability to define, create, and use interfaces on demand provides a stateless and agile server infrastructure.
The ways that interfaces are defined on the Cisco UCS VIC 1340 depends on several factors, including whether a port expander is installed and the chosen Cisco UCS fabric extender, as described in the sections that follow.
Cisco UCS VIC 1340 Connectivity to Cisco UCS 2200 Series Fabric Extender
The Cisco UCS VIC 1340 can be connected to a Cisco UCS 2200 Series Fabric Extender either by itself or with a port expander inserted in the blade server’s mezzanine connector. By itself, the Cisco UCS VIC 1340 provides four 10-Gbps interfaces, combined by port channels into two 20-Gbps interfaces, with one connected to each fabric (Figure 7).
Alone, the Cisco UCS VIC 1340 Provides a 20-Gbps Interface to Each Fabric as a Port Channel
If you add a mezzanine-based port expander to the blade server, the Cisco UCS VIC 1340 provides two 40-Gbps interfaces: one to each fabric. However, the interfaces are still four 10-Gbps interfaces combined in a port channel, and not true 40-Gbps interfaces (Figure 8).
Cisco UCS VIC 1340 Connectivity to Cisco UCS 2304 Fabric Extender
With the advent of the Cisco UCS 2304 Fabric Extender, the 40-Gbps interfaces to each fabric become true native 40-Gbps interfaces, and a hardware port channel is no longer required or used (Figure 9). True 40-Gbps interfaces can now be defined for the operating system and implemented directly on the Cisco UCS VIC 1340.
Rack Server Connectivity with Cisco UCS VIC 1385 and 1387 Adapters
Cisco UCS VIC 1385 and 1387 adapters offer, respectively, PCIe and mLOM connectivity to two 40-Gbps unified fabric uplinks. To achieve 80-Gbps connectivity to a pair of Cisco UCS 6300 Series Fabric Interconnects, the Cisco UCS VICs must be directly connected to 40-Gbps ports on the interconnects. All VIC functions are as described in the previous two sections, because all Cisco UCS VIC 1300 platform adapters use the same chip sets.
The Mezzanine-Based Port Expander Provides Two 40-Gbps Interfaces Through a Hardware Port Channel
The Cisco UCS 2304 Fabric Extender Exposes True Native 40-Gbps Unified Fabric Interfaces in Conjunction with the Cisco UCS VIC 1340 and an Optional Port Expander Card
The third generation of Cisco Unified Fabric for Cisco UCS can be deployed in a variety of use cases either as new fabric or to transition existing Cisco UCS domains to 40 Gbps.
Traditional Cisco UCS Fabric
Cisco UCS 6300 Series Fabric Interconnects deployed in a traditional Cisco UCS fabric can be used for traditional connectivity to a two-tier network using a core and aggregation network topology or a spine-and-leaf topology (Figure 10). Cisco UCS 6300 Series Fabric Interconnects deployed in a traditional Cisco UCS fabric. SANs and upstream enterprise switches connect to the fabric interconnects, and embedded Cisco UCS Manager provides management for the stateless fabric. Both Cisco UCS 2208XP and 2304 Fabric Extenders can be simultaneously connected to Cisco UCS 6300 Series Fabric Interconnects, providing a convenient transition to 40-Gbps fabrics without requiring upgrades for infrastructure that doesn’t need the additional bandwidth.
10- and 40-Gbps Fabric Connectivity with Third-Generation Cisco Unified Fabric
Cisco Unified Fabric with Cisco ACI
The Cisco UCS network fabric is ready to integrate into upstream Cisco ACI networks.
Cisco UCS 6300 Series Fabric Interconnects operate in end-host mode (the default, and most common, mode) or in switched mode. In switched mode, the fabric interconnects operate as switches and Spanning Tree Protocol is active, limiting the server links to active-failover pairs. In end-host mode, the fabric interconnects present server and virtual machine MAC addresses to the upstream network as if the interconnects themselves were the servers. Within the fabric, servers and virtual machines have active-active connectivity, providing better performance characteristics than in switched mode, which requires Spanning Tree Protocol.
With the fabric interconnects in end-host mode, and with virtual network interfaces connecting both servers and virtual machines to the fabric extenders with a single network hop, Cisco ACI leaf switches see their connections to fabric interconnects as if they were connected to a large number of servers. With Cisco UCS servers and virtual machines grouped into specific VLANs, and with each VLAN associated with a specific Cisco ACI endpoint group (EPG), any server appearing in that VLAN is automatically associated with the correct EPG. The Cisco ACI encapsulation normalization mechanism automatically wraps any upstream traffic from these servers in Virtual Extensible LAN (VXLAN) tunnels so that it is securely isolated, as if it were on its own physical network segment (Figure 11).
10- and 40-Gbps Fabric Connectivity with Third-Generation Cisco Unified Fabric
You can coordinate the definition of VLANs through Cisco UCS Manager and EPG membership through Cisco Application Policy Infrastructure Controller (APIC) in two ways to help ensure consistency and correct EPG association:
● Use Cisco UCS Director for maximum automation: Cisco UCS Director lets you define application containers that embody the multiple tiers of your application architecture. When you deploy an application, Cisco UCS Director, through the APIC, creates network containers consisting of EPGs, and it uses Cisco UCS Manager to make VLAN membership consistent.
Figure 12 shows the network architecture for Cisco UCS integration with Cisco ACI. In this example, the leaf-and-spine architecture of Cisco ACI connects to Cisco UCS 6332-16UP Fabric Interconnects. Two SANs are connected through 16-Gbps Fibre Channel interfaces on the fabric interconnects.
Third-Generation Cisco Unified Fabric for Cisco UCS in a Cisco ACI Deployment Using Both Blade Chassis and Rack Servers
Cisco UCS blade server chassis and rack servers both are connected to Cisco UCS through 40-Gbps connections to the Cisco UCS 6332-16UP Fabric Interconnects through the appropriate fabric extenders. The Cisco UCS 2304 Fabric Extender connects the blade chassis, and the optional Cisco Nexus 2348UPQ fabric extender connects the rack servers.
In this scenario, Cisco UCS Director causes Cisco UCS Manager to establish and coordinate VLANs for the Cisco ACI EPG. The APIC acts as a unifying point of automation and management for the Cisco ACI fabric, providing centralized access to all fabric information, optimizing the application lifecycle for scalability and performance, and supporting flexible application provisioning across both physical and virtual resources.
Cisco UCS was designed from the outset to facilitate stable infrastructure in the face of steady technology progression. As application demands grow, third-generation Cisco Unified Fabric allows organizations to move easily to support 40- Gbps throughput without the need for extensive upgrades or complex recabling efforts. By allowing you to retain your existing servers, interfaces, fabric extenders, and cabling, Cisco helps you protect your current investments while exploiting 40-Gbps technology to achieve greater throughput for critical applications. The longevity of the Cisco UCS 5100 Series Blade Server Chassis has now proven itself through three generations of Cisco Unified Fabric and multiple generations of Intel Xeon processors. And this steady technology progression lets you stay current and achieve your goals without any unnecessary impact on policy or data center operations.
For more information on Cisco Unified Computing System, visit https://www.cisco.com/site/us/en/products/computing/servers-unified-computing-systems/index.html.
For more information on Cisco UCS 6300 Series Fabric Interconnects visit https://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-6300-seriesfabric-interconnects/index.html.
What is the purpose of the Cisco UCS fabric interconnect? ›
The UCS fabric interconnect is a core part of Cisco's Unified Computing System, which is designed to improve scalability and reduce the total cost of ownership of data centers by integrating all components into a single platform, which acts as a single unit.What is the difference between fabric interconnect and switch? ›
The fabric interconnect always lives at the top of these UCS chassis, and is like a specialized switch. The most significant difference is that each connection is always active. These fabric interconnects typically have more bandwidth than traditional switches, too.How many fabric interconnects form a high availability cluster? ›
To use the cluster configuration, the two fabric interconnects must be directly connected together using Ethernet cables between the L1 (L1-to-L1) and L2 (L2-to-L2) high availability ports, with no other fabric interconnects in between.Which mode must be enabled on a Cisco UCS fabric interconnect to connect directly to a storage array? ›
Switch mode allows the fabric interconnect to connect directly to a storage device.What is Cisco fabric services Protocol? ›
Cisco Fabric Services (CFS) provides a common infrastructure for automatic configuration synchronization in the network. It provides the transport function and a set of common services to the features.What is a Cisco fabric? ›
Cisco Unified Fabric is the interconnection point for. applications, stored data, and users that can help you: • Build an intelligent, high-performing infrastructure. with multidimensional scalability for the data center. network.How does fabric interconnect work? ›
Each Fabric Interconnect is typically configured as a highly available clustered pair in production environments and provide active-active data traffic. By deploying in pairs, each server or blade chassis is connected to both Fabric Interconnects to protect against FI failure.Is fabric interconnect active or passive? ›
Fabric interconnects are typically deployed in active–active redundant pairs providing uniform access to storage arrays and other network resources.What is a fabric switch used for? ›
The networking switch fabric is the basic topology of how a network is laid out and connected to switch traffic on a data or circuit-switched network.What are the modes of fabric interconnect in UCS? ›
UCS Fabric Interconnect supports two types of switching modes: End Host Mode (EHM) and standard Ethernet switching mode. The default switching mode is the End Host Mode which eliminates the use of Spanning Tree Protocol (STP) on Fabric Interconnect.
How do you power off fabric interconnects in Cisco UCS? ›
UCS Fabric Interconnects are designed for continuous operation. In a production environment, there is no need to shut down or reboot Fabric Interconnects. Therefore, there is no power button on UCS Fabric Interconnects. In order to shut down UCS Fabric Interconnect, it is required to pull the power cable manually.Which is the default mode of fabric interconnect? ›
The default switching mode is the EHM, which eliminates the use of the Spanning Tree Protocol (STP) on Fabric Interconnect.What type of port on the fabric interconnects is responsible for connecting to the data center? ›
At the fabric interconnect, the server-facing Ethernet port receives the Ethernet and Fibre Channel traffic.What is the default password for Cisco Fabric Interconnect? ›
Cisco recommends changing the password once access is gained using the default “Cisco1234” password.What is the default password for UCS fabric interconnect? ›
In a web browser on the workstation, enter the IP address that you assigned to the CIMC during initial setup. On the login page, enter the default user name ( admin ) and default password ( password ), and then click Log In.What are the best practices of Cisco FabricPath? ›
As a best practice, you should explicitly set the root of all MDTs within each topology, so they provide optimal forwarding behavior. It is highly recommended that you configure a third Cisco FabricPath node with the third- highest priority to help ensure deterministic behavior.How does Cisco FabricPath work? ›
Cisco's Fabricpath is scalable and allows for many active uplinks. It does this by creating a layer-2 fabric or mesh of active links between switches. It is able to make all the links active by taking layer-3 concepts, and applying them at layer-2. FabricPath uses the IS-IS routing protocol.What is the difference between Cisco OTV and FabricPath? ›
OTV allows you to create an L2 overlay over L3. No need for ptp L2 links, Routed L3 is enough. FabricPath is basically a Mac-in-Mac encapsulation with ISIS propagating the topology. MACs still learned through dataplane (better controlled flooding and learning, but still flooding and learning).What is the difference between a network and a fabric? ›
A fabric refers collectively to the equipment and configuration that implements a network. A network fabric describes the network topology in which components pass data to each other through interconnecting switches.What is the difference between fabric connect and fabric attach? ›
The Fabric Attach extends Fabric technology functionality to network elements or hosts that are not SPB-capable. The Fabric Connect tab allows you to view topologies with the fabric-enabled sites in your network.
What are common uses for network fabric? ›
A network fabric can be used to address modern data center concerns like rapid resource scaling, infrastructure virtualization, working with containers, and connecting multiple data centers.What is fabric in Cisco switch? ›
The switch fabric is the core of the Cisco CRS 4-slot line card chassis. The switch fabric is implemented through switch fabric cards installed in the chassis. The switch fabric uses a cell-switched, buffered three-stage Benes switch fabric architecture.What is Cisco fabric in a box? ›
Fabric In a Box is used for smaller site, where the Edge is playing role of Control, Edge, and Border. Cisco Catalyst 9000 Series Switches have the capability to host fabric edge, control plane, border, and embedded wireless functionalities on a single switch.What is the switched fabric in a storage area network? ›
Fibre channel is a networking technology primarily used for storage area networking. FICON® itself is a protocol layer created by IBM® which builds upon the fibre channel transport protocol. A switched fabric is a fibre channel topology in which individual node ports are interconnected and managed by switches.What are active vs passive lines? ›
In a sentence written in the active voice, the subject of sentence performs the action. In a sentence written in the passive voice, the subject receives the action.What is the difference between IP fabric and Ethernet fabric? ›
IP fabrics provide Layer 3 service and must use overlay technologies to provide Layer 2 services over the network. Ethernet fabrics are typically simpler to install and operate. IP fabrics are typically more open and scalable than Ethernet fabrics.What are the three types of switching fabrics? ›
The three types of switching are switching via memory, switching via bus, and switching via an interconnection network.Is switching fabric hardware or software? ›
Switching fabric is a combination of hardware and software that controls traffic to and from a network node with the use of multiple switches.What is the difference between server port and uplink port in UCS? ›
Server Port: Port is configured for southbound connection to an IOM Fabric Extender (FEX) module in a blade chassis. Uplink Port: Port is configured for northbound connection to the upstream Ethernet switch. Uplink ports are always configured as trunk ports.What is uplink port in UCS? ›
Uplink Ethernet Ports - Uplink Ethernet ports handle ethernet traffic between the UCS fabric interconnect and the next layer of the network. All network-bound Ethernet traffic is pinned to one of these ports. You can configure uplink Ethernet ports on either the fixed module or an expansion module.
What is the difference between UCS B and UCS C? ›
B-series blade servers can only be managed through UCS Manager, whereas C-series rack servers can either be individually managed or can be managed by UCS Manager. Provisioning of servers can be provisioned using the UCS Manager before they are physically installed in Blade Chassis.How many ports does the Cisco UCS 6324 fabric interconnect module have in Cisco UCS Mini? ›
40 Gbps QSFP+ port plus 4 uplink SFP+ ports * 10Gbps plus 16 server internal ports * 10 Gbps plus 1 crosslink port * 10Gbps = 250 Gbps.How do I enable CDP on Cisco fabric interconnect? ›
To enable CDP the first thing you must do is to create a new Network Control policy. To do this go to the LAN tab in UCSM, expand Policies, right-click Network Control Policies to create a new policy. Name it something like “Enable-CDP” and select the option to enable CDP.How to disable RAID in Cisco UCS? ›
Disabling the Embedded RAID Controller in the BIOS (UCS C220 servers) Step 1 Boot the server and press F2 when prompted to enter the BIOS Setup utility. Step 2 Select the Advanced tab, then South Bridge . Step 3 Set Onboard SCU Storage Support to Disabled. Step 4 Press F10 to save your changes and exit the utility.What are the L1 L2 ports in UCS? ›
L1 & L2 ports are used for Fabric interconnect heart-beating and this connectivity ensures that both the FI's maintaining the identical data. 2. Laid the cabling between the chassis and Fabric interconnect. So that chassis can be discovered in the UCS manager.Which mode can the link between Fex and UCS FI operate? ›
FEX Connection Mode and Discovery
You can connect the FEX to the FI in two ways. Cisco UCS Manager FI discovers the FEX based on the FEX/Chassis discovery mode. The FEX connection modes are: Hard-Pinning mode: The server facing FEX ports are pinned to the connected uplink ports when the FEX is discovered.
The default switchport mode for newer Cisco switch Ethernet interfaces is dynamic auto. Note that if two Cisco switches are left to the common default setting of auto, a trunk will never form. switchport mode dynamic desirable: Makes the interface actively attempt to convert the link to a trunk link.What is required to cluster a pair of fabric interconnects? ›
To use the cluster configuration, the two fabric interconnects must be directly connected together using Ethernet cables between the L1 (L1-to-L1) and L2 (L2-to-L2) high availability ports, with no other fabric interconnects in between.What is the username and password for cisco management port? ›
When you log in for the first time, you need to enter HTTP username and password. By default, the username is admin and the password is cisco. You can use both HTTP and HTTPS when using the service port interface. HTTPS is enabled by default and HTTP can also be enabled.What is the password for cisco virtual machine? ›
2. If using a virtual machine, log in using the default username and password set to: cisco.
What is the default user password for unify? ›
The root account: Uses the default password: ubnt.What is the default username and password for cisco codec? ›
The default username and password is admin and no password only. If you are unable to login with this.What are the benefits of having Cisco UCS and Cisco MDS together? ›
- Operational savings from hassle-free firmware upgrades, solution automation with Cisco Intersight and operational simplification with smart zoning.
- Increased uptime with Cisco support and a single pane of glass for unified visibility, combined with deep traffic analysis and proactive troubleshooting.
While the servers compared are based on similar specs, the Cisco UCS provides a lower hardware cost by deploying simplified intra-chassis switching for blades and minimizing the number of adapters and ports required for rack servers.What is the function of a fabric border node in a Cisco SD access environment? ›
The Cisco SD-Access Border node is responsible for connecting fabric to rest of the world and hence we will focus on the different connectivity models that will be provided by the border node and discuss the various designs along with scale and platform support.What components connect the UCS fabric interconnects to the blade servers? ›
If you are using blade servers, you must have FEXes in the back of it to provide connectivity to the Cisco UCS system. The FEXes are connected to the Fabric Interconnects. There are two FEX slots in the Blade Chassis, one for A side connectivity and one for B side connectivity.What are the two components that are used to administer Cisco UCS? ›
UCS system has two different types of network adapters: UCS 82598KR-CI 10 Gigabit Ethernet Adapter and UCS M71KR-E/ UCS M71KR-Q Converged network adapter.What does FSM stand for in Cisco UCS? ›
This chapter describes the finite state machine (FSM) and how to use it to troubleshoot issues with Cisco UCS. This chapter includes the following sections: • Overview of the FSM.What is FSM in Cisco UCS? ›
An FSM is a workflow model, similar to a flow chart, that is composed of the following: • A finite number of stages (states) • Transitions between those stages. • Operations. The current stage in an FSM is determined by past stages and the operations performed to transition between the stages.What are the different modes of Cisco UCS? ›
UCS Fabric Interconnect supports two types of switching modes: End Host Mode (EHM) and standard Ethernet switching mode. The default switching mode is the End Host Mode which eliminates the use of Spanning Tree Protocol (STP) on Fabric Interconnect.
What are some of the benefits of Cisco Remote Workforce Solution Choose 3? ›
- Contact Center.
- Phones, headsets, and collaboration devices.
- Services for collaboration.
The Cisco UCS cluster is a grouping of hypervisors that can be distributed across multiple hosts.What is the recommended MTU size for a Cisco SD access fabric? ›
Ultimately, we recommend an MTU of 9100 across all network underlay devices, which includes your: Fabric edge nodes. Intermediate nodes.What is the difference between blade server and rack server in UCS? ›
Blade servers are managed by the Cisco UCS manager through the UCS Fabric, while rack servers can be managed both independently or by the Cisco UCS manager. Blade servers feature hot-swapping capabilities that allow components to be added or removed without switching off the servers.