With the migration from Gigabit Ethernet to 10 Gigabit Ethernet, cabling and network switching architectures have been reevaluated to guarantee a cost-effective and smooth transition. 10Gb ToR (Top of Rack) or leaf switch has evolved with significant performance gains and cost-per-port reduction. This post will introduce the benefits of ToR architecture and explains how to deploy 10G ToR/leaf switch for different size networks.
Why Use Top-of-Rack Architecture
ToR or leaf-spine is a network architecture design where there are only two tiers of switches between the servers and the core network. In ToR network design, a feature-rich 10GbE switch handles Layer2 and Layer3 processing, data bridging and Fibre Channel over Ethernet (FCoE) for an entire rack of servers. This approach contributes to an agile infrastructure because the ToR/leaf switches can support multiple I/O interfaces, including GbE, 10GbE and 40GbE. The 10G ToR/leaf switches utilized in the ToR architecture usually come with the advantage of low power consumption, ease of scale and simplified cabling complexity. When acting as a ToR/leaf switch, each 10G Ethernet switch can be placed just one hop away from another, no need to jump up and down in the tree design, enabling improved latency and bottlenecks. With a ToR design, you can eliminate cabling nightmares, minimize bottlenecks while building a network foundation for mission-critical applications that also provides a clear path for future growth.
Campus Network Applications
For campus networks applications, the 10GE switches work as aggregation or core switches in the ToR network architecture. Here we take FS S5850-48S6Q 10G ToR/leaf switch as an example to illustrate how to build a ToR network in campus networks. In the following application diagram, two FS S5850-48S6Q 10GE switches are utilized as aggregation switches as the bridge to build connections between 40G switches in the core network and gigabit switches in the access layer.
SMB (Small and Medium-Sized Business) Applications
For small and medium-sized businesses, ToR network architectures are becoming more preferable by IT managers than ever before. Because ToR architectures enable them to implement a single cabling model that can support Gigabit Ethernet and 10 Gigabit Ethernet and unified network fabric today, while supporting future 40 and 100 Gigabit Ethernet standards as they come to market. Using ToR architecture for fiber cable management, business IT managers have the flexibility to deploy preconfigured racks with different connectivity requirements in any rack position. For example, a rack of servers running multiple Gigabit Ethernet connections can be placed next to a rack of servers with 10 Gigabit Ethernet and FCoE connections to each server.
Data Center Applications
In hyper-scale data centers, there might be hundreds or thousands of servers that are connected to a network. In this case, ToR/leaf switches work in conjunction with spine switches in data centers to aggregate traffic from server nodes and then connect to the core of the network. Now given that we need to build a data center fabric with a primary goal of having at least 480 10G servers in the fabric. In this case, we can use FS S8050-20Q4C as spine switch and S5850-32S2Q as ToR/leaf switch. As shown in the figure below, the connections between spine switches (FS S8050-20Q4C) and ToR/leaf switches (FS S5850-32S2Q) are 40G, while connections between the leaf switches and servers are 10G. The port numbers on each spine switch determines the number of leaf switches we can use. But the maximum amount of 10G servers we can connect to ToR/leaf switch here is 24 because the ratio of reasonable bandwidth between leaf and spine switch cannot exceed 3:1. Thus the total amount of bandwidths we can get here is 480x10G.
Top-of-Rack Cabling Recommendations
ToR network architectures utilize available cabling media options with flexibility at the rack level, using various server patch cable types, while taking advantage of fiber uplinks from the rack for horizontal cabling. Investment in the cabling media for 10, 40, and 100 Gigabit Ethernet connectivity involves striking a balance among bandwidth, flexibility, and scalability. Although both fiber and copper can support 10G, 40G and 100G transmission, fiber is the recommended horizontal cabling media as it offers an optimal solution for high speed 40G and 100G transmission over relatively long distances. Note that 40G and 100G transmission calls for multiple fiber strands (OM3, OM4, and SMF fiber).
The choice of ToR networking architecture can substantially affect throughput, sustainability, optimum density and energy management. As the key element of building ToR networks, 10G ToR/leaf switch can help you scale up networking architecture while delivering low-latency and high-bandwidth links. FS S5850/N5850 series switches are high performance 10GbE ToR/leaf switches which can work with Broadcom, Cisco, Juniper, Arista switches, as well as other major brands. For more information about 10GbE ToR/leaf switches, please kindly visit www.fs.com.
Related Article: 10G ToR/Leaf Ethernet Switch: What Is the Right Choice?