10G SFP+ and 25G SFP28 Compatibility Analysis

In recent years, data centers are expanding at an unprecedented pace to drive the need for increasing bandwidth between the server and switches. 10GbE is not adequate bandwidth for today’s networks. There’s a tendency that 25GbE (based on SFP28) is on route to displace 10GbE (based on SFP+) from its leading role as the work horse in networking construction. As this transition takes place, questions about SFP+ and SFP28 compatibility arise for anyone who’s planning to upgrade the 10GbE to the higher and faster 25GbE. This post will provide a thorough presentation to 25GbE and clarify the compatibility issues between 10G SFP+ and 25G SFP28.

What Is 25G and Why Do We Need It?

25G Ethernet was based on the IEEE 802.3by standard and released in 2016. 25GbE specification makes use of single-lane 25 Gbps Ethernet links, providing a simpler path to future Ethernet speeds of 50 Gbps, 100 Gbps and beyond. By offering the advantages listed below, 25GbE is gained more and more momentum among service providers and data centers.

Backward Compatibility With 10GbE

The high performance 25G chips use single-lane 25G serdes technology similar in operation to 10GbE, supporting technology advancements from 10G in packaging and silicon. 25GbE allows existing switch architectures to support link speeds faster than 10G with no increase in cable/ trace interconnect.

Faster Network Performance

The 25G Ethernet based on the SFP28 form factor delivers 2.5 times more performance and bandwidth compared to 10G speeds. It also provides easy migration path to 50GE (2x25GE) & 100GE (4x25G), laying a path to higher networking speeds like 200G and 400G.

Significant Cost Benefits

25GbE delivers 2.5 times more data vs. 10GbE, thus reducing the power and cost per gigabit significantly. This power savings will in turn result in lower cooling requirements and operational expenditure for data center operators.

Available 25G Optical Modules and Cables

Every new Ethernet speed has gone through multiple pluggable form factor migrations to achieve higher density and lower power consumption goals. For instance, 10G moved successively to the X2 and XFP form factors before finally converging on the SFP+ form factor that allows for up to 48 ports per 1U. Similar form factor transitions happened for 40G (CFP to QSFP) and 100G (CFP, CFP2, CFP4 and QSFP), in achieving the highest density and lowest power. With the release of the 25GbE specification, 25 Gigabit Ethernet equipment is available on the market using the SFP28 form factors. For optical modules, FS offers cost-effective 25GBASE-SR, 25GBASE-LR, and 25G CWDM SFP28 transceivers to cut your hardware costs. For short-haul transmission, 25G SFP28 DAC (direct attach cable) and 100G QSFP28 to 4x SFP28 AOC (active optical cable) in various lengths are also available for all needs and specifications.

25G SFP28 DAC and AOC

10G SFP+ and 25G SFP28 Compatibility

With 10G and 25G Ethernet equipment coexisting on the market today, we may frequently encounter the compatibility issues related to SFP+ and SFP28 form factors. Anyway, the newer 25GbE technologies are backward compatible with 10GbE, allowing customers to build and cross-connect a heterogeneous-speed Ethernet network. Here we list the frequently asked questions coming through average customers.

1.What is the difference between SFP28 and SFP+?

The pinouts of SFP28 and SFP+ connectors are mating compatible. However, SFP+ is designed to operate at speed up to 10 Gb/s whereas SFP28 can handle 25Gbps, 10Gbps and even 1Gbps. SFP28 has increased bandwidth, superior impedance control and less crosstalk than the SFP+ solution. Besides, the SFP28 copper cable has significantly greater bandwidth and lower loss compared to the SFP+ version.

2.Can the SFP28 be used in SFP+ slot, and what speed will I get?

Theoretically, plugging an SFP28 transceiver or cable into the 10G interface is feasible for certain devices to get 10Gb/s data rate, but this solution is not recommended, because it would be limited by the NIC and switch port that you have. Only when your SFP28 module is 100% compatible with your server or switch can you ensure that the links can go seamlessly and efficiently.

3.Can the SFP+ be used in SFP28 slot, and what speed will I get?

Theoretically, plugging an SFP+ transceiver or cable into the 25G SFP28 slot is feasible. But you also need to make sure ensure that your existing modules will be compatible with your switch gear. That is to say, although a switch that accepts the SFP28 form factor can physically accept a SFP+ connector in the same port, it doesn’t mean that your SFP+ modules will work on your equipment.

Note: When shopping for new 25G leaf switches, if you have SFP+ modules you want to use, look for switches that accept the SFP28 form factor, which is physically capable of taking existing SFP+ modules. The same holds true for QSFP+ modules and QSFP28 ports. Carefully read the product specs to ensure that your existing modules will work with your new equipment.

Conclusion

As the majority of 25G switches and network interface cards offer backward compatibility to 10G, there is lots of flexibility to manage a gradual migration to higher speed servers and mix and match port speeds. Theoretically, all SFP28 based 25G ports on switches and 25G NICs can be used at 10G speed via port self-negotiation, but the premise should be that your existing modules are compatible with the NIC and switch port that you have. With a minimal premium for 25G based systems compared to 10G systems, it becomes a wise choice to deploy 25G capable systems to realize the performance advantages for migrations & future proofing initiatives.

Related Article:
10G – 25G – 100G Network Upgrade: An Inevitable Roadmap for Future Data Centers
FS 25G Portfolio for Data Center 25G/100G Leaf-Spine Network

How to Upgrade Cumulus Linux on N-Series Switches?

In the previous posts, we have introduced lots of details about Cumulus Linux, a network operating system. Also, we have talked about how to install Cumulus Linux on FS N-series switches like the 100GbE switch. In this post, we will solve another Cumulus Linux related question concerned by most of our users: how to upgrade Cumulus Linux on an N-series network switch?

Before Upgrading Cumulus Linux

Before any update, you should consider the migration and backup of some important data or files in case of any accident or emergency. In that way, being clear about the location of your configuration data is crucial before updating Cumulus Linux. Just like other Linux operating system distributions, the /etc directory is the primary location for all configuration data in Cumulus Linux. In the following figures, we list some files and as well as their directories that you are likely to do the backup.

Cumulus Linux Network Configuration Files

Figure 1: Cumulus Linux Network Configuration Files

Additional Commonly Used Files

Figure 2: Additional Commonly Used Files

Note: If you are using the root user account, consider including /root/. If you have custom user accounts, consider including /home//.

Upgrading Processes

Basically, there are two methods to upgrade Cumulus Linux: disk image install and package upgrade. According to different environment and preferences, the upgrade can be achieved in different ways. In this part, we will guide you on how to use these two ways one by one. By the way, if you are using MLAG to dual connect two switches in your environment, there are additional steps to follow. You can refer to the guide:Upgrade Switches in an MLAG Pair.

Solution 1: Installing A Disk Image

When you are performing a rolling upgrade in a production environment and using up-to-date and comprehensive automation scripts, the disk image installation will be recommended. It enables you to choose the exact release to which you want to upgrade. Moreover, it is the only method available to upgrade your switch like a 10GbE switch to a new release train (for example, from 2.5.6 to 3.7.0) or from a release earlier than 3.6.2.
To apply this upgrade, you need to use ONIE (open network install environment), which allows the installation of network operating systems (NOS) on a bare metal switch. FS N-series switches all include an ONIE installer. There are mainly six steps you can follow with:

1. Back up the configurations off the switch.

2. Download the Cumulus Linux image you want to install.

3. Install the disk image with the onie-install -a -i command, which boots the switch into ONIE. There are various ways to install the disk image, such as using FTP, a local file, or a USB drive. In figure 3, we demonstrate how to install image from a web server as your reference. After disk image installed, the switch can be rebooted.

Disk Image Install

Figure 3: Disk Image Install

4. Restore the configuration files to the new release—ideally with automation.

5. Verify correct operation with the old configurations on the new release.

6. Re-install third-party applications and associated configurations.

Solution 2: Upgrading Packages

Package upgrade is recommended if you are upgrading from Cumulus Linux 3.6.2 or later, or if you use third-party applications (package upgrade does not replace or remove third-party applications, unlike disk image install). When upgrading, configuration data stays in place. If the new release updates a configuration file that you changed previously, you are prompted for the version you want to use or if you want to evaluate the differences. Altogether, there are six steps as follows:

1. Back up the configurations from the switch.

2. Apply the following command to fetch the latest update metadata from the repository.

Command for the Latest Update Metadata

3. Review potential upgrade issues (in some cases, upgrading new packages might also upgrade additional existing packages due to dependencies). Run the following command to see the additional packages that will be installed or upgraded.

Command for Additional Packages

4. Upgrade all the packages to the latest distribution.

Command to the Latest Distribution

5. Reboot the switch if the upgrade messages indicate that a system restart is required.

Command to Reboot the Switch

6. Verify correct operation with the old configurations on the new version.

Questions & Answers

Besides the procedures to upgrade Cumulus Linux, we also list some questions concerned most by our FS customers.

1. How to Deal With Upgrade Failures?

Even the most well planned and tested upgrades can result in unforeseen problems. If you fail to upgrade Cumulus Linux, sometimes the best solution is to roll back to the previous state. You can refer to the following three techniques:

  • Back out individual packages: If you identify the problematic package, you can downgrade the affected package directly. In rare cases, you might need to restore the configuration files from backup or edit to back out any changes made automatically by the upgrade package.
  • Flatten and rebuild: use orchestration tools to re-install the previous OS release from scratch and then rebuild the configuration automatically.
  • Backup and restore: restore to a previous state, using a backup captured before the upgrade.
2. Will Future Software Updates for Cumulus Linux Be Free?

For FS N-series switches, the default validity of Cumulus Linux is one year. If Cumulus Linux OS is upgraded after 1 year, the customer can’t upgrade it for free and needs to renew the software support. Even if the software support service has been expired, you are allowed to use your current software as usual. By the way, we also provide a three-year software support fee option and a five-year software support fee option. The longer for OS support you choose, the more economical the switch will be.

3. Who Will Cover Technical Support?

FS will provide five-year tech support for the hardware. As for the problems about Cumulus Linux, FS and Cumulus will solve it together. Moreover, as one of the Linux distributions, the source code of Cumulus Linux is also freely available to everyone to use, modify or share such as GitHub. Just like the above question about the software update, the default validity of technical support for Cumulus Linux is one year. One year later, you can renew the support term.

Summary

In this post, we mainly focus on how to upgrade the Cumulus Linux on FS N-series switches. In addition, we list three after-sales related questions frequently asked by our FS customers.

Why 25G Ethernet Switches Are Still Necessary?

It has been about five years since the arrival of 25G Ethernet in 2014. For those years, the 25G Ethernet market has been filled with ups and downs. Facing with the broad adoption for 100G Ethernet and the upcoming new connection speeds of 200G/400G, the use of 25G devices, like the 25G switches, has been in doubt.

A Review of 25G Ethernet

25G Ethernet is one of the standards for Ethernet connectivity in a data center environment, developed by IEEE 802.3 task force P802.3by. Before 25G Ethernet was proposed, the next speed upgrade for data centers was expected to be 40G Ethernet (using four lanes of 10G) with a path to 100G defined as using 10 lanes of 10G. Now, with the 25G Ethernet standard, it supports to have four 25 Gbps lanes to achieve the speed of 100G Ethernet. In that way, it is said that 25G has paved the road for 100G.

25G Ethernet VS 40G Ethernet

Figure 1: 25G Ethernet VS 40G Ethernet

25G Ethernet Switch

With the 25G standard carried out, in 2016, its matching equipment was also available on the market, such as 25G SFP28 transceiver, DAC cable, 25G adapter, and 25GbE switch. Among those devices, the 25G Ethernet switch is the most representative one. Nowadays, the 25G switch market is mainly led by some branded vendors such as 25G Dell, Cisco, Juniper, Arista, and Mellanox switches. Usually, the 25G 48-port switch is the most popular type. Most 25G switches today offer two types of 25GbE interface form factors: QSFP28 that can support 4x25Gbps and SFP28 that can support 1x25Gbps. No matter for TOR (Top of Rack) switch or as the switch to deploy spine-leaf architecture, 25GbE switches will be the good choices.

Figure 2: FS 25G Ethernet Switch N8500-48B6C

Figure 2: FS 25G Ethernet Switch N8500-48B6C

Why Still Need 25G Ethernet Switches?

Switch Compatibility

The majority of 25G Ethernet switches in the market support backward compatibility. Because most of their matched optical transceivers are SFP28. And SFP28 is regarded as the enhanced version of SFP+, which is designed for 25G signal transmission. SFP28 utilizes the same form factor as SFP+, but the electrical interface is upgraded to handle 25Gbps per lane. As SFP28 adopts the same form factor as SFP+, SFP28 can connect with SFP+ ports and SFP+ transceiver can also connect with SFP28 ports. SFP28 is compatible with existing data center fiber cabling. Thus, you can greatly reduce the cost of re-architecture data centers and gain great flexibility in creating higher bandwidth during migration. To some extent, it can be both CapEx and OpEx savings.

Port and System Density

The 25G technology is similar to 10G, but the performance is increased by 2.5 times, thus reducing the power and cost per gigabit significantly. 25G Ethernet provides higher port and system density. For example, four 25 Gb/s data streams can be used to produce a 100G path over copper or fiber cable within a compact form factor. This approach also saves on energy consumption and requires fewer top-of-rack (ToR) switches and cables, which cuts much operational expenditure for data center operators in the end.

Price and Performance
Price Comparison by Connection Speed

Figure 3: Price Comparison by Connection Speed

As we can see in the Crehan forecast in figure 3, 25G delivers on both fronts with better price and performance. While 25G Ethernet is slightly more expensive than the 10G pricing, when valued with price and performance it is much cheaper on a per Gbit/s of bandwidth.

In fact, the 25GbE pricing is very competitive, with only a 30%-40% premium over 10GbE and this premium is expected to come down over time. To achieve these competitive pricing levels requires devices that are optimized to support 25GbE. In that case, deploy 25G devices, such as 25G Ethernet switches are necessary.

Summary

With the trend for higher Ethernet bandwidth, the demand for 10G Ethernet has been in decline. Before the 200G/400G Ethernet becomes mature, 25G shares lots of incomparable strengths to be considered as the proper choice to prepare for the upcoming migrations. In the context of that, 25G devices, such as 25 Ethernet switches are playing indispensable roles.

What Are the Commonalities of Switches Supporting Cumulus Linux

As the first full-featured Linux based operating system (OS), Cumulus Linux has injected great possibilities and new vitality in networking field in these two years. Due to its great effort in open networking, Cumulus Linux has been one of the three leading OSs in the market. The another two are IP Infusion OcNOS and Pica8 PICOS. Recently, the collaboration between FS and Cumulus Networks has been made. The N-series open switches from FS will be pre-installed the latest Cumulus Linux OS to customers. At the moment of their joint effort in achieving open networking, we are going to make an analysis of the similar features of the open manageable switch supported by Cumulus Linux.

FS Collaborates with Cumulus Networks

Figure 1: FS Collaborates with Cumulus Networks

An Overview of Cumulus Linux

Cumulus Linux is a flexible open network operating system, which can be installed on various open switches, including the layer 2 switch and layer 3 switches. The code used to build Linux is free and available for users to view or edit. Therefore, it looks like the world’s largest data center that allows users to automate, customize and scale using web-scale principles. After the installation of the Cumulus Linux OS, the open switch can act as a Linux server.

Cumulus Linux

Figure 2: Cumulus Linux

Similarities of Open Switches Supporting Cumulus Linux

Featured with supporting a broad partner ecosystem, the Cumulus Linux gives customers more options and flexibility in data center networking regarding switch type, CPU, chip type, and supported transceivers.

Switch Type

Generally, open switches that support Cumulus Linux are bare mental switches coming with open network install environment (ONIE). In that case, no matter you have a brite box switch like Cisco switch, or a white box switch like FS switch, Cumulus Linux can be accessible to them. Nowadays, in the market of open switches, the 32-port and 48-port switches with 40G/100G transmission speed are commonly applied by enterprise users. Considering their high-density and greater agility needs for networking, the open switches are mostly layer 3 switches so as to achieve spine-leaf or overlay architectures.

CPU

The open switch CPU that supports Cumulus Linux OS usually comes in three types: ARMv7, PowerPC, and x86_64. Among these three types, x86_64 is the most popular one, adopted by most vendors, such as Dell, HPE, Mellanox, and FS.

Chip Type
Chips of Open Switches

Figure 3: Chips of Open Switches

Currently, Broadcom chip and Mellanox chip are the major roles of switch chip. The Mellanox type is usually used by Mellanox itself or Penguin. Therefore, the Broadcom type dominates the largest switch chip market share, installed by the most brand vendors or the third party suppliers.

Supported Transceivers

Since most open switches support high-speed transmissions, the matching transceivers are QSFP28, QSFP+, and SFP28. Only some 10G and 1G open switches will need to use SFP+ and SFP transceivers. By the way, viewing the trend, you will find 25G Ethernet has been deployed by many enterprise users in recent years for high bandwidth need. Accordingly, the 25G open switch has been a more economical and efficient choice than 1G or 10G switches. Also, the 25G switch will be the best solution to pave the road for the upcoming 100G/400G Ethernet in the future.

Summary

Just like the agility and simplicity the Cumulus Linux has advocated, it brings a truly economical and open network environment for users. With so many choices for open switch type, CPU, chip, and supported transceivers, it liberates the choices for open switches, which begets an open networking market in the end.

DWDM Network over Long Distance Transmission

With the ever-increasing need for higher bandwidth, DWDM technology has been one of the most favorable optical transport network (OTN) applications. In this post, we will reveal FS.COM DWDM-based network solutions over various transmitting distances as well as some suggestions for the DWDM networks deployment.

DWDM Networks Basics

As usual, let’s review some basics of DWDM networks. In this part, we will figure out two questions: What is DWDM? What are the components of DWDM networks?

DWDM Technology
DWDM Networks

Figure 1: DWDM Networks

DWDM (Dense Wavelength Division Multiplexing) is an associate extension of optical networking. It can put data signals from different sources together on a single optical fiber pair, with each signal simultaneously carried on its own separate light wavelength. With DWDM, up to 160 wavelengths with a spacing of 0.8/0.4 nm (100 GHz/50 GHz grid) separate wavelengths or channels of data can be transmitted over a single optical fiber.

DWDM Networks Components

Conventionally, for DWDM networks, there are four devices showed as below that are commonly used by IT workers:

  • Optical transmitters/receivers
  • DWDM mux/demux filters
  • Optical add/drop multiplexers (OADMs)
  • Optical amplifiers transponders (wavelength converters)

DWDM Networks Over Long Distance Transmission Solutions

Scenario 1: 40 km Transmission
40km DWDM Network

Figure 2: 40km DWDM Network

For this case, the 80km DWDM SFP+ modules and 40ch DWDM Mux/Demuxs are recommended to use. Since the 80km DWDM SFP+ modules are able to support 10G transmission over 40 km, no additional device is needed under this scenario.

Scenario 2: 80 km Transmission
80km DWDM Network

Figure 3: 80km DWDM Network

Deploying this 80 km DWDM network, we will still use 80km DWDM SFP+ modules and 40ch DWDM Mux/Demuxs. The light source of 80km DWDM SFP+ modules might not be able to support such long transmission distance, as there might be a light loss during transmission. In this case, pre-amplifier (PA) is usually deployed before the location A and location B to improve the receiver sensitivity and extend signal transmission DWDM distance. Meanwhile, the dispersion compensation module (DCM) can be added to this link to handle the accumulated chromatic dispersion without dropping and regenerating the wavelengths on the link. The above diagram shows the deploying method of this 80km DWDM network.

Scenario 3: 100 km Transmission
100km DWDM Network

Figure 4: 100km DWDM Network

Under this scenario, the devices used in scenario 2 still need to remain. Since the transmission distance has been increased, the light power will be decreased accordingly. Besides that, you will also need to use booster EDFA (BA) to amplify the optical signal transmission of the 80km DWDM SFP+ modules.

By the way, if you want to extend DWDM transmission distance, you can read this post for solutions: Extend DWDM Network Transmission Distance With Multi-Service Transport Platform.

Factors to Consider in Deploying DWDM Networks

1. Being compatible with existing fiber plant. Some types of older fiber are not suitable for DWDM use. Currently, standard singlemode fiber (G. 652) accounts for the majority of installed fiber, supporting DWDM in the metropolitan area.

2. Having an overall migration and provisioning strategy. Because DWDM is capable of supporting massive growth in bandwidth demands over time without forklift upgrades, it represents a long-term investment. Your deployment should allow for flexible additions of nodes, such as OADMs, to meet the changing demands of customers and usages.

3. Network management tools. A comprehensive network management tool will be needed for provisioning, performance, monitoring, fault identification and isolation, and remedial action. Such a tool should be standards-based (SNMP, for example) and be able to interoperate with the existing operating system. For example, the FMT DWDM solutions from FS.COM are able to support kinds of network management, including NMU line-card, monitor online, simple management tool, and SNMP.

4. Interoperability issues. Because DWDM uses specific wavelengths for transmission, the DWDM wavelengths used must be the same on either end of any given connection. Moreover, other interoperability issues also need to be considered, including power levels, inter- and intra-channel isolation, PMD (polarization mode dispersion) tolerances, and fiber types. All these contribute to the challenges of transmission between different systems at Layer 1.

5. Strategy for protection and restoration. There might have hard failures (equipment failures, such as laser or photodetector, and fiber breaks) and soft failures such as signal degradation (for example, unacceptable BER). Therefore, you need to have a protection strategy while deploying a DWDM network.

6. Optical power budget or link loss budget. Since there might be an optical signal loss over the long distance transmission, it’s critical to have a link loss budget in advance.

Summary

Bringing great scalability and flexibility to fiber networks, the DWDM networks solutions obviously enjoys plenty of strengths, which is also proved to be future-proof. In this post, we make a revelation of the DWDM-based network over long distance transmission. Also, some tips for deploying a DWDM network has also been shared for your reference.