FREE SHIPPING on Orders Over US$79
United States

FS PicOS® Switch N8560-32C Proven in Keysight RoCEv2 Testing

GeorgeUpdated at Jan 4th 20251 min read

To meet the high throughput demands of high-performance networks, switches in RoCEv2 deployments must deliver superior bandwidth and low latency for RDMA performance. While many vendors promise these capabilities, few can truly deliver. FS, however, consistently exceeds expectations. Our switches are highly praised by customers and industry experts alike.
"At Keysight, we support our customers by providing high performance, reliable test, and validation systems. The successful validation of FS data center switches using Keysight’s RoCEv2 test solution underscores our dedication to supporting the most demanding data center environments."
— Daisy Sun, Senior Product Manager, Network Testing & Security Solutions at Keysight Technologies
RoCEv2 Test Overview
Keysight's RoCEv2 test solution validates the promise of its performance in FS PicOS® data center switches. The market-leading RoCEv2 benchmarking test solution was introduced in February 2024 and includes a high-density cost-effective test hardware platform (AresONE-S 400GE), the IxNetwork test application, and the RoCEv2 license.
Testing was conducted by running a typical one-hop many-to-one RoCE performance test on FS data center switches with specific switch configurations. RoCEv2 endpoints were emulated by using high-density AresONE traffic load appliances. The test report showed that FS data center switches can maintain high RDMA bandwidth and low latency even on congested RoCE networks, aligning with high industry standards.
Test topology and configuration instructions
The test was conducted using the PicOS® switch N8560-32C, a 32-port Ethernet L3 switch with Broadcom chips that provides line-rate switching across all ports. The switch software version used was 4.4.5/eb33bb7768, and the Keysight Ixia network test application software version was 10.10.2403.5.
Test Environment
The test environment is set up as a standard ridge architecture switch, where the switch configuration is fixed and the dynamic server environment requires no network changes at the same time, ensuring a benchmark for test solution validation.
Description of the topology environment and networking
Currently, due to device resources, only the one-hop 2-to-1 and 3-to-1 scenarios were tested in the RoCEv2 test.
Queue 3 Enables the
PFC
and
ECN
to run RDMA Data traffic. Queue 7 runs the CNP traffic of the ECN. QoS scheduling trusts the DSCP mode, adopts the SP congestion management mode, and sets the MTU of ports on the switch to 9216.
The end-side tool uses Keysight's Ixia Network RoCEv2 solution to perform RDMA write performance test.
Auto-Cost Reference-Bandwidth
Switch QoS Configuration
QoS scheduling trusts DSCP mode and adopts SP congestion management mode;
set class-of-service forwarding-class class1 local-priority 3
set class-of-service forwarding-class class3 local-priority 7
set class-of-service scheduler s1 mode "SP"
set class-of-service scheduler s1 guaranteed-rate 30000000
set class-of-service scheduler s3 mode "SP"
set class-of-service scheduler s3 guaranteed-rate 1000000
set class-of-service classifier c1 trust-mode "dscp"
set class-of-service classifier c1 forwarding-class class1 code-point 26
set class-of-service classifier c1 forwarding-class class3 code-point 56
set class-of-service scheduler-profile ss1 forwarding-class class1 scheduler "s1"
set class-of-service scheduler-profile ss1 forwarding-class class3 scheduler "s3"
set class-of-service forwarding-class class1 local-priority 3
set class-of-service forwarding-class class3 local-priority 7
set class-of-service scheduler s1 mode "SP"
set class-of-service scheduler s1 guaranteed-rate 30000000
set class-of-service scheduler s3 mode "SP"
set class-of-service scheduler s3 guaranteed-rate 1000000
set class-of-service classifier c1 trust-mode "dscp"
set class-of-service classifier c1 forwarding-class class1 code-point 26
set class-of-service classifier c1 forwarding-class class3 code-point 56
set class-of-service scheduler-profile ss1 forwarding-class class1 scheduler "s1"
set class-of-service scheduler-profile ss1 forwarding-class class3 scheduler "s3"
set class-of-service interface xe-1/1/3 classifier "c1"
set class-of-service interface xe-1/1/3 scheduler-profile "ss1"
Switch PFC Priority Mapping
Use queue 3 to carry RoCEv2 traffic. In the switch, each panel port needs to enable PFC flow control for priority 3. The command line is as follows:
set class-of-service pfc-profile p1 code-point 3
set class-of-service pfc-profile p1 code-point 0 drop true
set class-of-service pfc-profile p1 code-point 2 drop true
set class-of-service pfc-profile p1 code-point 4 drop true
set class-of-service pfc-profile p1 code-point 5 drop true
set class-of-service pfc-profile p1 code-point 6 drop true
set class-of-service pfc-profile p1 code-point 7 drop true
set class-of-service pfc-profile p1 code-point 1 drop true
set class-of-service interface xe-1/1/3 pfc-profile "p1"
Switch PFC Deadlock Configuration
This section describes how to set the watchdog attribute of PFC watchdog with priority 3 to 5 time slices and restore time to 15 time slices. Set the time precision to low precision (100ms per time slice).
set class-of-service pfc-watchdog code-point 3 restore-interval 15
set class-of-service pfc-watchdog code-point 3 detect-interval 5
set class-of-service interface xe-1/1/3 pfc-watchdog code-point 3 enable true
set class-of-service interface xe-1/1/3 pfc-watchdog code-point 7 enable true
Switch Static ECN and Buffer Configuration
ECN: Explicit Congestion Notification Displays a congestion notification. When the egress port is congested, the ECN is used to indicate that the switch is congested. When the switch's egress buffer reaches a specified threshold, the switch changes the ECN bit in the packet header to label the data as ECN. After the ECN tagged data reaches the receiver, the receiver generates CNP and sends it to the sender. After receiving CNP, the receiver takes measures to reduce the sending speed. Enable ECN on queue 3 on the port
set interface ethernet-switching-options buffer egress-queue 3 shared-ratio 100
set interface ethernet-switching-options buffer egress-queue 7 shared-ratio 10
set interface gigabit-ethernet xe-1/1/3 wred queue 3 min_thresh 80000
set interface gigabit-ethernet xe-1/1/3 wred queue 3 max_thresh 8000000
set interface gigabit-ethernet xe-1/1/3 wred queue 3 drop_probability 50
set interface gigabit-ethernet xe-1/1/3 wred queue 3 ecn_thresh 1
set interface gigabit-ethernet xe-1/1/3 wred queue 3 enable true
End-to-end tool Ixia Network configuration
1) Set MTU to 4096
2) DSCP value is 26
3) QP value is 1
4)RDMA write buffer size is 1M
Test result analysis
One-Hop 2-To-1 Test Results
Test results: There is no packet loss on the switch side, no packet loss on the end side, no PFC, the overall RDMA bandwidth is 97.96G, and the average RDMA delay is 4246ns. These results reflect FS's ability to deliver stable and high-performance networking even in demanding scenarios, ensuring minimal disruption to critical applications.
RDMA bandwidth test results:
RDMA delay test results:
One-Hop 3-To-1 Test
Test results: There is no packet loss on the switch side, no packet loss on the end side, no PFC, the overall RDMA bandwidth is 98.54G, and the average RDMA delay is 5813ns. This further reinforces the reliability of FS switches in delivering optimal performance across varied configurations.
RDMA bandwidth test results:
RDMA delay test results:
The above test results effectively validate the robust performance of the PicOS® N8560-32C switch, powered by the Broadcom BCM56870 Trident 3 Chip, which offers advantages in high-speed data transmission, low latency, and high throughput.
In addition to high-speed transmission, network reliability and scalability are equally important when facing the complexities of future network environments. The N8560-32C features redundant, hot-swappable power supplies and intelligent fans to prevent disruptions caused by power or fan failures, ensuring uninterrupted operation and improving device reliability and availability. The PicOS® software supported by the switch provides stable network performance to enhance system reliability. Its modular design and dual-partition mechanism further improve system flexibility and stability. Moreover, the software supports a rich set of protocols, including EVPN-VXLAN, MLAG, and EVPN Multihoming, helping enterprises build flexible and scalable network architectures.
FS Switch Quality Commitment
In today's interconnected world, reliable network performance cannot be compromised. It is the backbone of business operations, and any weak link can lead to a significant drop in productivity and revenue. The Keysight RoCEv2 testing tool effectively validates the exceptional high-bandwidth and low-latency performance of the PicOS® switch in data center environments. Additionally, the switch leverages the unified PicOS® and AmpCon™-DC platforms, enabling automated configuration and management to enhance the efficiency and stability of data center operations.
FS is always looking for solid and reliable measures to enhance our ability to deliver high-performance networking products and solutions. In the future, FS will do everything it can to continue delivering best-in-class switches to enhance your network and drive your digital transformation forward!