N5860 and N8560 and NC8200 Series Switches EVPN-VXLAN Configuration

2026-04-10 - N5860 and N8560 and NC8200 Series Switches EVPN-VXLAN Configuration 1. Overview Virtual Extensible Local Area Network (VXLAN) is a virtual Ethernet based on the physical IP (overlay) network. It is a technology that encapsulates layer 2 (L2) Ethernet frames within layer 3 User Datagram Protocol (UDP) packets. VXLAN has a 24-bit VXLAN network identifier (VNI). It allows users to create up to 16,000,000 isolated virtual networks to meet the requirements of multi-tenant environments and scale expansion, far surpassing the widely used Virtual Local Area Network (VLAN) technology that is limited to 4,000 isolated networks. VXLAN uses the IP multicast method to encapsulate multicast, broadcast, and unknown unicast packets, effectively controlling the broadcast domain in multi-tenant environments. With the transformation of data centers, more and more virtual machines are deployed. In addition, as virtual machines must be migrated in L2 environments, scales of L2 networks increase. VXLAN can extend L2 networks over layer 3 (L3) networks, so that virtual machines can be moved to L3 networks interconnected to L2 networks without changing the IP addresses and MAC addresses, thereby ensuring service continuity. Protocols and Standards RFC7348: Virtual eXtensible Local Area Network (VXLAN)-- A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks 2. Applications Applications Description EVPN-based Multi-tenant Centralized Deployment ApplicabletothecentralizeddeploymentscenariowithEthernetvirtualprivatenetwork(EVPN)enabled. EVPN-based Multi-tenant Distributed Deployment Applicable to the distributed deployment scenario with EVPN enabled. EVPN-based Single-tenant VXLAN Routing Deployment Applicable to the VXLAN routing deployment scenario with a single tenant. EVPN-based Multi-tenant VXLAN Route Deployment Applicable to the VXLAN route deployment scenario with multiple tenants SDN Controller–based Centralized All-active Anycast Gateway Deployment Applicable to the scenario for deploying all-active anycast gateways basedonthesoftware-defined networking (SDN) controller in a centralized manner datacenters Deployment of an EVPN Distributed Network to Be Compatible with Non-EVPN VTEP Devices Applicable to the deployment scenario in which dynamic and static tunnels coexist. Deployment of L2 Subinterfaces to Access a VXLAN ApplicabletothedeploymentscenarioinwhichhostsaccessVXLANsthroughL2subinterfaces. VNI Mapping–based Data Center Interconnection Deployment ApplicabletothescenarioinwhichVXLANsacrossdifferentdatacentersare interconnected using the VNI mapping technology. 2.1 EVPN-based Multi-tenant Centralized Deployment Scenario VPN routing and forwarding (VRF) networks are usually allocated to different tenants to support the multi-tenant application in a data center. Multiple VXLANs can be assigned to each tenant. VXLANs of the same tenant can be mutually accessed through the L3 router, while VXLANs of different tenants cannot be mutually accessed, as shown in Figure 2-1. Tenant A rents VRF-10, which includes VXLAN 10 and VXLAN 20. Servers HOST-1 and HOST-2 belong to VXLAN 10 and Servers HOST-3 and HOST-4 belong to VXLAN 20. Tenant B rents VRF-20, which includes VXLAN 100. Servers HOST-5 and HOST-6 belong to VXLAN 100. The networks of Tenant A and Tenant B are isolated from each other. The entire network is formed by a Border Gateway Protocol (BGP) network and includes CORE and TOR switches. The BGP neighbor relationship is formed between every two devices and the BGP-EVPN protocol family is supported. All VXLAN gateways on the network are deployed in the core switches in a centralized manner. Figure 2- 1 page_5_img_1_0f2d2cb7.jpeg Packets between HOST-1 and HOST-2 are forwarded through TOR-1 at L2 within the VXLAN. Packets between HOST-3 and HOST-4 are forwarded through TOR-1 > CORE > TOR-2 at L2 within the VXLAN. Packets between HOST-5 and HOST-6 are forwarded through TOR-2 at L2 within the VXLAN. Packets between VXLAN 10 and VXLAN 20 are forwarded through TOR-1 > CORE > TOR-2 at L3 across the VXLANs. VRF-10 and VRF-20 cannot communicate with each other. Remarks: CORE indicates a core switch that supports the VXLAN function. When centralized all-active anycast gateways are deployed, multiple core gateways exist and the VXLAN gateways deployed on the core gateways are the same. TOR1 and TOR2 are access switches that support the VXLAN function. HOST-1, HOST-2, HOST-3, HOST-4, HOST-5, and HOST-6 are servers in the data center. Deployment Configure an Internet Protocol version 4 (IPv4) unicast routing protocol, for example, the Open Shortest Path First (OSPF) protocol, on the switches to ensure that unicast routes are reachable. Configure the BGP routing protocol (supporting EVPN) on the switches to establish neighbor relationships between each other. Deploy the VXLAN gateway on the core switches. Deploy the VXLAN bridge on the TOR switches. 2.2 EVPN-based Multi-tenant Distributed Deployment Scenario The EVPN-based multi-tenant distributed deployment applies to data center networks that support multiple tenants. The difference between this deployment and the EVPN-based multi-tenant centralized deployment described in section 2.2.1 lies in that: On the distributed deployment network, gateways are deployed on the TOR switches, as shown in Figure 2-2. Tenant A rents VRF-10, which includes VXLAN 10 and VXLAN 20. Tenant B rents VRF-20, which includes VXLAN 100. The networks of Tenant A and Tenant B are isolated from each other. The entire network is formed by a BGP network and includes CORE and TOR switches. The BGP neighbor relationship is formed between every two devices and the BGP-EVPN protocol family is supported. VXLAN gateways are deployed on TOR switches on the network. Anycast gateways can be deployed so that the IP addresses and MAC addresses of all gateways on the network are kept consistent. In this way, the gateway configuration does not need to be modified no matter which TOR switch a virtual machine of a customer is migrated to. VXLANs are unnecessarily deployed on the core switches. ARP suppression can be configured on TOR switches to curb flooding of ARP packets, and the TOR switches respond to ARP requests from hosts as a proxy. The ARP proxy function can be enabled on the TOR switches for all or some VXLANs. In this way, L2 traffic in VXLANs is isolated and server communication traffic in the same VXLAN is forwarded at L3 rather than at L2. ND suppression can be configured on TOR switches to curb flooding of IPv6 ND protocol packets, and the TOR switches respond to IPv6 NS multicast packets from hosts as a proxy. Figure 2- 2 page_6_img_1_df6af556.jpeg Packets between HOST-1 and HOST-4 are forwarded through TOR-1 > TOR-2 at L2 within the VXLAN Packets between HOST-1 and HOST-2 are forwarded through TOR-1 at L3 across the VXLANs. Packets between HOST-1 and HOST-5 are forwarded through TOR-1 > TOR-2 at L3 across the VXLANs. VRF-10 and VRF-20 cannot communicate with each other. If the ARP proxy function is configured on VXLAN 10, packets between HOST-1 and HOST-4 are forwarded through TOR-1 > TOR-2 at L3 within the VXLAN. Remarks CORE indicates a core switch that supports theBGP-EVPN function. TOR1 andTOR2 are access switches that support theVXLAN function. HOST-1, HOST-2, HOST-3, HOST-4, HOST-5, and HOST-6 are servers in the data center Deployment Configure an IPv4 unicast routing protocol, for example, the OSPF protocol, on the switches to ensure that unicast routes are reachable. Configure the BGP routing protocol (supporting EVPN) on the switches to establish neighbor relationships between each other. Deploy the VXLAN bridge on the core switches if required. Deploy the VXLAN gateway on the TOR switches. (Optional) Deploy ARP suppression on the TOR switches. (Optional) Deploy ARP proxy on the TOR switches. (Optional) Deploy IPv6 ND suppression on the TOR switches. (Optional) Deploy the EVPN protocol packet control function on the TOR switches to reduce the traffic of EVPN packets. 2.3 EVPN-based Single-tenant VXLAN Routing Deployment Scenario Single-tenant VXLAN route deployment is shown in Figure 2-3. In this scenario, only the VRF-10 is deployed, which includes VXLAN 10 and VXLAN 20. The border devices are connected to the external network. These devices are deployed in VRF-10 (including VXLAN 90) and interconnect with the external network at L3 via the overlay router interface. The entire network is formed by a BGP network and includes TOR and border devices. The BGP neighbor relationship is formed between every two devices (except between Border-1 and Border-2) and the BGP-EVPN protocol family is supported. The TOR and border devices must use a symmetric VXLAN (VXLAN 100) for interconnection with each other. The border devices import network routes to the TOR switches through the symmetric VXLAN. VXLAN gateways are deployed on TOR switches on the network. Anycast gateways can be deployed so that the IP addresses and MAC addresses of all gateways on the network are kept consistent. In this way, the gateway configuration does not need to be modified no matter which TOR switch a virtual machine of a customer is migrated to. Figure 2-3 page_8_img_1_c7e5d6ad.jpeg Packets between HOST-1 and HOST-3 are forwarded through TOR-1 > TOR-2 at L2 within the VXLAN. Packets between HOST-1 and HOST-2 are forwarded through TOR-1 at L3 across the VXLANs. To access the external network, HOST-1 forwards packets to the border device through TOR1 at L3 across the VXLANs, and then the border device forwards the packets to the external network at L3. Deployment Configure an IPv4 unicast routing protocol, for example, the OSPF protocol, on the switches to ensure that unicast routes are reachable. Configure the BGP routing protocol (supporting EVPN) on the switches to establish neighbor relationships between each other (except between the border devices). Deploy the VXLAN on the border devices for L3 interconnection with the external network. Deploy the VXLAN gateway on the TOR switches. 2.4 EVPN-based Multi-tenant VXLAN Route Deployment Scenario VRF networks are usually allocated to different tenants to support the multi-tenant application in a data center. Multiple VXLANs can be assigned to each tenant. VXLANs of the same tenant can be mutually accessed through the L3 router, while VXLANs of different tenants cannot be mutually accessed, as shown in Figure 2-4. Tenant A rents VRF-10, which includes VXLAN 10 and VXLAN 20. Tenant B rents VRF-20, which includes VXLAN 30. The border devices are connected to the external network. These devices are deployed in VRF-30 (including VXLAN 90) and interconnect with the external network at L3 via the overlay router interface. The networks of Tenant A and Tenant B are isolated from each other. The entire network is formed by a BGP network and includes TOR and border devices. The BGP neighbor relationship is formed between every two devices (except between Border-1 and Border-2) and the BGP-EVPN protocol family is supported. The TOR and border devices must use a symmetric VXLAN (VXLAN 100 and VXLAN 200) for interconnection with each other. The border devices import network routes to the TOR switches through the symmetric VXLAN. VXLAN gateways are deployed on TOR switches on the network. Anycast gateways can be deployed so that the IP addresses and MAC addresses of all gateways on the network are kept consistent. In this way, the gateway configuration does not need to be modified no matter which TOR switch a virtual machine of a customer is migrated to. Figure 2-4 page_9_img_1_30b95c99.jpeg Packets between HOST-1 and HOST-4 are forwarded through TOR-1 > TOR-2 at L2 within the VXLAN. Packets between HOST-1 and HOST-2 are forwarded through TOR-1 at L3 across the VXLANs. To access the external network, HOST-1 forwards packets to the border device through TOR1 at L3 across the VXLANs, and then the border device forwards the packets to the external network at L3. Deployment Configure an IPv4 unicast routing protocol, for example, the OSPF protocol, on the switches to ensure that unicast routes are reachable. Configure the BGP routing protocol (supporting EVPN) on the switches to establish neighbor relationships between each other (except between the border devices). Deploy the VXLAN on the border devices for L3 interconnection with the external network. Deploy the VXLAN gateway on the TOR switches. 2.5 SDN Controller–based Centralized All-active Anycast Gateway Deployment Scenario SND controller–based centralized all-active anycast gateway deployment applies to data center networks that support the control of an SND controller, as shown in Figure 2-5. 1. VXLAN overlay network topology: In this scenario, the VXLAN overlay network is a two-layer structure including a core layer and an access layer. 1. TOR switches serve as VXLAN bridges to directly connect to servers (virtual machines). 2. Core switches serve as VXLAN gateways. Multiple all-active VXLAN physical gateways are deployed in a centralized manner. The physical gateways are in the all-active state. The anycast function is deployed on each physical gateway and the same IP address and MAC address are configured on all gateways to form a logical gateway. The fault of any particular physical gateway does not affect the normal operation of the logical gateway. 3. Virtual tunnel end points (VTEPs), including VXLAN bridges and VXLAN gateways, interconnect with each other through the L3 underlay network. 4. At the underlay layer, an L3 network connection is established between each TOR switch and each physical gateway. However, all physical gateways are virtualized into one logical gateway VTEP to communicate with the external network. Only one VXLAN tunnel is established between a TOR switch and the logical gateway VTEP. Traffic on the tunnel is balanced to multiple physical gateways via the equal-cost multi-path routing (ECMP). 5. No VXLAN tunnel or direct physical link is established between physical gateways. 6. On the server (virtual machine), only one logical gateway is visible. 7. VRF networks are allocated to multiple tenants. Networks of tenants are isolated from each other. 2. SDN controller management: On the network, the administrator can configure the overlay network topology through the SDN controller and deliver the configurations to VTEPs. The administrator can also monitor the status of the overlay topology and network traffic through the SDN controller. In addition, the administrator can manage the servers (virtual machines) on the entire network through the cloud management platform. The SDN controller can associate with the cloud management platform to acquire the configuration information (such as IP address and MAC address) of the virtual machine and deliver the configuration information to VTEPs. After the information is delivered, VXLAN forwarding entries are generated and synchronized on VTEPs. 3. VXLAN device automatic-learning capability VTEPs can automatically learn the MAC address and the ARP routing table of the host if required, which can be used as an emergency solution for the case that the SDN controller fails. The automatic-learning function can be enabled according to the actual deployment. Figure 2-5 page_1_img_1_c7ef2c44.jpeg Deployment Deploy the VXLAN bridging function on TOR switches and the VXLAN gateway function on the core switches. Configure an IPv4 unicast routing protocol, for example, the OSPF protocol, on all VTEPs (including the TOR and core switches) to ensure that unicast routes are reachable. On the core gateways, assign the gateway anycast IP addresses to different routing domains to avoid IP conflicts. 2.6 Deployment of an EVPN Distributed Network to Be Compatible with Non-EVPN VTEP Devices Scenario In a data center where an EVPN-based multi-tenant distributed network is deployed, one VTEP device that does not support the BGP-EVPN protocol(forexample, a virtual switch supporting theVXLAN protocol)is connected.See the figurebelow. BGP is deployed on the TOR and CORE switches.They mutually establish BGP neighbor relationship sand support the EVPN routing protocol. The VXLAN anycast gateways aredeployed on the TOR switches and network-wide gateways share the same IP address and MAC address. TOR switches are directly connect edtoservers(virtualmachines) and core switches are connected to external networks. VXLANs do not need to be deployed on the core switches. ARP suppression can be configured on TOR switches to curb flooding of ARP packets, and the TOR switches respond to ARP requests from hosts as a proxy. The ARPproxy function canbe enabled on the TOR switches for all or someVXLANs. In this way, L2 traffic in VXLANs is isolated and server communication traffic in the same VXLAN is forwarded at L3 rather than at L2. ND suppression can be configured on TOR switches to curb flooding of IPv6 ND protocol packets, and the TORswitches respond to IPv6 NS multicast packets from hosts as a proxy. When the device does not support the data center interconnection tunnel function, BGP-EVEN can be configured on VTEP-1 and all VTEP devices on the network so that VTEP-1 establish VXLAN tunnels with other VTEP devices, therebyformingafullmeshnetwork. The figure below shows the topology. Figure 2- 6 page_2_img_1_f1a649b7.jpeg Note: Blue lines in the figure indicate the VXLAN tunnels that the manually configured VTEP-1 establishes with other VTEPs Deployment Configure an IPv4 unicast routing protocol (such as OSPF) on switches to ensure that unicast routes are reachable. Configure the BGP routing protocol (supporting EVPN) on the TOR and core switches so that the switches establish neighbor relationships mutually. Deploy the VXLAN gateway on the TOR switches and the VXLAN bridge on VTEPs. Configure core switches to interconnect to external networks at L3. (Optional) Deploy ARP suppression on the TOR switches. (Optional) Deploy ARP proxy on the TOR switches. (Optional) Deploy IPv6 ND suppression on the TOR switches. 2.7 Deployment of L2 Subinterfaces to Access a VXLAN Scenario A server can access a VXLAN through an L2 subinterface and the access using other subinterfaces is not affected. Figure 2-7 page_3_img_1_f6db7193.jpeg On the TOR, configure the VLAN or untagged access mode for subinterfaces and configure a VXLAN instance (that is, gateway). Deployment Complete the function configuration on virtual machines on virtual servers as well as on the physical server. Create an L2 subinterface on the TOR switch, configure VXLAN encapsulation and VLAN or untagged encapsulation rule for the subinterface. Create an overlay router interface on the TOR switch and configure the VXLAN gateway IP address. Configure the VXLAN instance to associate with the overlay router interface on the TOR switch to implement VXLAN routing. 3. Features Basic Concepts VXLAN Packet Format A VXLAN encapsulates the Ethernet frames into UDP packets and transmits them on the IP core network. The VXLAN defines a VTEP entity, which encapsulates the data generated by the virtual machine into the UDP headers, and sends the data out. After the encapsulation, the MAC address and VLAN information of the virtual machine no longer serves as the basis for data forwarding. The VTEP entity can be software, a hardware server, or other device. If the VTEP function is directly integrated into a hypervisor (also called virtual machine monitor), all virtual machine traffic is marked with new VXLAN tags and UDP headers before entering the switch. This is equivalent to creating a tunnel between any two virtual machines. As the VLAN information of the virtual machine is externally invisible, a new VXLAN label (VNI) is added. VNIs replace VLANs to represent different VXLAN segments. Same as the forwarding behavior of VLANs, only the virtual machines with the same VNI in the same VXLAN segment can communicate with each other. The new UDP header and VNI form a new frame structure. After receiving the data frame sent from the virtual machine, a VTEP encapsulates four elements (which are the VXLAN header, outer UDP header, outer IPv4 header, and outer Ethernet frame header from inside out) to form a new frame header. In the new frame header, the original source and destination MAC addresses, inner VLAN tag, and Ethernet type that are carried by the inner data frame remain the same. The format of an encapsulated VXLAN frame is as follows: Figure 3-1 image.png 3.1 Packet Format VXLAN Header Information Figure 3-2 page_5_img_1_845cb8cc.jpeg A VXLAN header has 64 bits. In the design of the current protocol version, the sole purpose of a VXLAN header is to carry the 24-bit VNI assigned by the VTEP. Flag (8 bits): The I bit must be set to 1 to indicate a valid VNI, and the R bit must be set to 0. VXLAN segment ID/VNI: Includes 24 bits and indicates the VXLAN network identifier. Only the virtual machines that belong to the same VXLAN can communicate with each other. Reserved: The 24th bit and 8th bit are reserved, and are set to 0. Outer UDP Header Figure 3-3 page_5_img_2_2bb6a8c4.jpeg The definitions of the fields of the UDP header are as follows: Source Port: Indicates the source port ID of the UDP packet. Assigned by the VTEP, the source port ID is the result of the hash operation of the L2 header of the data frame. This hash result can serve as the basis for traffic load balancing. Dest Port: Indicates the destination port ID. The port ID assigned by the Internet Assigned Numbers Authority (IANA) is 4789. UDP Length: Indicates the length of the UDP header. UDP Checksum: Indicates the UDP checksum, which is set to 0 for transmission. Outer IP Header Figure 3-4 page_5_img_3_c4ca73f6.jpeg The definitions of the fields of the outer IP header are as follows: Source IPv4 Address: Identifies the IP address of the VTEP that corresponds to the virtual machine. Destination IPv4 Address: Indicates the unicast or multicast IP address. If it is a unicast IP address, it indicates the IP address of the VTEP corresponding to the virtual machine to be communicated with. The IP address of the outer IP header is no longer the address of the virtual machines of both communication parties, but the address of the VTEPs at both ends of the tunnel. If the hypervisor directly takes over the work of the VTEP, the IP address is the IP address of the NIC of the server that runs the hypervisor. If the VTEP is an access switch, the IP address is the IP address of an egress interface or the IP address of an L3 switch virtual interface (SVI). Outer Ethernet Header Figure 3-5 page_6_img_1_8726c309.jpeg The definitions of the fields of the outer Ethernet header are as follows: Destination MAC address: Next-hop MAC address directed to the IP address of the destination VTEP. Source MAC address: MAC address of the local VTEP. VLAN tag: Optional. 3.2 Forwarding Model VXLAN Bridging Principle VXLAN encapsulates Ethernet packets within UDP packets to transmit them on the IP network. On the receiver, the VXLAN packets are decapsulated into Ethernet packets and then forwarded, as shown in Figure 1-13. Figure 3-6 page_7_img_1_9ec1687f.jpeg Switch TOR1 receives the common Ethernet packet, and then encapsulates the packet into a VXLAN packet. The VXLAN packet is forwarded in the IP core network. As shown in Figure 1-13, R forwards the VXLAN packet. Switch TOR2 receives the VXLAN packet, and then decapsulates and forwards it at L2 of the LAN. Overview Feature Description VXLAN Bridging and Forwarding Encapsulates broadcast, multicast, and unknown unicast packets into IP multicast packets to realize flooding. The well-known unicast packets are encapsulated and forwarded by searching the VXLAN address table for the MAC address and IP address. VXLAN Routing Principle VXLANs interconnect with each other through the VXLAN IP gateway, as shown in Figure 1-14. Figure 3-7 page_8_img_1_8713245a.jpeg To implement cross-VXLAN communication, Server A first sends a packet to the IP gateway, which is deployed on TOR3. The packet sent by Server A is encapsulated by TOR1 into a VXLAN packet and then sent to TOR3. After receiving the VXLAN packet, TOR3 finds that the destination MAC address is the local MAC address and sends the packet to TOR2 after VXLAN routing. After receiving the packet from TOR3, TOR2 decapsulates the packet and sends it to Server B. Overview Feature Description VXLAN Routing and Forwarding Implements cross-VXLAN communication and supports communication between a conventional IP network and a VXLAN. A VXLAN router can serve as a VXLAN IP gateway. 3.3 Forwarding Process Working Principle As shown in Figure 1-15, three servers use a VXLAN to achieve L2 interconnection on the IP network. The VXLAN VNI is 100. Figure 3-8 page_9_img_1_01307b16.jpeg The VXLAN packet forwarding process is described by using an example in which Server A sends an Address Resolution Protocol (ARP) request to Server B and Server B returns an ARP response. Figure 3-9 page_9_img_2_4d2542ae.jpeg Server A sends an ARP request, which is a broadcast packet. After receiving the ARP request, switch TOR1 floods the broadcast packet in tunnel header replication mode, encapsulates it into two unicast packets, and sends them to TOR2 and TOR3 through tunnels. (Switch TOR1 floods the broadcast packet to all tunnels. The tunnel between TOR1 and TOR2, and tunnel between TOR1 and TOR3 are created.) The IP core network forwards the multicast VXLAN packet. Figure 3-10 page_10_img_2_9eb8483f.jpeg After receiving the VXLAN packet, TOR3 decapsulates the packet into an Ethernet packet and implements VXLAN address learning (the VXLAN ID is 100, the MAC address is 0000.0000.0001, and the IP address is 192.168.1.100). Figure 3-11 page_10_img_3_40f3e9ba.jpeg After receiving the VXLAN packet, TOR2 decapsulates the packet into an Ethernet packet, implements address learning (the VXLANID is100,theMAC address is0000.0000.0001,and theIPaddress is 192.168.1.100) andforwards the packet. Then,ServerB receives the ARP request packet and returns a response packet. Figure 3-12 page_1_img_2_05ec1129.jpeg After receiving the ARP response packet from Server B, TOR2 searches the address table and finds that the destination IP address is192.168.1.100.Then,TOR2encapsulates thepacketintoaunicastVXLANpacket(theouter sourceIPaddress is 192.168.2.100) destined for the switch at 192.168.1.100. Figure 3-13 page_2_img_2_3e83e667.jpeg The IP core network forwards the VXLAN packet. TOR1 receives the ARP response packet encapsulated in the VXLAN, decapsulates the packet into an Ethernet packet, implementsVXLAN address learning (theVXLAN ID is 100, the MAC address is 0000.0000.0002, and the IPaddress is 192.168.2.100), and forwards the packet.Then,ServerA receives theARP response packet. Multicast VXLAN Packet Flooding A VXLAN uses multicast packets to flood broadcast, multicast, and unknown unicast packets. After receiving an ARP request packet, TOR1 encapsulates the packetintoamulticastVXLANpacket and sends it toTOR2 and TOR3, as shown in Figure 1-21. Figure 3-14 page_2_img_3_5e6ebdd0.jpeg VTEP AddressLearning As shown in Figure 3-14, in the process of using multicast packets to flood broadcast, multicast, and unknown unicast packets, TOR2 and TOR3 learn theVTEP information during decapsulation, and therefore establish neighbor relationships. Related Configuration Configuring VXLAN Type Instance No VXLAN instanceis configuredon theswitches bydefault. Run the vxlan vni-number command to create aVXLAN instance. Configuring VLAN Associated with VXLAN Instance Run the extend-vlan vlan-id command in VXLAN instance configuration mode to configure the associated VLAN. 4. Configuration 4.1 Configuring VXLAN SDN Configuration Effect Create a VXLAN instance and associate it with the overlay router interface and overlay tunnel interface. Provide the VXLAN routing (IP gateway) function to achieve cross-VXLAN communication. The VXLAN configurations can be delivered by the SDN controller over communication mechanisms such as Network Configuration Protocol (NETCONF), or can be implemented by CLI configuration. Configure the anycast gateway and an anycast MAC address to provide centralized anycast all-active gateway function. The centralized anycast all-active gateways serve as one logical gateway (VTEP) to communicate with external devices and use the same VTEP IP address. Only one tunnel is configured between each TOR bridge device and the logical gateway. Packets are balanced to physical gateways via the underlay ECMP to achieve the gateway all-active function. Enable the SDN controller to deliver the host routes and VXLAN forwarding flow table to the gateways and the gateways generate routes and entries through automatic learning. You can run the configuration commands to enable or disable the automatic learning function on the gateways. When the SDN controller malfunctions, the automatic learning function ensures that the VXLAN works normally. Notes The VXLAN configurations can be delivered by the SDN controller over communication mechanisms such as NETCONF, or can be implementedby CLI configuration.Only configurationdelivery fromtheSDN controlleris recommendedinnormal cases. The VXLAN instances require support from existing unicast routes on the network. Therefore, an IPv4 unicast routing protocol, for example, the OSPF protocol, must be configured on the network devices. On the centralized anycast gateways, assign thegateway anycast IPaddresses to differentrouting domains to avoidIPconflicts. Configuration Steps Creating VXLANInstances Mandatory. Creating Overlay Router Interfaces Mandatory forVXLAN gateways. Configuring Overlay Router Interfaces as Anycast Mandatory for centralized anycast gateways. Configuring Anycast MAC Address Mandatory for centralized anycast gateways. Configuring OverlayTunnel Mandatory. Configuring Source and Destination IP Addresses for Overlay Tunnel Mandatory. Associating VXLAN Instance with Overlay Router Interface Mandatory for gateways. AssociatingVXLANInstancewithVLAN Mandatory forTOR bridges. Associating VXLAN Instance with Overlay Tunnel Mandatory. This is used to statically designate a VXLAN tunnel. Configuring Storm Control of VXLAN Instances Optional. This function is required only when the storm rate needs to be limited based on VXLAN instances. Configuring Static VXLAN MAC Address Table Optional.TheVXLAN MACaddress table delivered by theSDN controlleris represented as a staticVXLAN MACaddress table. Youcan also configure the staticVXLAN MAC address table via CLI configuration. Configuring VXLAN UDP Destination Port Optional.AstheVXLANUDPdestinationportusedbyearlydevicesmaynotbePort4789,youcanrunthis commandtoachieve compatibility. In addition, you can also run this command to customize theVXLAN UDP destination port. TheVXLAN UDP destination port 4789 designated by IANA is used by default. Enabling ARP Automatic Learning Optional. ARP automatic learning is enabled by default. AftertheARPautomaticlearningfunctionisenabled,thegatewayscanautomaticallylearntheAPRentries withoutrelyingon the SDN control to deliver. Enabling IPv6 ND Automatic Learning Optional. IPv6 ND automatic learning is enabled by default. AftertheIPv6NDautomaticlearningfunctionisenabled,thedevicecanautomaticallylearnthehostNDentries, withnoneed to thoroughly rely on ND entries delivered by the SDN controller. Verification After SDN-VXLAN is enabled, virtual machines can communicate with each other. Run the show vxlan vni-number command to check whether the VXLAN devices can learn their mutual VTEP neighbor relationships. Run the show vxlan mac command to check whether theVXLAN MAC address is learned. Run the showarpcommand to checkwhether all local/remote entries are learned. Run the showiproute command tocheck whether the routes ofVXLAN IP gateways are learned. Runtheshowipv6neighborscommandtocheckwhetheralllocal/remoteIPv6NDentriesarelearned.Run theshowipv6 route command to check whether the routes of theVXLAN IPv6 gateways are learned. Run the show vxlan udp-port command to display theVXLAN UDP destination port. Related Commands Creating or EnteringVXLAN Instances Command vxlan vni-number Parameter Description vni-number: Indicates the VNI. The value ranges from 1 to 16777215. Command Mode Global configuration mode Usage Guide N/A Associating VXLAN Instance with VLAN Command extend-vlan vlan-id-list Parameter Description vlan-id-list: Indicates the VLAN ID queue. The VLAN ID ranges from 1 to 4094. Command Mode VXLAN configuration mode Usage Guide Usethis command to associate theVXLAN instance with theVLAN. After receiving theVLAN packet, the device will be associated with the VLAN instance. Creating Overlay Router Interfaces Command interface OverlayRouter port-id Parameter Description port-id: Indicates the ID of an overlay router interface. The ID ranges from 1 to 16,777,215. Command Mode Global configuration mode Usage Guide Similar to SVI in a VLAN, this interface serves as the VXLAN IP gateway in the VXLAN routing environment. Configuring VRF Network for Overlay Router Interface Command vrf forwarding vrf-name Parameter Description vrf-name: Indicates the VRF network to which the overlay router interface belongs. Command Mode Overlay router interface configuration mode Usage Guide AllocateVRF networkstodifferentVXLAN tenants.The trafficofVXLAN instancesofdifferentVRF networks is isolated from each other. Configuring IP Address for Overlay Router Interface Command ip address ip-address mask Parameter Description ip-address: Indicates the IP address of the overlay router interface. mask: Indicates the subnet mask. Command Mode Overlay router interface configuration mode Usage Guide Similartothe IPaddressoftheSVIinaVLAN,this IPaddressservesastheaddressoftheVXLAN IPgatewayintheVXLAN routing environment. Configuring an IPv6Address fortheOverlayRouterInterface Command ipv6 address ip-address mask Parameter Description ip-address: Indicates the IPv6 address of the overlay router interface. mask: Indicates the subnet mask. Command Mode Overlay router interface configuration mode Usage Guide This IPv6 address serves as the VXLAN IPv6 gateway address in the VXLAN routing environment. It is similar to the IP address of an SVI in a VLAN. Configuring the Overlay Router Interface as an Anycast Interface Command anycast-gateway Parameter Description N/A Command Mode Overlay router interface configuration mode Usage Guide Configure the gateway as an anycast gateway. Associating VXLAN Instance with Overlay Router Interface Command router-interface interface-name Parameter Description interface-name: Indicates the name of the overlay router interface. Command Mode VXLAN configuration mode Usage Guide Different VXLANs cannot be associated with the same overlay router interface. Configuring Virtual MAC Address for Any cast Gateways Command fabric anycast-gateway-mac mac-addr Parameter Description mac-addr: Indicates the MAC address. The format is xxxx.xxxx.xxxx. Command Mode Global configuration mode Usage Guide All gateways on which the anycast function is enabled use this MAC address as the gateway MAC address. ThevirtualMACaddressforananycastgateway mustnotbethesameasthe localMACaddressorthesameasthe MAC address of any device on the overlay network. Creating Overlay Tunnel Interfaces Command interface OverlayTunnel port-id Parameter Description port-id: Indicates the ID of the overlay tunnel interface. The ID ranges from 1 to 6144. Command Mode Global configuration mode Usage Guide This interface is used to statically create an overlay tunnel.Youcan run the tunnel-interface command to associate it with a VXLAN. Configuring Source IP Address for Tunnel of Overlay Tunnel Interface Command tunnel source ip-address Parameter Description ip-address: Indicates the source IP address of a tunnel. Command Mode Overlay tunnel interface configuration mode Usage Guide Usethiscommand todesignateasource IPaddressfortheoverlaytunnel. Designatethis IPaddressastheoutersource IP address of a packet for encapsulation and forwarding. Configuring Destination IP Address for Tunnel of Overlay Tunnel Interface Command tunnel destination ip-address Parameter Description ip-address: Indicates the destination IP address of a tunnel. Command Mode Overlay tunnel interface configuration mode Usage Guide Usethiscommand todesignate adestination IPaddressfortheoverlaytunnel. Designate this IPaddress astheouter destination IPaddressofa packetforencapsulationand forwarding.Thedestination IPaddressofthetunnelisglobally unique. Differentoverlaytunnelscannot beconfigured withthesamedestination IPaddress. Otherwise, aconflictoccurs. Associating VXLAN Instance with Overlay Tunnel Interface Command tunnel-interface interface-name Parameter Description interface-name: Indicates the name of the overlay tunnel interface. Command Mode VXLAN configuration mode Usage Guide Use this command to designate the VXLAN VTEP statically. Configuring Storm Control of VXLAN Instances Command storm-control {broadcast | multicast | unicast} [kbps-value | pps pps-value] Parameter Description kbps-value: Indicates the rate limit value (unit: kbit/s). pps-value: Indicates the rate limit value (unit: packet count/s). Command Mode VXLAN configuration mode Usage Guide Configure the storm control when the storm rate needs to be limited based on the VNI. Configuring Static VXLAN MAC Address Table Command vxlan mac static mac-addr vni vxlan-id interface interface-name Parameter Description mac-addr: Indicates the MAC entry address. The format is xxxx.xxxx.xxxx. vxlan-id: Indicates the VNI of the MAC entry. interface-name: Indicates the next hop egress of the MAC table. It can be an overlay tunnel interface, an Ethernet interface, or an aggregate port (AP). vid: Indicates the ID of a VLAN to which the MAC entry belongs. Command Mode Global configuration mode Usage Guide 1. Use this command to deliver the static VXLAN MAC entries via the SDN controller or configure the static VXLAN MAC entries via CLI static configuration. This command is mainly used for setting the host forwarding table. 2. When the next-hop interface isnot an overlay tunnel interface, aVID must be configured and the VID isnot required for overlay tunnel interfaces. Configuring VXLAN UDP Destination Port Command vxlan udp-port port-number Parameter Description port-number: Indicates the UDP destination port ID. The value ranges from0 to65535 and the default value is 4789. Command Mode Global configuration mode Usage Guide Note that the UDP destination port cannot be same as commonly used UDP ports. Configuration Example Only configuration related to theVXLAN is described below. Only IPv4 configuration is used as an example below and the IPv6 scenario configuration is largely the same as the IPv4 scenario configuration. VXLAN Configuration Instance Scenario Figure 4-1 page_9_img_1_560c938d.jpeg Configuration Steps Configure an IPv4 unicast routing protocol such as the OSPF protocol on TOR-1, TOR-2, CORE-1, and CORE-2 to ensure that unicast routes are reachable. Configure loopback IP addresses on TOR-1, TOR-2, CORE-1, and CORE-2 and distribute packets via the unicast routing protocol. The VTEP IP addresses of CORE-1 and CORE-2 must be the same and be allocated to different routing domains. Configure a VXLAN on the virtual server and designate the gateway address of the virtual machine. Establish a BGP neighbor relationship between CORE-1 and CORE-2 and configure the BGP-EVPN routing protocol on them. The following configuration can be delivered by the SDN controller: Create VXLAN instances VXLAN10 and VXLAN20 on TOR-1, and associate them with VLAN10 and VLAN20 respectively. Configure the address learning mode as SDN controller advertisement. Configure two overlay tunnels to connect TOR-1 with TOR-2 and CORE. Associate VXLAN10 and VXLAN20 with the two tunnels separately. Create VXLAN instance VXLAN20 on TOR-2 and associate it with VLAN20. Configure the address learning mode as SDN controller advertisement. Configure two overlay tunnels to connect TOR-2 with TOR-1 and CORE-2. Associate VXLAN20 with the two tunnels. Create VXLAN instances VXLAN10 and VXLAN20 on CORE-1. Configure the address learning mode as SDN controller advertisement. Configure the anycast MAC address. Configure two overlay router gateway interfaces and configure their IP addresses as 10.1.1.1/24 and 10.1.2.1/24 respectively. Associate VXLAN10 with the overlay router gateway interface with the IP address 10.1.1.1/24. Associate VXLAN20 with the overlay router gateway interface with the IP address 10.1.2.1/24. Configure two overlay tunnels to connect CORE-1 with TOR-1 and TOR-2. Associate VXLAN10 and VXLAN20 with the two tunnels separately. Enable the synchronization function on all-active VXLAN gateways to synchronize the automatically learned entries between thegateways. The configuration of CORE-2 is same with that of CORE-1. TOR1 TOR1# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR1(config)#interface loopback0 TOR1(config-if-Loopback 0)# ip address 1.1.1.1 255.255.255.255 TOR1(config-if-Loopback 0)# exit TOR1(config)# interface OverlayTunnel 1 TOR1(config-if-OverlayTunnel 1)# tunnel source 1.1.1.1 TOR1(config-if-OverlayTunnel 1)# tunnel destination 2.2.2.2 TOR1(config-if-OverlayTunnel 1)# exit TOR1(config)# interface OverlayTunnel 2 TOR1(config-if-OverlayTunnel 2)# tunnel source 1.1.1.1 TOR1(config-if-OverlayTunnel 2)# tunnel destination 3.3.3.3 TOR1(config-if-OverlayTunnel 2)# exit TOR1(config)# vxlan 10 TOR1(config-vxlan)# tunnel-interface OverlayTunnel 1 TOR1(config-vxlan)# tunnel-interface OverlayTunnel 2 TOR1(config-vxlan)# extend-vlan10 TOR1(config-vxlan)# end TOR1(config)# vxlan 20 TOR1(config-vxlan)# tunnel-interface OverlayTunnel 1 TOR1(config-vxlan)# tunnel-interface OverlayTunnel 2 TOR1(config-vxlan)# extend-vlan20 TOR1(config-vxlan)# end TOR2 TOR2# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR2(config)#interface loopback0 TOR2(config-if-Loopback 0)# ip address 2.2.2.2 255.255.255.255 TOR2(config-if-Loopback 0)# exit TOR2(config)# interface OverlayTunnel 1 TOR2(config-if-OverlayTunnel 1)# tunnel source 2.2.2.2 TOR2(config-if-OverlayTunnel 1)# tunnel destination 1.1.1.1 TOR2(config-if-OverlayTunnel 1)# exit TOR2(config)# interface OverlayTunnel 2 TOR2(config-if-OverlayTunnel 2)# tunnel source 2.2.2.2 TOR2(config-if-OverlayTunnel 2)# tunnel destination 3.3.3.3 TOR2(config-if-OverlayTunnel 2)# exit TOR2(config)# vxlan 20 TOR2(config-vxlan)# tunnel-interface OverlayTunnel 1 TOR2(config-vxlan)# tunnel-interface OverlayTunnel 2 TOR2(config-vxlan)# extend-vlan20 TOR2(config-vxlan)# end CORE1、CORE2 Create VXLAN instances VXLAN10 and VXLAN20 on CORE-1. Set the address learning mode to SDN controller advertisement. Configure an anycast MAC address. Configure two overlay router gateway interfaces and set their IP addresses to 10.1.1.1/24 and 10.1.2.1/24 respectively. Configure VXLAN10 to associate with the overlay router gateway interface with the IP address of 10.1.1.1/24. Configure VXLAN20 to associate with the overlay router gateway interface with the IP address of 10.1.2.1/24. Configure two overlay tunnels reachable to TOR1 and TOR2 respectively. Configure VXLAN10 and VXLAN20 to associate with the two tunnels respectively. Configure loopback 1 on both CORE-1 and CORE-2 and set the IP address to 3.3.3.4 and 3.3.3.5 for loopback1. The two core switches establish a BGP neighbor relationship through loopback 1. Configure the L2VPN EVPN address family activation command. TOR1# configure terminal Enter configuration commands, one per line. End with CNTL/Z. CORE1(config)# fabric anycast-gateway-mac 0000.1234.5678 CORE1(config)# interface loopback 0 CORE1(config-if-Loopback 0)# ip address 3.3.3.3 255.255.255.255 CORE1(config-if-Loopback 0)# exit CORE1(config)# interface loopback1 CORE1(config-if-Loopback 0)# ip address 3.3.3.4 255.255.255.255 CORE1(config-if-Loopback 0)# exit CORE1(config)# route bgp 10000 CORE1(config-router)# neighbor 3.3.3.5 remote-as 10000 CORE1(config-router)# neighbor 3.3.3.5 update-source Loopback 1 CORE1(config-router)# address-family l2vpn evpn CORE1(config-router-af)# neighbor 3.3.3.5 activate CORE1(config-router-af)# neighbor 3.3.3.5 send-community extended CORE1(config-router-af)# exit-address-family CORE1(config-router)# exit CORE1(config)# interface OverlayTunnel 1 CORE1(config-if-OverlayTunnel 1)# tunnel source 3.3.3.3 CORE1(config-if-OverlayTunnel 1)# tunnel destination 2.2.2.2 CORE1(config-if-OverlayTunnel 1)# exit CORE1(config)# interface OverlayTunnel 2 CORE1(config-if-OverlayTunnel 2)# tunnel source 3.3.3.3 CORE1(config-if-OverlayTunnel 2)# tunnel destination 1.1.1.1 CORE1(config-if-OverlayTunnel 2)# exit CORE1(config)# interface overlayrouter1 CORE1(config-if-OverlayRouter 1)# ip address 10.1.1.1/24 CORE1(config-if-OverlayRouter 1)# anycast-gateway CORE1(config-if-OverlayRouter 1)# exit CORE1(config)# interface overlayrouter2 CORE1(config-if-OverlayRouter 2)# ip address 10.1.2.1/24 CORE1(config-if-OverlayRouter 2)# anycast-gateway CORE1(config-if-OverlayRouter 2)# exit CORE1(config)# vxlan 10 CORE1(config-vxlan)# tunnel-interface OverlayTunnel 1 CORE1(config-vxlan)# tunnel-interface OverlayTunnel 2 CORE1(config-vxlan)# router-interface OverlayRouter 1 CORE1(config-vxlan)# end CORE1(config)# vxlan 20 CORE1(config-vxlan)# tunnel-interface OverlayTunnel 1 CORE1(config-vxlan)# tunnel-interface OverlayTunnel 2 CORE1(config-vxlan)# router-interface OverlayRouter 2 CORE1(config-vxlan)# end Verification Verify that the virtual machines of HOST-1, HOST-2, HOST-3, and HOST-4 can ping each other. Display the static MAC entries of hosts delivered by the SDN controller on TOR and core switches. TOR1(config)# show vxlan mac Vxlan MAC Address Type Location Interface Vlan ---------- -------------------- -------- -------- ------------------------------ ------ 10 0000.0000.0001 STATIC LOCAL GigabitEthernet 0/1 10 10 0000.1234.5678 STATIC REMOTE OverlayTunnel 2 - 20 0000.0000.0002 STATIC LOCAL GigabitEthernet 0/2 20 20 0000.0000.0003 STATIC REMOTE OverlayTunnel 1 - 20 0000.0000.0004 STATIC REMOTE OverlayTunnel 1 - 20 0000.1234.5678 STATIC REMOTE OverlayTunnel 2 TOR2(config)# show vxlan mac Vxlan MAC Address Type Location Interface Vlan ---------- -------------------- -------- -------- ------------------------------ ------ 10 0000.0000.0001 STATIC REMOTE OverlayTunnel 1 - 10 0000.1234.5678 STATIC REMOTE OverlayTunnel 2 - 20 0000.0000.0002 STATIC REMOTE OverlayTunnel 1 - 20 0000.0000.0003 STATIC LOCAL GigabitEthernet 0/1 20 20 0000.0000.0004 STATIC LOCAL GigabitEthernet 0/2 20 20 0000.1234.5678 STATIC REMOTE OverlayTunnel 2 - CORE1# show vxlan mac Vxlan MAC Address Type Location Interface Vlan ---------- -------------------- -------- -------- ------------------------------ ------ 10 0000.0000.0001 STATIC REMOTE OverlayTunnel 2 - 20 0000.0000.0002 STATIC REMOTE OverlayTunnel 2 - 20 0000.0000.0003 STATIC REMOTE OverlayTunnel 1 - 20 0000.0000.0004 STATIC REMOTE OverlayTunnel 1 CORE2# show vxlan mac Vxlan MAC Address Type Location Interface Vlan ---------- ------------------------------------------------------------------------ 10 0000.0000.0001 STATIC REMOTE OverlayTunnel 2 20 0000.0000.0002 STATIC REMOTE OverlayTunnel 2 20 0000.0000.0003 STATIC REMOTE OverlayTunnel 1 20 0000.0000.0004 STATIC REMOTE OverlayTunnel 1 4.2 Configuring VXLAN EVPN Configuration Effect Enable the control plane learning function to implement VXLAN tunnel learning, MAC address learning, and route learning via control plane protocols, thereby finally implementing VXLAN bridging, VXLAN routing, and data communication between VXLANs and between a VXLAN and an external network. Support functions such as anycast gateways, symmetric VXLAN instances, and ARP suppression in EVPN control plane mode. Notes The VXLAN instances require support from existing unicast routes on the network. Therefore, an IPv4 unicast routing protocol, for example, the OSPF protocol must be configured on the network devices. The MP-BGP-EVPN protocol is required for VXLANs to implement VXLAN tunnel learning, MAC address learning, and route learning. Therefore, the devices on the network must complete BGP-related configurations. Configuration Steps Configuring Loopback Interface Associated with Local End Mandatory. Configure the IP address of a loopback interface as the IP address of the local VTEP. One VTEP device can be associated with only one loopback interface to serve as the VXLAN VTEP IP address. If the L3 egress is an overlay router interface during static route configuration, the next-hop IP address cannot be set to the VTEP IP address. Configuring Virtual MAC Address for Anycast Gateways Optional. Configure a unified virtual MAC address for all anycast gateways on the network. The anycast function can be enabled on the VXLAN overlay router interface of the local device only after the virtual MAC address is configured. Configuring ARP Suppression Optional. After ARP suppression is enabled, the switch responds to the ARP request from the host as a proxy, reducing the flooded ARP data. ARP suppression is generally enabled on the TOR bridge devices in a centralized deployment scenario, or on the distributed gateways in a distributed deployment scenario. Configuring ARP Proxy Optional. After ARP suppression is enabled on a VTEP device, you can enable the ARP proxy function on an overlay router interface. After ARP proxy is enabled, the VTEP device responds to ARP requests from hosts as a proxy and the MAC address used for proxy response is the gateway MAC address configured on the VTEP device. In this way, the MAC address in the ARP request responses are the MAC address of the VTEP device, and the traffic between hosts in the same VNI is forwarded at L3. ARP proxy can be enabled only on VXLAN gateways and is generally enabled on distributed gateways in distributed deployment scenarios. Configuring IPv6 ND Suppression Optional. After IPv6 ND suppression is enabled, the VTEP device responds to NS multicast packets from hosts as a proxy, to reduce flooded NS multicast packets on the network. IPv6 ND suppression is generally enabled on distributed gateways in distributed deployment scenarios. Configuring the EVPN Protocol Packet Control Function In symmetric EVPN deployment scenarios, the EVPN protocol packet control function can be configured on TOR switches to reduce the traffic of EVPN packets. Currently, the EVPN protocol packet control function includes the following: Extracting MAC entries from EVPN MAC-IP type-2 routes (ARP entries) on a L2-VPN VXLAN instance Extracting MAC entries from EVPN MAC-IPv6 type-2 routes (IPv6 ND entries) on a L2-VNI VXLAN instance Banning synchronization of the local MAC address to the remote VTEP through EVPN messages on an L2-VNI VXLAN instance Banning delivery of the MAC addresses remotely synchronized through EVPN messages to the local MAC address table on an L2-VNI VXLAN instance Stopping an L2-VNI VXLAN instance from generating EVPN type-2 routes Configuring Remote ARP Packet Learning Mandatory for centralized gateways and not recommended for other devices. After the remote ARP packet learning function is enabled, the VXLAN gateways can learn the VXLAN route entries from the encapsulated VXLAN ARP packets received from the VXLAN tunnels. Configuring Remote IPv6 ND Protocol Packet Learning Mandatory for centralized gateways and not recommended for other devices. After the remote IPv6 ND protocol packet learning function is enabled, the VXLAN gateway can learn IPv6 ND entries from VXLAN-encapsulated IPv6 protocol packets received from VXLAN tunnels. Creating VXLAN Instances Mandatory. Associating VXLAN Instance with Overlay Router Interface Mandatory for VXLAN gateways. Only after the VXLAN is associated with the overlay router interface, the device can provide the VXLAN routing function and serve as a VXLAN IP gateway. Associating VXLAN Instance with VLAN Mandatory for VXLAN devices directly connected to the host. Only after a VLAN is associated with a VXLAN instance, packets of the VLAN can be encapsulated into VXLAN packets and then forwarded. After a VLAN is associated with a VXLAN, all packets of the VLAN will be encapsulated into VXLAN packets. Therefore, an SVI cannot be used as the VLAN IP gateway on the device. Configuring Storm Control of VXLAN Instances Optional. This function is required only when the storm rate needs to be limited based on VXLAN instances. Configuring VXLAN UDP Destination Port Optional. As the VXLAN UDP destination port used by early devices may not be Port 4789, you can run this command to achieve compatibility. In addition, you can also run this command to customize the VXLAN UDP destination port. The VXLAN UDP destination port 4789 designated by IANA is used by default. Configuring Symmetric Instances Optional. Symmetric instances need to be configured only in symmetric scenarios. Only one symmetric instance can be configured for each VRF network. After a symmetric instance is configured in a VRF network, L3 forwarding of other asymmetric instances is all switched to the symmetric instance for implementation. Configuring Static VXLAN Network Routes Optional. Configure the static VXLAN network routes based on VXLAN instances if required. Verification Based on EVPN control plane learning, VXLAN tunnels, VXLAN MAC entries, and VXLAN route entries can be formed. Run the following commands for verification. Run the show vxlan vni-number command to check whether the local and remote VXLAN devices can learn mutual VTEP neighbor relationships. Run the show vxlan mac command to check whether the VXLAN MAC address is learned. Run the show arp command to check whether the ARP entry of the VXLAN IP gateway is learned. Run the show ipv6 neighbors command to check whether all local/remote IPv6 ND entries are learned. Run the show vxlan udp-port command to display the VXLAN UDP destination port. Related Commands Configuring Loopback Interface Associated with Local End Command source loopback loopback-port-id Parameter Description Loopback-port-id: Indicates the ID of the loopback interface. Command Mode VTEP configuration mode Usage Guide The local VETP IP address is the configured loopback interface IP address. Configuring Virtual MAC Address for Anycast Gateways Command fabric anycast-gateway-mac mac-addr Parameter Description mac-addr: Indicates the MAC address. The format is xxxx.xxxx.xxxx. Command Mode Global configuration mode Usage Guide All gateways on which the anycast function is enabled use this MAC address as the gateway MAC address. ThevirtualMACaddressforananycastgateway mustnotbethesameasthe localMACaddressorthesameasthe MAC address of any device on the overlay network. Configuring Remote ARP Packet Learning Command remote arp learn enable Parameter Description N/A Command Mode VTEP configuration mode Usage Guide Enable or disable the remote ARP packet learning function globally.After this function is enabled, the VXLAN gateways can learn the VXLAN route entries from the encapsulated VXLAN ARP packets received from the VXLAN tunnels. Configuring Remote IPv6 ND Protocol Packet Learning Command remote nd learn enable Parameter Description N/A Command Mode VTEP configuration mode Usage Guide Enable or disable the remote IPv6 ND packet learning function globally. After this function is enabled, the device can learn IPv6 ND entries from the VXLAN-encapsulated IPv6 NS packets received from VXLAN tunnels. Configuring ARP Suppression Command arp suppress enable Parameter Description N/A Command Mode VTEP configuration mode Usage Guide Enable or disable ARP suppressionglobally.AfterARPsuppression isenabled,theswitchresponds totheARP requests from the hostasa proxy.TheVNI-basedARPsuppressionmaybealsosupported, dependingontheproducttype.Youcan configure global ARP suppression or VNI-based ARP suppression based on the actual application scenario. Configuring VNI-based ARP Suppression Command arp suppress enable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide Enable or disable VNI-based ARP suppression. After ARP suppression is enabled, the switch responds toARP requests from hosts as a proxy. The global ARP suppression may be also supported, depending on the product type. You can configure global ARP suppression or VNI-based ARP suppression based on the actual application scenario. Configuring ARP Proxy Command route-in-vni Parameter Description N/A Command Mode Overlay router interface configuration mode Usage Guide After the intra-VNI routing function (ARP proxy) is enabled on an overlay router interface, the VTEP device uses its gateway MAC address to respond to all ARP requests from hosts in theVNI, towhich the overlay router interface belongs, when serving as an ARP proxy. In this way, the communication traffic between hosts in the same VNI is forwarded through VXLAN routes. Configuring Global IPv6 ND Suppression Command nd suppress enable Parameter Description N/A Command Mode VTEP configuration mode Usage Guide Enable or disable the global IPv6 ND suppression function. After IPv6 ND suppression is enabled, the device responds to IPv6 NS multicast packets from hosts as a proxy. The VNI-based IPv6 ND suppression may be also supported, depending on the product type. You can configure global IPv6 ND suppression or VNI-based IPv6 ND suppression based on the actual application scenario. Configuring VNI-based IPv6 ND Suppression Command nd suppress enable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide Enable or disable the VNI-based IPv6 ND suppression function. After IPv6 ND suppression is enabled, the device responds to IPv6 NS multicast packets from hosts as a proxy. The global IPv6 ND suppression may be also supported, depending on the product type. Youcan configure global IPv6 ND suppression or VNI-based IPv6 ND suppression based on the actual application scenario. Extracting MAC Entries from EVPN MAC-IP Type-2 Routes (ARP Entries) Command evpn arp mac-learning enable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide After this command is configured, the device parses one ARP entry and one MAC entry from a MAC-IP type-2 route synchronized from the VXLAN-EVPN neighbor. This command is disabled by default and the device parsesoneARP entry but no MAC entry from a MAC-IP type-2 route synchronized from the VXLAN-EVPN neighbor. This command is configured on aVXLAN instance and affects only the EVPN entry parsing of theVXLAN instance. Other VXLAN instances, for which this command is not configured, are not affected. This command can be used in combination with theevpn mac advertise disable command. After they are executed, the network-wide VXLAN-EVPN neighbors synchronize only MAC-IP type-2 routes but no MAC-only type-2 routes. All devices parse and extract MAC entries from MAC-IP type-2 routes. This command is configured on L2-VNI VXLAN instances. Extracting MAC Entries from EVPN MAC-IPv6 Type-2 Routes (IPv6 ND Entries) Command evpn nd mac-learning enable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide After this command is configured, the device parses one IPv6 ND entry and one MAC entry from a MAC-IPv6 type-2 route (IPv6 ND entry) synchronized from the VXLAN-EVPN neighbor. This command is disabled by default and the device parses one IPv6 ND entry but no MAC entry from a MAC-IPv6 type-2 route synchronized from the VXLAN-EVPN neighbor. This command is configured on aVXLAN instance and affects only the EVPN entry parsing of theVXLAN instance. Other VXLAN instances, for which this command is not configured, are not affected. This command can be used in combination with theevpn mac advertise disable command. After they are executed, the network-wide VXLAN-EVPN neighbors synchronize only MAC-IPv6 type-2 routes but no MAC-only type-2 routes. All devices parse and extract MAC entries from MAC-IPv6 type-2 routes. This command is configured on L2-VNI VXLAN instances. Configuring an L2-VNI VXLAN Instance Not to Synchronize the Local MAC Address to the Remote VTEP Through EVPN Messages Command evpn mac advertise disable Parameter Description N/A Command VXLAN configuration mode Mode Usage Guide This command is not configured on a device by default. The device generates one MAC-only type-2 route through the VXLAN-EVPN protocol based on a locally learned MAC entry, and synchronizes the type-2 route to the EVPN neighbor (that is, remote VTEP). Then, the remote VTEP can learn the MAC entry from the MAC-only type-2 route. After this command is configured, the device does not generate VXLAN-EVPN MAC-only type-2 routes based on MAC entries, and therefore, it will not advertise MAC-only type-2 routes to the EVPN neighbor. This command is configured on aVXLAN instance and affects only whether the VXLAN instance generates MAC-only type-2 routes. Other VXLAN instances, for which this command is not configured, can still generate MAC-only type-2 routes. This command can be used in combination with theevpn arp mac-learning enable and evpn nd mac-learning enable commands. After they are executed, the network-wide VXLAN-EVPN neighbors synchronize only MAC-IP type-2 routes but no MAC-only type-2 routes.All devicesparse and extract MAC entries from MAC-IP or MAC-IPv6 type-2 routes. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances with the symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. Configuring an L2-VNI VXLAN Instance Not to Deliver MAC Addresses Remotely Synchronized Through EVPN Messages to the Local MAC Address Table Command evpn mac inactive Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide After this command is configured, the device does not learn MAC entries from VXLAN-EVPN type-2 routes (MAC-IP or MAC-only type-2 routes) synchronized from neighbors. This command isnot configuredona device by default. The device learns MAC entriesfromVXLAN-EVPN type-2 routes synchronized from neighbors. This command is configured on aVXLAN instance and affectsonlywhethertheVXLAN instance learns MACentriesfrom VXLAN-EVPN type-2 routes. Other VXLAN instances, for which this command is not configured, can still learn MAC entries. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances with the symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. Configuring an L2-VNI VXLAN Instance Not to Generate EVPN Type-2 Routes Command evpn rt-2 advertise disable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide This command is not configured on a device by default. The device generates one MAC-only type-2 route through the VXLAN-EVPN protocol based on a locally learned MAC entry, and synchronizes the type-2 route to the EVPN neighbor (that is, remote VTEP). Then, the remote VTEP learns the MAC entry from the MAC-only type-2 route. In addition, the devicegenerates one MAC-IPtype-2 route through theVXLAN-EVPN protocol basedona locally learned ARP entry and synchronizesthetype-2 routetotheEVPN neighbor.Then, the remoteVTEP learnstheARP entry and host routefromthe MAC-IP type-2 route. The device generates one MAC-IPv6 type-2 route through the VXLAN-EVPN protocol basedona locally learned IPv6 ND entry, and synchronizes the type-2 route to the EVPN neighbor. Then, the remote VTEP learns the IPv6 ND entry and host route from the MAC-IPv6 type-2 route. After this command is configured, the MAC entries, ARP entries, and IPv6 ND entries of the device are not used to generate VXLAN-EVPN type-2 routes and therefore, no type-2 route is advertised to the EVPN neighbor. Thiscommand isconfigured onaVXLAN instanceand affectsonly whether theVXLAN instancegeneratestype-2 routes. Other VXLAN instances, for which this command is not configured, can still generate type-2 routes. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances with the symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. Creating Overlay Router Interfaces Command interface OverlayRouter port-id Parameter Description port-id: Indicates the ID of an overlay router interface. The ID ranges from 1 to 16,777,215. Command Mode Global configuration mode Usage Guide Similar to SVI in a VLAN, this interface serves as the VXLAN IP gateway in the VXLAN routing environment. Configuring IP Address for Overlay Router Interface Command ip address ip-address mask Parameter Description ip-address: Indicates the IP address of the overlay router interface. mask: Indicates the subnet mask. Command Mode Interface configuration mode Usage Guide Similartothe IPaddressoftheSVIinaVLAN,this IPaddressservesastheaddressoftheVXLAN IPgatewayintheVXLAN routing environment. Configuring an IPv6 Address for the Overlay Router Interface Command ipv6 address ip-address mask Parameter Description ip-address: Indicates the IPv6 address of the overlay router interface. mask: Indicates the subnet mask. Command Mode Overlay router interface configuration mode Usage Guide This IPv6 address serves as the VXLAN IPv6 gateway address in the VXLAN routing environment. It is similar to the IP address of an SVI in a VLAN. Associating Overlay Router Interface with VRF Network Command vrf forwarding table name Parameter Description Table name: Indicates the VRF network associated with the overlay router interface. Command Mode Interface configuration mode Usage Guide Use this command to associate an overlay router interface with aVRF network in the VXLAN routing environment, to implement VXLAN L3 route isolation. Creating or Entering VXLAN Instances Command vxlan vni-number Parameter Description vni-number: Indicates the VNI. The value ranges from 1 to 16777215. Command Mode Global configuration mode Usage Guide N/A Configuring Symmetric Instances Command symmetric Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide Nosymmetric instance is configured by default. Symmetric instances are used tomanagetheL3 forwarding entries of all asymmetric instances of the VRF networks associated with the symmetric instances. Associating VXLAN Instance with Overlay Router Interface Command router-interface interface-name Parameter Description interface-name: Indicates the name of the overlay router interface. Command Mode VXLAN configuration mode Usage Guide Different VXLANs cannot be associated with the same overlay router interface. Configuring VXLAN UDP Destination Port Command vxlan udp-port port-number Parameter Description port-number: Indicates the UDP destination port ID. The value ranges from0 to65535 and the default value is 4789. Command Mode Global configuration mode Usage Guide Note that the UDP destination port cannot be same as commonly used UDP ports. Configuring Storm Control of VXLAN Instances Command storm-control {broadcast | multicast | unicast} [kbps-value | pps pps-value] Parameter Description kbps-value: Indicates the rate limit value (unit: kbit/s). pps-value: Indicates the rate limit value (unit: packet count/s). Command Mode VXLAN configuration mode Usage Guide Configure the storm control when the storm rate needs to be limited based on the VNI. Configuration Example Only configuration related to the VXLAN is described below. Only IPv4 configuration is used as an example below and the IPv6 scenario configuration is largely the same as the IPv4 scenario configuration. 4.2.1 Configuring EVPN-based Multi-tenant Centralized Scenario Figure 4-2 page_2_img_1_3d32c80f.jpeg Configuration Steps Configure an IPv4 unicast routing protocol such as the OSPF protocol on CORE, TOR-1, and TOR-2 to ensure that unicast routes are reachable. Configure the BGP-EVPN routing protocol on CORE, TOR-1, and TOR-2 to establish BGP neighbor relationships between the three devices and to support the EVPN protocol family. Configure the EVI for BGP-EVPN on CORE, TOR-1, and TOR-2. For details, see BGP-EVPN Configuration Guide. Configure a VXLAN on the virtual server and designate the gateway address of the virtual machine. Associate the VTEP with the loopback interface on TOR-1, TOR-2, and CORE to establish tunnels. Create VXLAN instances on TOR-1, TOR-2, and CORE and associate the VXLAN instances with VLANs. Create overlay router interfaces and configure the VXLAN gateway IP address on CORE. Configure different VRF networks for different overlay router interfaces to determine their respective tenants. Associate VXLAN instances with overlay router interfaces on CORE to realize VXLAN routing. Enable the remote ARP packet learning function on CORE to generate VXLAN routing entries dynamically. (Optional) Configure ARP suppression on TOR-1 and TOR-2 to reduce the ARP packets entering the VXLAN. HOST Configuring the IP address and gateway according to Figure 1-23 (the detailed configuration on the server is omitted herein). CORE The configuration of the OSPF, and Ethernet interface is omitted herein. The following describe only the VXLAN configuration. CORE# configure terminal Enter configuration commands, one per line. End with CNTL/Z. CORE(config)# interface Loopback 1 CORE(config-if- Loopback 1)# ip address 1.1.1.1/32 CORE(config-if- Loopback1)#exit CORE(config)# vtep CORE(config-vtep)# source loopback 1 CORE(config-vtep)#remote arp learn enable CORE(config-vtep)# exit CORE(config) # ip vrf vrf-10 CORE(config-v rf)# rd 10:10 CORE(config-vrf)#route-target both1000:1000 CORE(config-vrf)# exit CORE(config) # ip vrf vrf-20 CORE(config-v rf)# rd 20:20 CORE(config-vrf)#route-target both2000:2000 CORE(config-vrf)# exit CORE(config)# int overlayrouter 10 CORE(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 CORE(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 CORE(config-if-OverlayRouter 10)# exit CORE(config)# int overlayrouter 20 CORE(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 CORE(config-if-OverlayRouter 20)# ip address 10.1.2.1/24 CORE(config-if-OverlayRouter 20)# exit CORE(config)# int overlayrouter 100 CORE(config-if-OverlayRouter 100)# ip vrf forwarding vrf-20 CORE(config-if-OverlayRouter 100)# ip address 10.1.3.1/24 CORE(config-if-OverlayRouter 100)# exit CORE(config)# vxlan 10 CORE(config-vxlan)# router-interface OverlayRouter 10 CORE(config-vxlan)# exit CORE(config)# vxlan 20 CORE(config-vxlan)# router-interface OverlayRouter 20 CORE(config-vxlan)# exit CORE(config)# vxlan 100 CORE(config)# vxlan 100 CORE(config-vxlan)# router-interface OverlayRouter 100 CORE(config-vxlan)# exit CORE(config)# router bgp 64512 CORE(config-router)# neighbor 2.2.2.2 remote-as 64512 CORE(config-router)# neighbor 3.3.3.3 remote-as 64512 CORE(config-router)# neighbor 2.2.2.2 update-source Loopback 1 CORE(config-router)#neighbor3.3.3.3 update-sourceLoopback1 CORE(config-router)# address-family l2vpn evpn CORE(config-router-af)#neighbor 2.2.2.2activate CORE(config-router-af)#neighbor 3.3.3.3activate CORE(config-router-af)# neighbor 2.2.2.2 route-reflector-client CORE(config-router-af)# neighbor 3.3.3.3 route-reflector-client CORE(config-router-af)# exit CORE(config-router)# address-family ipv4 vrf vrf-10 CORE(config-router-af)# network 10.1.1.0 mask 255.255.255.0 CORE(config-router-af)# network 10.1.2.0 mask 255.255.255.0 CORE(config-router-af)# exit CORE(config-router)# address-family ipv4 vrf vrf-20 CORE(config-router-af)# network 10.1.3.0 mask 255.255.255.0 CORE(config-router-af)# exit CORE(config-router)# exit CORE(config)# evpn CORE(config-evpn)# vni 10 CORE(config-evpn-vni)# rd auto CORE(config-evpn-vni)# route-target both auto CORE(config-evpn-vni)# exit CORE(config-evpn)# vni 20 CORE(config-evpn-vni)# rd auto CORE(config-evpn-vni)# route-target both auto CORE(config-evpn-vni)# exit CORE(config-evpn)# vni 100 CORE(config-evpn-vni)# rd auto CORE(config-evpn-vni)# route-target both auto CORE(config-evpn-vni)# exit TOR1 TOR1# configure terminal Enter configuration commands, one per line. End with CNTL/Z. TOR1(config)# interface Loopback 1 TOR1(config-if-Loopback1)#ipaddress2.2.2.2/32 TOR1(config-if- Loopback 1)# exit TOR1(config)# vtep TOR1(config-vtep)# source loopback 1 TOR1(config-vtep)# arp suppress enable TOR1(config-vtep)# exit TOR1(config)# vxlan 10 TOR1(config-vxlan)# extend-vlan 10 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# vxlan 20 TOR1(config-vxlan)# extend-vlan 20 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# router bgp 64512 TOR1(config-router)# neighbor 1.1.1.1 remote-as 64512 TOR1(config-router)# neighbor 1.1.1.1 update-source loopback 1 TOR1(config-router)# address-family l2vpn evpn TOR1(config-router-af)# neighbor 1.1.1.1 activate TOR1(config-router-af) #exit TOR1(config-router)# exit TOR1(config)# evpn TOR1(config-evpn)# vni 10 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 20 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR2 TOR2# configure terminal Enter configuration commands, one per line. End with CNTL/Z. TOR2(config)# interface Loopback 1 TOR2(config-if-Loopback1)#ipaddress3.3.3.3/32 TOR2(config-if- Loopback 1)# exit TOR2(config)# vtep TOR2(config-vtep)# source loopback 1 TOR2(config-vtep)# arp suppress enable TOR2(config-vtep)# exit TOR2(config)# vxlan 100 TOR2(config-vxlan)# extend-vlan 100 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# vxlan 20 TOR2(config-vxlan)# extend-vlan 20 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# router bgp 64512 TOR2(config-router)# neighbor 1.1.1.1 remote-as 64512 TOR2(config-router)# neighbor 1.1.1.1 update-source loopback 1 TOR2(config-router)# address-family l2vpn evpn TOR2(config-router)# neighbor 1.1.1.1 activate TOR2(config-router-af)# exit TOR2(config-router)# exit TOR2(config)# evpn TOR2(config-evpn)# vni 20 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 100 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit Verification Verify that HOST-1, HOST-2, HOST-3, and HOST-4 can ping each other. Verify that HOST-5 and HOST-6 can ping each other. Verify that HOST-1, HOST-2, HOST-3, and HOST-4 cannot ping HOST-5 and HOST-6. Verify that the virtual machines can be migrated between the hosts on the same VXLAN and can access the network normally after migration without modifying the configuration. TOR1#sho vxlan VXLAN Total Count: 2 VXLAN Capacity : 8000 VXLAN 10 Symmetric property : FALSE RouterInterface : - ExtendVLAN 10 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 2.2.2.2 1.1.1.1 dynamic VXLAN 20 Symmetric property : FALSE RouterInterface : - ExtendVLAN 20 VTEP Adjacency Count: 2 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 2.2.2.2 1.1.1.1 dynamic OverlayTunnel 6146 2.2.2.2 3.3.3.3 dynamic CORE#sho vxlan VXLAN Total Count: 3 VXLAN Capacity : 8000 VXLAN 10 Symmetric property : FALSE RouterInterface : OverlayRouter 10 (non-anycast) ExtendVLAN : - VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel6147 1.1.1.1 2.2.2.2 dynamic VXLAN 20 Symmetric property : FALSE RouterInterface :OverlayRouter20(non-anycast) ExtendVLAN : - VTEP Adjacency Count: 2 VTEP Adjacency List : Interface SourceIP Destination IP Type --- OverlayTunnel 6147 1.1.1.1 2.2.2.2 dynamic OverlayTunnel 6148 1.1.1.1 3.3.3.3 dynamic VXLAN 100 Symmetric property : FALSE RouterInterface :OverlayRouter100(non-anycast) ExtendVLAN :- VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel6148 1.1.1.1 3.3.3.3 dynamic 4.2.2 Configuring EVPN-based Multi-tenant Centralized All-active Anycast Gateway Scenario Figure 4-3 page_9_img_1_cc79be82.jpeg Configuration Steps Configure an IPv4 unicast routing protocol such as the OSPF protocol on CORE, TOR-1, and TOR-2 to ensure that unicast routes are reachable. Configure the BGP-EVPN routing protocol on CORE, TOR-1, and TOR-2 to establish BGP neighbor relationships between the three devices and to support the EVPN protocol family. Configure the EVI for BGP-EVPN on CORE, TOR-1, and TOR-2. For details, see BGP-EVPN Configuration Guide. Configure a VXLAN on the virtual server and designate the gateway address of the virtual machine. (Omitted). Associate the VTEP with the loopback interface on TOR-1, TOR-2, and CORE to establish tunnels. Note that the same loopback interface IP address needs to be configured on CORE1 and CORE2 as the VTEP IP address for tunnel establishment. After the loopback interface IP address is configured, no tunnel is established between CORE1 and CORE2. On TOR1, only one tunnel whose VTEP IP address is 1.1.1.1 and one tunnel whose VTEP IP address is 3.3.3.3 can be viewed. On TOR2, only one tunnel whose VTEP IP address is 1.1.1.1 and one tunnel whose VTEP IP address is 2.2.2.2 can be viewed. On CORE1, only one tunnel whose VTEP IP address is 2.2.2.2 and one tunnel whose VTEP IP address is 3.3.3.3 can be viewed. On CORE2, only one tunnel whose VTEP IP address is 2.2.2.2 and one tunnel whose VTEP IP address is 3.3.3.3 can be viewed. Create VXLAN instances on TOR-1, TOR-2, and CORE and associate the VXLAN instances with VLANs. Create overlay router interfaces and configure the VXLAN gateway IP address on CORE. Configure different VRF networks for different overlay router interfaces to determine their respective tenants. Associate VXLAN instances with overlay router interfaces to realize VXLAN routing. Note that the overlay router interface configurations on CORE1 and CORE2 must be the same. That is, the IP addresses and masks configured for the overlay router interfaces associated with the same VXLAN instance must be the same on CORE1 and CORE2 and belong to the same tenant (VRF). In addition, an anycast gateway must be configured for all overlay router interfaces. Configure the same anycast gateway MAC address on CORE1 and CORE2 to ensure that all VXLAN anycast gateways on CORE use the same MAC address. Enable the remote ARP packet learning function on CORE to generate VXLAN routing entries dynamically. (Optional) Configure ARP suppression on TOR-1 and TOR-2 to reduce the ARP packets entering the VXLAN. HOST Configure the IP address and gateway according to Figure 1-24 (the detailed configuration on the server is omitted herein). CORE The configuration of the OSPF and Ethernet interface is omitted herein. The following describes only the VXLAN configuration. Note: VXLAN configuration on CORE1 and CORE2 is the same. The following configuration applies to CORE1 and CORE2: Configure loopback 1 on both CORE1 and CORE2 and set the IP address to 1.1.1.2 and 1.1.1.3 for loopback 1. The two core switches establish a BGP neighbor relationship through loopback 1. Configure the L2VPN EVPN address family activation command. CORE# configure terminal Enter configuration commands, one per line. End with CNTL/Z. CORE(config)# interface Loopback 0 CORE(config-if- Loopback 0)# ip address 1.1.1.1/32 CORE(config-if- Loopback 0)# exit CORE(config)#int loopback 1 CORE(config-if-Loopback 1)# ip address 1.1.1.2/32 CORE(config-if- Loopback 1)# exit CORE(config)# vtep CORE(config-vtep)# source loopback 0 CORE(config-vtep)#remote arp learn enable CORE(config-vtep)# exit CORE(config)# fabric anycast-gateway-mac 0011.2233.2016 CORE(config)# ip vrf vrf-10 CORE(config-vrf)# rd 10:10 CORE(config-vrf)# route-target both 1000:1000 CORE(config-vrf)# exit CORE(config)# ip vrf vrf-20 CORE(config-vrf)# rd 20:20 CORE(config-vrf)# route-target both 2000:2000 CORE(config-vrf)# exit CORE(config)# int overlayrouter 10 CORE(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 CORE(config-if-OverlayRouter10)#ipaddress10.1.1.1/24 CORE(config-if-OverlayRouter 10)# anycast-gateway CORE(config-if-OverlayRouter 10)#exit CORE(config)# int overlayrouter 20 CORE(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 CORE(config-if-OverlayRouter 20)# ip address 10.1.2.1/24 CORE(config-if-OverlayRouter 20)# anycast-gateway CORE(config-if-OverlayRouter 20)# exit CORE(config)# vxlan 20 CORE(config)#int overlayrouter100 CORE(config-if-OverlayRouter 100)# ip vrf forwarding vrf-20 CORE(config-if-OverlayRouter 100)# ip address 10.1.3.1/24 CORE(config-if-OverlayRouter 100)# anycast-gateway CORE(config-if-OverlayRouter 100)# exit CORE(config)# vxlan 10 CORE(config-vxlan)# router-interface OverlayRouter 10 CORE(config-vxlan)#exit CORE(config)# vxlan 20 CORE(config-vxlan)# router-interface OverlayRouter 20 CORE(config-vxlan)#exit CORE(config)# vxlan 100 CORE(config-vxlan)# router-interface OverlayRouter 100 CORE(config-vxlan)#exit CORE(config)# router bgp 64512 CORE(config-router)# neighbor 1.1.1.3 remote-as 64512 CORE(config-router)# neighbor 1.1.1.3 update-source loopback 1 CORE(config-router)# neighbor 2.2.2.2 remote-as 64512 CORE(config-router)# neighbor 2.2.2.2 update-source loopback 1 CORE(config-router)# neighbor 3.3.3.3 remote-as 64512 CORE(config-router)# neighbor 3.3.3.3 update-source loopback 1 CORE(config-router)# address-family l2vpn evpn CORE(config-router)# neighbor 1.1.1.3 activate CORE(config-router)# neighbor 2.2.2.2 activate CORE(config-router)# neighbor 33.3.3.3 activate CORE(config-router-af)# exit CORE(config-router)# address-family ipv4 vrf vrf-10 CORE(config-router-af)# network 10.1.1.0 mask 255.255.255.0 CORE(config-router-af)# network 10.1.2.0 mask 255.255.255.0 CORE(config-router-af)# exit CORE(config-router)# address-family ipv4 vrf vrf-20 CORE(config-router-af)# network 10.1.3.0 mask 255.255.255.0 CORE(config-router-af)#exit CORE(config-router)# exit CORE(config)# evpn CORE(config- evpn)# vni 10 CORE(config-evpn-vni)# rd auto CORE(config-evpn-vni)# route-target both auto CORE(config-evpn-vni)# exit CORE(config-evpn)# vni 20 CORE(config-evpn-vni)# rd auto CORE(config-evpn-vni)# route-target both auto CORE(config-evpn-vni)# exit CORE(config-evpn)# vni 100 CORE(config-evpn-vni)# rd auto CORE(config-evpn-vni)# route-target both auto CORE(config-evpn-vni)# exit TOR1 TOR1# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR1(config)# interface Loopback 1 TOR1(config-if-Loopback1)#ipaddress2.2.2.2/32 TOR1(config-if- Loopback 1)# exit TOR1(config)# vtep TOR1(config-vtep)# source loopback 1 TOR1(config-vtep)# arp suppress enable TOR1(config-vtep)# exit TOR1(config)# vxlan 10 TOR1(config-vxlan)# extend-vlan 10 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# vxlan 20 TOR1(config-vxlan)# extend-vlan 20 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# router bgp 64512 TOR1(config-router)#neighbor1.1.1.2remote-as64512 TOR1 (config-router)# neighbor 1.1.1.2 update-source loopback 1 TOR1(config-router)#neighbor 1.1.1.3 remote-as 64512 TOR1 (config-router)# neighbor 1.1.1.3 update-source loopback 1 TOR1(config-router)#neighbor 3.3.3.3 remote-as 64512 TOR1 (config-router)# neighbor 3.3.3.3 update-source loopback 1 TOR1(config-router)# address-family l2vpn evpn TOR1(config-router-af)# neighbor 1.1.1.2 activate TOR1(config-router-af)# neighbor 1.1.1.3 activate TOR1(config-router-af)# neighbor 3.3.3.3 activate TOR1(config-router-af)# exit TOR1(config-router)# exit TOR1(config)# evpn TOR1(config-evpn)# vni 10 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 20 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR2 TOR2# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR2(config)# interface Loopback 1 TOR2(config-if-Loopback1)#ipaddress3.3.3.3/32 TOR2(config-if- Loopback 1)# exit TOR2(config)# vtep TOR2(config-vtep)# source loopback 1 TOR2(config-vtep)# arp suppress enable TOR2(config-vtep)# exit TOR2(config)# vxlan 100 TOR2(config-vxlan)# extend-vlan 100 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# vxlan 20 TOR2(config-vxlan)# extend-vlan 20 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# router bgp 64512 TOR2(config-router)#neighbor1.1.1.2remote-as64512 TOR2 (config-router)# neighbor 1.1.1.2 update-source loopback 1 TOR2(config-router)#neighbor 1.1.1.3 remote-as 64512 TOR2 (config-router)# neighbor 1.1.1.3 update-source loopback 1 TOR2(config-router)#neighbor 2.2.2.2 remote-as 64512 TOR2 (config-router)# neighbor 2.2.2.2 update-source loopback 1 TOR2(config-router)# address-family l2vpn evpn TOR2(config-router-af)# neighbor 1.1.1.2 activate TOR2(config-router-af)# neighbor 1.1.1.3 activate TOR2(config-router-af)# neighbor 2.2.2.2 activate TOR2(config-router-af)# exit TOR2(config-router)# exit TOR2(config)# evpn TOR2(config-evpn)# vni 20 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 100 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit Verification Verify that HOST-1, HOST-2, HOST-3, and HOST-4 can ping each other. Verify that HOST-5 and HOST-6 can ping each other. Verify that HOST-1, HOST-2, HOST-3, and HOST-4 cannot ping HOST-5 and HOST-6. Verifythat virtual machines can bemigrated between the hostsonthe sameVXLANand can accessthe network normally after migration without modifying the configuration. TOR1#sho vxlan VXLAN Total Count: 2 VXLAN Capacity : 8000 VXLAN 10 Symmetric property : FALSE RouterInterface : - ExtendVLAN 10 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 2.2.2.2 1.1.1.1 dynamic VXLAN 20 Symmetric property : FALSE RouterInterface : - ExtendVLAN 20 VTEP Adjacency Count: 2 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 2.2.2.2 1.1.1.1 dynamic OverlayTunnel 6146 2.2.2.2 3.3.3.3 dynamic CORE#sho vxlan VXLAN Total Count: 3 VXLAN Capacity : 8000 VXLAN 10 Symmetric property : FALSE RouterInterface : OverlayRouter 10 (anycast) ExtendVLAN 10 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel6147 1.1.1.1 2.2.2.2 dynamic VXLAN 20 Symmetric property : FALSE RouterInterface : OverlayRouter 20 (anycast) Extend VLAN 20 VTEP Adjacency Count: 2 VTEP Adjacency List : Interface SourceIP Destination IP Type --- OverlayTunnel 6147 1.1.1.1 2.2.2.2 dynamic OverlayTunnel 6148 1.1.1.1 3.3.3.3 dynamic VXLAN 100 Symmetric property : FALSE RouterInterface : OverlayRouter 100 (anycast) Extend VLAN 100 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type OverlayTunnel6148 1.1.1.1 3.3.3.3 dynamic 4.2.3 Configuring EVPN-based Multi-tenant Distributed Scenario (Enabling Anycast Gateway) Figure 4-4 page_7_img_1_74590c4f.jpeg Configuration Steps Configure an IPv4 unicastrouting protocol such as the OSPF protocol on CORE,TOR-1, and TOR-2 to ensure that unicast routes are reachable. ConfiguretheBGP-EVPNroutingprotocolonCORE,TOR-1,andTOR-2toestablishBGPneighborrelationships between the four devices and to support the EVPN protocol family. Configure the EVI for BGP-EVPN on TOR-1 and TOR-2. For details, see BGP-EVPN Configuration Guide. Configure a VXLAN on the virtual server and designate the gateway address of the virtual machine. Associate theVTEP with loopback interface onTOR-1 andTOR-2 to establish tunnels. Configure the anycast gateway MAC address onTOR-1 andTOR-2 to ensure that allVXLAN anycast gateways on the network use the same MAC address. CreateVXLAN instances onTOR-1 andTOR-2 and associatetheVXLAN instances withVLANs. Create overlay router interfaces on TOR-1 and TOR-2 and configure theVXLAN gateway IP address for the interfaces. ConfiguredifferentVRFnetworksfordifferentoverlayrouterinterfacestodeterminetheirrespectivetenants. ConfiguretheanycastgatewaytoensurethatallVXLANgatewaysonthenetworkusethesameIPaddressand MAC address. As the anycast gateway function is enabled, the overlay router interfaces associated with the same VXLAN onTOR-1 andTOR-2 must be configured withthe sameVXLAN gateway IPaddress. AssociateVXLANinstanceswithoverlay routerinterfacesonTOR-1andTOR-2torealizeVXLANrouting. (Optional)ConfigureARPsuppressiononTOR-1andTOR-2toreducetheARPpacketsenteringtheVXLAN. (Optional) Configure ARP proxy on the overlay router interfaces ofTOR-1 and TOR-2 so that theVXLAN gateway usesthegatewayMACaddress torespond asaproxy andVXLANnetwork trafficisforwardedonlyat L3. HOST ConfiguringtheIPaddressand gatewayaccording toFigure 1-25 (thedetailed configurationon the serveris omitted herein). CORE VXLAN may be not configured on the core switches. The configuration of the OSPF and BGP is omitted herein. TOR1 TOR1# configure terminal Enterconfigurationcommands, one per line. End withCNTL/Z. TOR1(config)# interface Loopback 1 TOR1(config-if-Loopback1)#ipaddress2.2.2.2/32 TOR1(config-if- Loopback 1)# exit TOR1(config)# vtep TOR1(config-vtep)# source loopback 1 TOR1(config-vtep)# arp suppress enable TOR1(config-vtep)# exit TOR1(config)# fabric anycast-gateway-mac 0011.2233.2016 TOR1(config)# ip vrf vrf-10 TOR1(config-vrf)# rd 10:10 TOR1(config-vrf)# route-target both 1000:1000 TOR1(config-vrf)# exit TOR1(config)# ip vrf vrf-20 TOR1(config-vrf)# rd 20:20 TOR1(config-vrf)# route-target both 2000:2000 TOR1(config-vrf)# exit TOR1(config)# int overlayrouter 10 TOR1(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR1(config-if-OverlayRouter 10)# anycast-gateway TOR1(config-if-OverlayRouter10)#route-in-vni //Optional.Itneedstobeusedincombinationwiththearpsuppress enable command. TOR1(config-if-OverlayRouter 10)# exit TOR1(config)# int overlayrouter 20 TOR1(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 20)# ip address 10.1.2.1/24 TOR1(config-if-OverlayRouter 20)# anycast-gateway TOR1(config-if-OverlayRouter20)#route-in-vni //Optional.Itneedstobeusedincombinationwiththearpsuppress enable command. TOR1(config-if-OverlayRouter 20)# exit TOR1(config)# int overlayrouter 100 TOR1(config-if-OverlayRouter 100)# ip vrf forwarding vrf-20 TOR1(config-if-OverlayRouter 100)# ip address 10.1.3.1/24 TOR1(config-if-OverlayRouter 100)# anycast-gateway TOR1(config-if-OverlayRouter100)#route-in-vni //Optional.Itneedstobeusedincombinationwiththearpsuppress enable command. TOR1(config-if-OverlayRouter 100)# exit TOR1(config)# vxlan 10 TOR1(config-vxlan)# extend-vlan 10 TOR1(config-vxlan)# router-interface OverlayRouter 10 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# vxlan 20 TOR1(config-vxlan)# extend-vlan 20 TOR1(config-vxlan)# router-interface OverlayRouter 20 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# vxlan 100 TOR1(config-vxlan)# extend-vlan 100 TOR1(config-vxlan)# router-interface OverlayRouter 100 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# router bgp 64512 TOR1(config-router)# neighbor 3.3.3.3 remote-as 64512 TOR1(config-router)# neighbor 3.3.3.3 update-source loopback 1 TOR1(config-router)# address-family l2vpn evpn TOR1(config-router-af)# neighbor 3.3.3.3 activate TOR1(config-router-af)# exit TOR1(config-router)# exit TOR1(config)# evpn TOR1(config-evpn)# vni 10 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 20 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 100 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR2 TOR2# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR2(config)# interface Loopback 1 TOR2(config-if-Loopback1)#ipaddress3.3.3.3/32 TOR2(config-if- Loopback 1)# exit TOR2(config)# vtep TOR2(config-vtep)# source loopback 1 TOR2(config-vtep)# arp suppress enable TOR2(config-vtep)# exit TOR1(config)# fabric anycast-gateway-mac 0011.2233.2016 TOR2(config)# ip vrf vrf-10 TOR2(config-vrf)# rd 10:10 TOR2(config-vrf)# route-target both 1000:1000 TOR2(config- vrf)# exit TOR2(config) # ip vrf vrf-20 TOR2(config- vrf)# rd 20:20 TOR2(config-vrf)# route-target both 2000:2000 TOR2(config-vrf)# exit TOR2(config)# int overlayrouter 10 TOR2(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR2(config-if-OverlayRouter 10)# anycast-gateway TOR2(config-if-OverlayRouter10)#route-in-vni //Optional.Itneedstobeusedincombinationwiththearp suppress enable command. TOR2(config-if-OverlayRouter 10)# exit TOR2(config)# intoverlayrouter 20 TOR2(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 20)# ip address 10.1.2.1/24 TOR2(config-if-OverlayRouter 20)# anycast-gateway TOR2(config-if-OverlayRouter20)#route-in-vni //Optional.Itneedstobeusedincombinationwiththearp suppress enable command. TOR2(config-if-OverlayRouter 20)# exit TOR2(config)# intoverlayrouter 100 TOR2(config-if-OverlayRouter 100)# ip vrf forwarding vrf-20 TOR2(config-if-OverlayRouter 100)# ip address 10.1.3.1/24 TOR2(config-if-OverlayRouter 100)# anycast-gateway TOR2(config-if-OverlayRouter100)#route-in-vni //Optional.Itneedstobeusedincombinationwiththe arpsuppress enable command. TOR2(config-if-OverlayRouter 100)# exit TOR2(config)# vxlan 10 TOR2(config-vxlan)# extend-vlan 10 TOR2(config-vxlan)# router-interface OverlayRouter 10 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# vxlan 20 TOR2(config-vxlan)# extend-vlan 20 TOR2(config-vxlan)# router-interface OverlayRouter 20 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# vxlan 100 TOR2(config-vxlan)# extend-vlan 100 TOR2(config-vxlan)# router-interface OverlayRouter 100 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan) # exit TOR2(config)# router bgp 64512 TOR2(config-router)# neighbor 2.2.2.2 remote-as 64512 TOR2(config-router)# neighbor 2.2.2.2 update-source loopback 1 TOR2(config-router)# address-family l2vpn evpn TOR2(config-router-af)# neighbor 2.2.2.2 activate TOR2(config-router-af)# exit TOR2(config-router)# exit TOR2(config)# evpn TOR2(config-evpn)# vni 10 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 20 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 100 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit Verification Verify that HOST-1, HOST-2, HOST-4, and HOST-5 can ping each other. Verify that HOST-3 and HOST-6 can ping each other. Verify that HOST-1, HOST-2, HOST-4, and HOST-5 cannot ping HOST-3 and HOST-6. Verifythatthevirtual machinescan bemigratedbetweenthe hostsonthesameVXLANand canaccessthe network normally after migration without modifying the configuration. TOR1#sho vxlan VXLAN Total Count: 3 VXLAN Capacity : 8000 VXLAN 10 Symmetric property : FALSE Router Interface : OverlayRouter 10 (anycast) Extend VLAN 10 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface Source IP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 2.2.2.2 3.3.3.3 dynamic VXLAN 20 Symmetric property : FALSE Router Interface : OverlayRouter 20 (anycast) Extend VLAN 20 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface Source IP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel6145 2.2.2.2 3.3.3.3 dynamic VXLAN 100 Symmetric property : FALSE RouterInterface : OverlayRouter 100 (anycast) Extend VLAN 100 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel6145 2.2.2.2 3.3.3.3 dynamic Common Errors Whensymmetricdeploymentisdisabled,allTORswitchesofthesameVRFnetworkmusthavealltheVXLAN gatewaysofthe VRF network configured on them. For example, VRF-10 includes VXLAN 10 and VXLAN 20, and therefore all gateways of VXLAN 10 and VXLAN 20 must be configured onTOR-1 and TOR-2 . Otherwise, VXLAN 10 and VXLAN 20 cannot communicate with each other Ifyou expect to deploy only required gateways instead ofdeploying allgatewayson all TOR switches, apply symmetric deployment. For details, see section“Configuring EVPN-based Multi-tenant Distributed Scenario (Symmetric Deployment)." MakesurethattheglobalanycastMACaddressisnotthesameasthatofanydeviceontheVXLAN. 4.2.4 Configuring EVPN-based Multi-tenant Distributed Scenario (Symmetric Deployment) Figure 4-5 image.png Configuration Steps Configure an IPv4 unicastrouting protocol such as the OSPF protocol on CORE,TOR-1, and TOR-2 to ensure that unicast routes are reachable. ConfiguretheBGP-EVPNroutingprotocolonCORE,TOR-1,andTOR-2toestablishBGPneighborrelationships between the four devices and to support the EVPN protocol family. Configure the EVI for BGP-EVPN on TOR-1 and TOR-2. For details, see BGP-EVPN Configuration Guide. Configure a VXLAN on the virtual server and designate the gateway address of the virtual machine. Associate theVTEP with loopback interface onTOR-1 andTOR-2 to establish tunnels. Configure the anycast gateway MAC address onTOR-1 andTOR-2 to ensure that allVXLAN anycast gateways on the network use the same MAC address. CreateVXLAN 10,VXLAN20, andVXLAN 100 onTOR-1 and associate them withVLANs. CreateVXLAN 10 andVXLAN 100 onTOR-2 and associate them withVLANs. Createoverlay routerinterfaces forVXLAN 10,VXLAN20,andVXLAN100 onTOR-1 andTOR-2 (TOR-2donothave VXLAN 20), and configure theVXLAN gateway IP address for them. Configure different VRF networks for different overlayrouterinterfacestodeterminetheirrespectivetenants.Configuretheanycastgatewaytoensurethatall VXLAN gateways on the network use the same IP address and MAC address. As the anycast gateway function is enabled, the overlay router interfaces associated with the same VXLAN on TOR-1 and TOR-2 must be configured with the sameVXLAN gateway IP address. CreateVXLAN 11 andVXLAN 101 onTOR-1 andTOR-2 and configure them as symmetricVXLANs to serve as L3 routingVXLAN of the correspondingVRF networks. L3 routes between allVXLANs of the same VRF network are advertisedviathesymmetricVXLANs.Inaddition,thesymmetricVXLANsarealsousedforL3routingand forwarding. Create overlay routerinterfaces forVXLAN 11 andVXLAN 101 onTOR-1 and TOR-2. Configure differentVRF networks for the overlay router interfaces. VXLAN 11 and VXLAN 101 serve as the symmetric VXLANs of the corresponding VRFnetworks. AssociateVXLANinstanceswithoverlay routerinterfacesonTOR-1andTOR-2torealizeVXLANrouting. (Optional)ConfigureARPsuppressiononTOR-1andTOR-2toreducetheARPpacketsenteringtheVXLAN. (Optional) Configure ARP proxy on the overlay router interfaces belonging to the L2-VNIs (VXLANs 10, 20, and 100) on TOR-1 and TOR-2 so that the traffic of hosts in the same VXLAN is forwarded at L3. This function needs to be enabled together with ARP suppression. The configuration command is route-in-vni.  (Optional) Configure no synchronization of EVPN entries in ARP proxy deployment scenario: On the L2-VNI VXLAN instances (VXLANs 10, 20, and 100) onTOR-1 andTOR-2, configure not to advertiseorreceive MAC-onlyEVPN type-2 routes.  Configure not to advertise EVPN MAC-only type-2 routes to reduce EVPN route synchronization between devices.The configuration command is evpn mac advertise disable.  Configure not to deliver MAC entries synchronized by EVPNs to reduce the occupancy of hardware entry resources.The configuration command is evpn mac inactive. The two commands above can be configured independently and they do not affect each other.  (Optional) Configure no synchronization of EVPN entries in scenarios where ARP proxy is not deployed: On the L2-VNI VXLAN instances (VXLANs 10, 20, and 100) on TOR-1 and TOR-2, configure not to advertise EVPN MAC-only type-2 routes and configure to extract MAC addresses from MAC-IP type-2 routes.  Configure not to advertise EVPN MAC-only type-2 routes to reduce EVPN route synchronization between devices.The configuration command is evpn mac advertise disable.  Configure to extract MAC addresses from MAC-IP type-2 routes so that the device can learn MAC entries even ifneighborsdonot adverseMAC-only type-2routes.Theconfigurationcommandisevpnarpmac-learning enable. Thetwocommandsabove mustbe usedincombination.Otherwise, thedevicecannot learnMAC entriesfrom neighbors and L2 forwarding cannot implemented in VXLANs. HOST ConfiguringtheIPaddressand gatewayaccording toFigure 1-26 (thedetailed configurationon the serveris omitted herein). CORE VXLAN may be not configured on the core switches. The configuration of the OSPF and BGP is omitted herein. TOR1 TOR1# configure terminal Enterconfigurationcommands,oneperline. TOR1 TOR1 TOR1# configure terminal Enterconfigurationcommands, one per line. End withCNTL/Z. TOR1(config)# interface Loopback 1 TOR1(config-if-Loopback1)#ipaddress2.2.2.2/32 TOR1(config-if- Loopback 1)# exit TOR1(config)# vtep TOR1(config-vtep)# source loopback 1 TOR1(config-vtep)# arp suppress enable TOR1(config-vtep)# exit TOR1(config)# fabric anycast-gateway-mac 0011.2233.2016 TOR1(config)# ip vrf vrf-10 TOR1(config-vrf)# rd 10:10 TOR1(config-vrf)# route-target both 1000:1000 TOR1(config-vrf)# exit TOR1(config)# ip vrf vrf-20 TOR1(config-vrf)# rd 20:20 TOR1(config-vrf)# route-target both 2000:2000 TOR1(config-vrf)# exit TOR1(config)# int overlayrouter 10 TOR1(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR1(config-if-OverlayRouter 10)# anycast-gateway TOR1(config-if-OverlayRouter10)#route-in-vni //Optional.ItisusedtoenableARPproxyandneeds tobeusedin combination with the arp suppress enable command. TOR1(config-if-OverlayRouter 10)# exit TOR1(config)# intoverlayrouter 20 TOR1(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 20)# ip address 10.1.2.1/24 TOR1(config-if-OverlayRouter 20)# anycast-gateway TOR1(config-if-OverlayRouter20)#route-in-vni //Optional.ItisusedtoenableARPproxyandneeds tobeusedin combination with the arp suppress enable command. TOR1(config-if-OverlayRouter 20)# exit TOR1(config)# int overlayrouter 11 TOR1(config-if-OverlayRouter 11)# anycast-gateway TOR1(config-if-OverlayRouter 11)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 11)# exit TOR1(config)# int overlayrouter 100 TOR1(config-if-OverlayRouter 100)# ip vrf forwarding vrf-20 TOR1(config-if-OverlayRouter 100)# ip address 10.1.3.1/24 TOR1(config-if-OverlayRouter 100)# anycast-gateway TOR1(config-if-OverlayRouter 100)# route-in-vni TOR1(config-if-OverlayRouter 100)# exit TOR1(config)# int overlayrouter 101 TOR1(config-if-OverlayRouter 101)# ip vrf forwarding vrf-20 TOR1(config-if-OverlayRouter 101)# exit TOR1(config)# vxlan 10 TOR1(config-vxlan)# extend-vlan 10 TOR1(config-vxlan)# router-interface OverlayRouter 10 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)#evpn mac advertise disable TOR1(config-vxlan)#evpn mac inactive TOR1(config-vxlan)#evpn arp mac-learning enable TOR1(config-vxlan)# exit TOR1(config)# vxlan 20 TOR1(config-vxlan)# extend-vlan 20 TOR1(config-vxlan)# router-interface OverlayRouter 20 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)#evpn mac advertise disable //Optional. TOR1(config-vxlan)#evpn mac inactive //Optional. In ARP proxy deployment scenarios, it is used to reduce the synchronization of EVPN entries. TOR1(config-vxlan)#evpnarpmac-learningenable //Optional. InscenariosinwhichARPproxy isnot deployed, itis used to reduce the synchronization of EVPN entries. TOR1(config-vxl an)# exit TOR1(config)# vxlan 11 TOR1(config-vxlan) # symmetric TOR1(config-vxlan)#router-interface OverlayRouter 11 TOR1(config-vxlan)# exit TOR1(config)# vxlan 100 TOR1(config-vxlan)# extend-vlan 100 TOR1(config-vxlan)# router-interface OverlayRouter 100 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)#evpn mac advertise disable //Optional. TOR1(config-vxlan)#evpn mac inactive //Optional. In ARP proxy deployment scenarios, it is used to reduce the synchronization of EVPN entries. TOR1(config-vxlan)#evpnarpmac-learningenable //Optional. InscenariosinwhichARPproxy isnotdeployed, itis used to reduce the synchronization of EVPN entries. TOR1(config-vxlan)# exit TOR1(config)# vxlan 101 TOR1(config-vxlan)# symmetric TOR1(config-vxlan)# router-interface OverlayRouter 101 TOR1(config-vxlan)# exit TOR1(config)# router bgp 64512 TOR1(config-router)# neighbor 3.3.3.3 remote-as 64512 TOR1(config-router)# neighbor 3.3.3.3 update-source loopback 1 TOR1(config-router)# address-family l2vpn evpn TOR1(config-router-af)# neighbor 3.3.3.3 activate TOR1(config-router-af)# exit TOR1(config-router)# exit TOR1(config)# evpn TOR1(config-evpn)# vni 10 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 11 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 100 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 101 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR2 TOR2# configure terminal Enterconfigurationcommands, one per line. End withCNTL/Z. TOR2(config)# interface Loopback 1 TOR2(config-if- Loopback 1)# ip address 3.3.3.3/32 TOR2(config-if- Loopback 1)# exit TOR2(config)# vtep TOR2(config-vtep)# source loopback 1 TOR2(config-vtep)# arp suppress enable TOR2(config-vtep)# exit TOR2(config)# fabric anycast-gateway-mac 0011.2233.2016 TOR2(config)# ip vrf vrf-10 TOR2(config-vrf)# rd 10:10 TOR2(config-vrf)# route-target both 1000:1000 TOR2(config-vrf)# exit TOR2config) # ip vrf vrf-20 TOR2(config- vrf)# rd 20:20 TOR2(config-vrf)# route-target both 2000:2000 TOR2(config-vrf)# exit TOR2(config)# int overlayrouter 10 TOR2(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR2(config-if-OverlayRouter 10)# anycast-gateway TOR2(config-if-OverlayRouter 10)# route-in-vni //Optional. It is used to enable ARP proxy and needs to be used in combination with the arp suppress enable command. TOR2(config-if-OverlayRouter 10)# exit TOR2(config)# int overlayrouter 11 TOR2(config-if-OverlayRouter 11)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 11)# exit TOR2(config)# int overlayrouter 100 TOR2(config-if-OverlayRouter 100)# ip vrf forwarding vrf-20 TOR2(config-if-OverlayRouter 100)# ip address 10.1.3.1/24 TOR2(config-if-OverlayRouter 100)# anycast-gateway TOR2(config-if-OverlayRouter 100)# route-in-vni TOR2(config-if-OverlayRou ter 100)# exit TOR2(config)# int overlayrouter 101 TOR2(config-if-OverlayRouter 101)# ip vrf forwarding vrf-20 TOR2(config-if-OverlayRouter 101)# exit TOR2(config)# vxlan 10 TOR2(config-vxlan)# extend-vlan 10 TOR2(config-vxlan)# router-interface OverlayRouter 10 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)#evpn mac advertise disable //Optional. TOR2(config-vxlan)#evpn mac inactive //Optional. In ARP proxy deployment scenarios, it is used to reduce the synchronization of EVPN entries. TOR2(config-vxlan)#evpnarpmac-learningenable //Optional.InscenariosinwhichARPproxyisnot deployed,itis used to reduce the synchronization of EVPN entries. TOR2(config-vxlan)# exit TOR2(config)# vxlan 11 TOR2(config-vxlan)# symmetric TOR2(config-vxlan)#router-interface OverlayRouter 11 TOR2(config-vxlan)# exit TOR2(config)# vxlan 100 TOR2(config-vxlan)# extend-vlan 100 TOR2(config-vxlan)# router-interface OverlayRouter 100 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)#evpn mac advertise disable //Optional. TOR2(config-vxlan)#evpn mac inactive //Optional. In ARP proxy deployment scenarios, it is used to reduce the synchronization of EVPN entries. TOR2(config-vxlan)#evpnarpmac-learningenable //Optional. InscenariosinwhichARPproxy isnotdeployed, itis used to reduce the synchronization of EVPN entries. TOR2(config-vxlan)# exit TOR2(config)# int overlayrouter 101 TOR2(config-vxlan)# symmetric TOR2(config-vxlan)# router-interface OverlayRouter 101 TOR2(config-vxlan)# exit TOR2(config)# router bgp 64512 TOR2(config-router)# neighbor 2.2.2.2 remote-as 64512 TOR2(config-router)# neighbor 2.2.2.2 update-source loopback 1 TOR2(config-router)# address-family l2vpn evpn TOR2(config-router-af)# neighbor 2.2.2.2 activate TOR2(config-router-af)# exit TOR2(config-router)# exit TOR2(config)# evpn TOR2(config-evpn)# vni 10 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 11 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 100 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 101 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit Verification VerifythatHOST-1,HOST-2,andHOST-4canpingeachother. Verifythat HOST-3and HOST-6can ping each other. Verify that HOST-1, HOST-2, and HOST-4 cannot ping HOST-3 and HOST-6. Verifythat the virtual machines can be migratedbetween thehosts on the sameVXLANand can access the network normally after migration without modifying the configuration. TOR1#sho vxlan VXLAN Total Count: 5 VXLAN Capacity : 8000 VXLAN 10 Symmetric property : FALSE RouterInterface : OverlayRouter 10 (anycast) ExtendVLAN 10 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 2.2.2.2 3.3.3.3 dynamic VXLAN 11 Symmetric property : TRUE RouterInterface : OverlayRouter 11 (anycast) ExtendVLAN : - VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 2.2.2.2 3.3.3.3 dynamic VXLAN 20 Symmetric property : FALSE RouterInterface : OverlayRouter 20 (anycast) ExtendVLAN 20 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 2.2.2.2 3.3.3.3 dynamic VXLAN 100 Symmetric property : FALSE RouterInterface : OverlayRouter 100 (anycast) ExtendVLAN 100 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel6145 2.2.2.2 3.3.3.3 dynamic VXLAN 101 Symmetric property : TRUE RouterInterface : OverlayRouter 101 (anycast) ExtendVLAN :- VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel6145 2.2.2.2 3.3.3.3 dynamic OverlayTunnel6145 2.2.2.2 3.3.3.3 dynamic Common Errors Make sure that the global anycast MAC address is not the same as that of any device on theVXLAN. 4.2.5 Configuring EVPN-based Single-tenant VXLAN Routing Scenario Figure 4-6 page_3_img_2_2eab6bc1.jpeg Configuration Steps Configure an IPv4 unicast routing protocol such as the OSPF protocol on Border-1, Border-2,TOR-1, and TOR-2 to ensure that unicast routes are reachable. Configure theBGP-EVPN routing protocol onBorder-1,Border-2,TOR-1, andTOR-2 to establishBGP neighbor relationships between the devices (except between Border-1 and Border-2) and to support the EVPN protocol family. Configure the EVI for BGP-EVPN on TOR-1 and TOR-2. For details, see BGP-EVPN Configuration Guide. Configure a VXLAN on the virtual server and designate the gateway address of the virtual machine. AssociatetheVTEPwiththeloopbackinterfaceonTOR-1,TOR-2,Border-1,andBorder-2toestablishtunnels. Configure the anycast gateway MAC address onTOR-1 andTOR-2 to ensure that allVXLAN anycast gateways on the network use the same MAC address. CreateVXLAN10andVXLAN20onTOR-1andassociatethemwithVLANs. CreateVXLAN10andVXLAN20onTOR-2andassociatethemwithVLANs. CreateVXLAN90onBorder-1and associateVXLAN90withaVLAN. CreateVXLAN90onBorder-2and associateVXLAN90withaVLAN. CreateoverlayrouterinterfacesforVXLAN10andVXLAN20onTOR-1andTOR-2,andconfiguretheVXLAN gatewayIPaddressforthem.ConfigurethesameVRFnetworkfortheoverlayrouterinterfacestodeterminetheir respective tenants. Configure the anycast gateway to ensure that all VXLAN gateways on the network use the same IP address and MAC address. As the anycast gateway function is enabled, the overlay router interfaces associated withthesameVXLANonTOR-1andTOR-2mustbeconfiguredwiththesameVXLANgatewayIPaddress. CreateVXLAN100onTOR-1,TOR-2,Border-1,andBorder-2.ConfigureVXLAN100asasymmetricVXLANtoserveas theL3routingVXLANofthecorrespondingVRFnetwork.L3routesbetweenallVXLANsofthesameVRFnetwork are advertised via the symmetric VXLAN. In addition, the symmetric VXLAN is also used for L3 routing and forwarding. Create overlay router interfaces forVXLAN 100 onTOR-1 and TOR-2 and configure the sameVRF network for the overlay routerinterfaces.VXLAN 100 serves as the symmetricVXLAN of theVRF network. Create overlay router interfaces for VXLAN 100 on Border-1 and Border-2, and configure the sameVRF network for theoverlayrouterinterfaces,sothatVXLAN100servesasthesymmetricVXLANoftheVRFnetwork.Configure VXLANgateway IPaddresses forBorder-1andBorder-2(differentIPaddresses fordifferentdevices). Create overlay router interfaces for VXLAN 90 on Border-1 and Border-2. Configure the same VRF network for the overlay router interfaces and configure theVXLAN gateway IP address. AssociateVXLANinstances withoverlay routerinterfaces onTOR-1,TOR-2,Border-1,andBorder-2torealizeVXLAN routing. (Optional)ConfigureARPsuppressiononTOR-1andTOR-2toreducetheARPpacketsenteringtheVXLAN. HOST ConfiguringtheIPaddressand gatewayaccording toFigure 1-27 (thedetailed configurationon the serveris omitted herein). TOR1 TOR1# configure terminal Enterconfigurationcommands, one per line. End withCNTL/Z. TOR1(config)# interface Loopback 1 TOR1(config-if-Loopback1)# ipaddress 1.1.1.1/32 TOR1(config-if- Loopback 1)# exit TOR1(config)# vtep TOR1(config-vtep)# source loopback 1 TOR1(config-vtep)# arp suppress enable TOR1(config-vtep)# exit TOR1(config)# fabric anycast-gateway-mac 0011.2233.2016 TOR1(config)# ip vrf vrf-10 TOR1(config-vrf)# rd 10:10 TOR1(config-vrf)# route-target export 1000:1000 TOR1(config-vrf)# exit TOR1(config)# int overlayrouter 10 TOR1(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR1(config-if-OverlayRouter 10)# anycast-gateway TOR1(config-if-OverlayRo uter 10)# exit TOR1(config)# int overlayrouter 20 TOR1(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 20)# ip address 20.1.1.1/24 TOR1(config-if-OverlayRouter 20)# anycast-gateway TOR1(config-if-OverlayRo uter 20)# exit TOR1(config)# int overlayrouter 100 TOR1(config-if-OverlayRouter 100)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 100)# ip address 100.1.4.1/24 TOR1(config-if-OverlayRouter 100)#exit TOR1(config)# vxlan 10 TOR1(config-vxlan)# extend-vlan 10 TOR1(config-vxlan)# router-interface OverlayRouter 10 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# vxlan 20 TOR1(config-vxlan)# extend-vlan 20 TOR1(config-vxlan)# router-interface OverlayRouter 20 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxl an)# exit TOR1(config)# vxlan 100 TOR1(config-vxlan) # symmetric TOR1(config-vxlan)# router-interface OverlayRouter 100 TOR1(config-vxlan)# exit TOR1(config)# router bgp 64512 TOR1(config-router)# neighbor 2.2.2.2 remote-as 64512 TOR1(config-router)# neighbor 2.2.2.2 update-source loopback 1 TOR1(config-router)# neighbor 3.3.3.3 remote-as 64512 TOR1(config-router)# neighbor 3.3.3.3 update-source loopback 1 TOR1(config-router)# neighbor 4.4.4.4 remote-as 64512 TOR1(config-router)# neighbor 4.4.4.4 update-source loopback 1 TOR1(config-router)# address-family l2vpn evpn TOR1(config-router-af)# neighbor 2.2.2.2 activate TOR1(config-router-af)# neighbor 3.3.3.3 activate TOR1(config-router-af)# neighbor 4.4.4.4 activate TOR1(config-router-af)#advertise ipv4unicast TOR1(config-router-af)# exit TOR1(config-router)# address-family ipv4 vrf vrf-10 TOR1(config-router-af)# redistribute connected TOR1(config-router-af)# exit TOR1(config-r outer)# exit TOR1(config) # evpn TOR1(config- evpn)# vni 10 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 20 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 100 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# route-target import 1000:1000 TOR1(config-evpn-vni)# exit TOR2 TOR2# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR2(config)# interface Loopback 1 TOR2(config-if-Loopback1)#ipaddress2.2.2.2/32 TOR2(config-if- Loopback 1)# exit TOR2(config)# vtep TOR2(config-vtep)# source loopback 1 TOR2(config-vtep)# arp suppress enable TOR2(config-vtep)# exit TOR2(config)# fabric anycast-gateway-mac 0011.2233.2016 TOR2(config)# ip vrf vrf-10 TOR2(config-vrf)# rd 10:10 TOR2(config-vrf)# route-target export 1000:1000 TOR2(config-vrf)# exit TOR2(config)# int overlayrouter 10 TOR2(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR2(config-if-OverlayRouter 10)# anycast-gateway TOR2(config-if-OverlayRouter 10)# exit TOR2(config)# int overlayrouter 20 TOR2(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 20)# ip address 20.1.1.1/24 TOR2(config-if-OverlayRouter 20)# anycast-gateway TOR2(config-if-OverlayRouter 20)# exit TOR2(config)# int overlayrouter 100 TOR2(config-if-OverlayRouter 100)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 100)# ip address 100.1.3.1/24 TOR2(config-if-OverlayRouter 100)#exit TOR2(config)# vxlan 10 TOR2(config-vxlan)# extend-vlan 10 TOR2(config-vxlan)# router-interface OverlayRouter 10 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# vxlan 20 TOR2(config-vxlan)# extend-vlan 20 TOR2(config-vxlan)# router-interface OverlayRouter 20 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# vxlan 100 TOR2(config-vxlan)# symmetric TOR2(config-vxlan)# router-interface OverlayRouter 100 TOR2(config-vxlan)# exit TOR2(config)# router bgp 64512 TOR2(config-router)# neighbor 1.1.1.1 remote-as 64512 TOR2(config-router)# neighbor 1.1.1.1 update-source loopback 1 TOR2(config-router)# neighbor 3.3.3.3 remote-as 64512 TOR2(config-router)# neighbor 3.3.3.3 update-source loopback 1 TOR2(config-router)# neighbor 4.4.4.4 remote-as 64512 TOR2(config-router)# neighbor 4.4.4.4 update-source loopback 1 TOR2(config-router)# address-family l2vpn evpn TOR2(config-router-af)# neighbor 1.1.1.1 activate TOR2(config-router-af)# neighbor 3.3.3.3 activate TOR2(config-router-af)# neighbor 4.4.4.4 activate TOR2(config-router-af)#advertiseipv4unicast TOR2(config-router-af)# exit TOR2(config-router)# address-family ipv4 vrf vrf-10 TOR2(config-router-af)# redistribute connected TOR2(config-router-af)# exit TOR2(config-router)# exit TOR2(config)# evpn TOR2(config-evpn)# vni 10 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 20 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 100 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# route-target import 1000:1000 TOR2(config-evpn-vni)# exit Border 1 Border1 Border1# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. Border1(config)# interface Loopback 1 Border1(config-if- Loopback 1)# ip address 3.3.3.3/32 Border1(config-if- Loopback 1)# exit Border1(config)# vtep Border1(config-vtep)# source loopback 1 Border1(config-vtep)# arp suppress enable Border1(config-vtep)# exit Border1(config)# fabric anycast-gateway-mac 0011.2233.2016 Border1(config)# ip vrf vrf-10 Border1(config-vrf)# rd10:10 Border1(config-vrf)# route-target export 1000:1000 Border1(config-vrf)# exit Border1(config)# int overlayrouter 90 Border1(config-if-OverlayRouter 90)# ip vrf forwarding vrf-10 Border1(config-if-OverlayRouter 90)# ip address 90.1.1.1/24 Border1(config-if-OverlayRouter 90)# anycast-gateway Border1(config-if-OverlayRouter 90)# exit Border1(config)# vxlan 90 Border1(config-vxlan)# extend-vlan 90 Border1(config-vxlan)# router-interface OverlayRouter 90 Border1(config-vxlan)# arp suppress enable Border1(config-vxlan)# exit Border1(config)# int overlayrouter 100 Border1(config-if-OverlayRouter 100)# ip vrf forwarding vrf-10 Border1(config-if-OverlayRouter 100)# ip address 100.1.1.1/24 Border1(config-if-OverlayRouter 100)#exit Border1(config)# vxlan 100 Border1(config-vxlan) # symmetric Border1(config-vxlan)# router-interface OverlayRouter 100 Border1(config-vxlan)# exit Border1(config)# router bgp 64512 Border1(config-router)# neighbor 1.1.1.1 remote-as 64512 Border1(config-router)# neighbor 1.1.1.1 update-source loopback 1 Border1(config-router)# neighbor 2.2.2.2 remote-as 64512 Border1(config-router)# neighbor 2.2.2.2 update-source loopback 1 Border1(config-router)# neighbor 4.4.4.4 remote-as 64512 Border1(config-router)# neighbor 4.4.4.4 update-source loopback 1 Border1(config-router)# address-family l2vpn evpn Border1(config-router-af)# neighbor 1.1.1.1 activate Border1(config-router-af)# neighbor 2.2.2.2 activate Border1(config-router-af)# neighbor 4.4.4.4 activate Border1(config-router-af)# advertise ipv4 unicast Border1(config-router-af)# exit Border1(config-router)# address-family ipv4 vrf vrf-10 Border1(config-router-af)# redistribute static Border1(config-router-af)# exit Border1(config-router)# exit Border1(config)# evpn Border1(config-evpn)# vni 100 Border1(config-evpn-vni)# rd auto Border1(config-evpn-vni)# route-target both auto Border1(config-evpn-vni)# route-target import 1000:1000 Border1(config-evpn-vni)# exit Border 2 Border2 Border2# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. Border2(config)# interface Loopback 1 Border2(config-if- Loopback 1)# ip address 4.4.4.4/32 Border2(config-if- Loopback 1)# exit Border2(config)# vtep Border2(config-vtep)# source loopback 1 Border2(config-vtep)# arp suppress enable Border2(config-vtep)# exit Border2(config)# fabric anycast-gateway-mac 0011.2233.2016 Border2(config)# ip vrf vrf-10 Border2(config-vrf)# rd10:10 Border2(config-vrf)# route-target export 1000:1000 Border2(config-vrf)# exit Border2(config)# int overlayrouter 90 Border2(config-if-OverlayRouter 90)# ip vrf forwarding vrf-10 Border2(config-if-OverlayRouter 90)# ip address 90.1.2.1/24 Border2(config-if-OverlayRouter 90)# anycast-gateway Border2(config-if-OverlayRouter 90)# exit Border2(config-vxlan)# arp suppress enable Border2(config-vxlan)# exit Border2(config)# int overlayrouter 100 Border2(config-if-OverlayRouter 100)# ip vrf forwarding vrf-10 Border2(config-if-OverlayRouter 100)# ipaddress 100.1.2.1/24 Border2(config-if-OverlayRouter 100)#exit Border2(config)# vxlan 90 Border2(config-vxlan)# extend-vlan 90 Border2(config-vxlan)# router-interface OverlayRouter 90 Border2(config-vxlan)# arp suppress enable Border2(config-vxlan)# exit Border2(config)# vxlan 100 Border2(config-vxlan)# symmetric Border2(config-vxlan)# router-interface OverlayRouter 100 Border2(config-vxlan)# exit Border2(config)# router bgp 64512 Border2(config-router)# neighbor 1.1.1.1 remote-as 64512 Border2(config-router)# neighbor 1.1.1.1 update-source loopback 1 Border2(config-router)# neighbor 2.2.2.2 remote-as 64512 Border2(config-router)# neighbor 2.2.2.2 update-source loopback 1 Border2(config-router)# neighbor 3.3.3.3 remote-as 64512 Border2(config-router)# neighbor 3.3.3.3 update-source loopback 1 Border2(config-router)# address-family l2vpn evpn Border2(config-router-af)# neighbor 1.1.1.1 activate Border2(config-router-af)# neighbor 2.2.2.2 activate Border2(config-router-af)# neighbor 3.3.3.3 activate Border2(config-router-af)# advertise ipv4 unicast Border2(config-router-af)# exit Border2(config-router)# address-family ipv4 vrf vrf-10 Border2(config-router-af)# redistribute static Border2(config-router-af)# exit Border2(config-router)# exit Border2(config)# evpn Border2(config-evpn)#vni 100 Border2(config-evpn-vni)# rd auto Border2(config-evpn-vni)# route-target both auto Border2(config-evpn-vni)# route-target import 1000:1000 Border2(config-evpn-vni)# exit Verification Verify that HOST-1, HOST-2, HOST-3, and HOST-4 can ping each other. Verify that the virtual machines can be migrated between the HOSTs on the same VXLAN and can access the network normally after migration without modifying the configuration. Border1# sh vxlan VXLAN Total Count: 3 VXLAN Capacity : 8000 VXLAN 90 Symmetric property : FALSE RouterInterface : overlayrouter 90 (anycast) ExtendVLAN 90 VTEP Adjacency Count: 1 Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6146 3.3.3.3 2.2.2.2 dynamic VXLAN 100 Symmetric property : TRUE RouterInterface : overlayrouter 100 (non-anycast) ExtendVLAN : - VTEP Adjacency Count: 1 Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6146 3.3.3.3 2.2.2.2 dynamic 4.2.6 Configuring EVPN-based Multi-tenant VXLAN Routing Scenario Figure 4 - 7 page_3_img_1_67916478.jpeg Configuration Steps Configure an IPv4 unicast routing protocol such as the OSPF protocol on Border-1, Border-2,TOR-1, and TOR-2 to ensure that unicast routes are reachable. Configure theBGP-EVPN routing protocol onBorder-1,Border-2,TOR-1, andTOR-2 to establishBGP neighbor relationships between the devices (except between Border-1 and Border-2) and to support the EVPN protocol family. Configure the EVI for BGP-EVPN on TOR-1 and TOR-2. For details, see BGP-EVPN Configuration Guide. Configure a VXLAN on the virtual server and designate the gateway address of the virtual machine. AssociatetheVTEPwiththeloopbackinterfaceonTOR-1,TOR-2,Border-1,andBorder-2toestablishtunnels. Configure the anycast gateway MAC address onTOR-1 andTOR-2 to ensure that allVXLAN anycast gateways on the network use the same MAC address. CreateVXLAN10,VXLAN20,andVXLAN30onTOR-1andassociatethemwithVLANs. CreateVXLAN10,VXLAN20,andVXLAN30onTOR-2andassociatethemwithVLANs. CreateVXLAN90onBorder-1and associateVXLAN90withaVLAN. CreateVXLAN90onBorder-2and associateVXLAN90withaVLAN. Createoverlay routerinterfaces forVXLAN 10,VXLAN 20, andVXLAN 30 onTOR-1 andTOR-2 and configurethe VXLAN gateway IP address for them. Configure different VRF networks for different overlay router interfaces to determinetheirrespectivetenants.ConfiguretheanycastgatewaytoensurethatallVXLANgatewaysonthe network use the same IP address and MAC address. As the anycast gateway function is enabled, the overlay router interfaces associated with the same VXLAN on TOR-1 andTOR-2 must be configured with the sameVXLAN gateway IP address. Create overlay routerinterfaces forVXLAN 100 andVXLAN 200 onTOR-1 andTOR-2 and configure differentVRF networksfortheoverlayrouterinterfaces.VXLAN100andVXLAN200serveasthesymmetricVXLANsofthe corresponding VRFnetworks. Create overlay router interfaces forVXLAN 100 and VXLAN 200 on Border-1 and Border-2. Configure differentVRF networks for the overlay router interfaces so that VXLAN 100 and VXLAN 200 serve as the symmetricVXLANs of the correspondingVRFnetworks.ConfigureVXLANgatewayIPaddressesforBorder-1andBorder-2(differentIP addresses for different devices). Create overlay router interfaces for VXLAN 90 on Border-1 and Border-2. Configure different VRF networks for the OverlayRouter interfaces and configure theVXLAN gateway IP address. AssociateVXLANinstances withoverlay routerinterfaces onTOR-1,TOR-2,Border-1,andBorder-2torealizeVXLAN routing. (Optional)ConfigureARPsuppressiononTOR-1andTOR-2toreducetheARPpacketsenteringtheVXLAN. HOST ConfiguringtheIPaddressand gatewayaccordingtoFigure 1-28 (the detailedconfiguration on theserverisomitted herein). TOR1 TOR1# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR1(config)# interface Loopback 1 TOR1(config-if-Loopback1)#ipaddress1.1.1.1/32 TOR1(config-if- Loopback 1)# exit TOR1(config)# vtep TOR1(config-vtep)# source loopback 1 TOR1(config-vtep)# arp suppress enable TOR1(config)# fabric anycast-gateway-mac 0011.2233.2016 TOR1(config-vtep)# exit TOR1(config)# ip vrf vrf-10 TOR1(config-vrf)# rd 10:10 TOR1(config-vrf)# route-target export 1000:1000 TOR1(config-vrf)# exit TOR1(config)# ip vrf vrf-20 TOR1(config-vrf)# rd 20:20 TOR1(config-vrf)# route-target export 2000:2000 TOR1(config-vrf)# exit TOR1(config)# int overlayrouter 10 TOR1(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR1(config-if-OverlayRouter 10)# anycast-gateway TOR1(config-if-OverlayRouter 10)# exit TOR1(config-vxlan)# exit TOR1(config)# int overlayrouter 20 TOR1(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 20)# ip address 20.1.1.1/24 TOR1(config-if-OverlayRouter 20)# anycast-gateway TOR1(config-if-OverlayRouter 20)# exit TOR1(config)# int overlayrouter 30 TOR1(config-if-OverlayRouter 30)# ip vrf forwarding vrf-20 TOR1(config-if-OverlayRouter 30)# ip address 30.1.1.1/24 TOR1(config-if-OverlayRouter 30)# anycast-gateway TOR1(config-if-OverlayRo uter 30)# exit TOR1(config)# int overlayrouter 100 TOR1(config-if-OverlayRouter 100)# ip vrf forwarding vrf-10 TOR1(config-if-OverlayRouter 100)# ip address 100.1.4.1/24 TOR1(config-if-OverlayRouter 100)#exit TOR1(config)# int overlayrouter 200 TOR1(config-if-OverlayRouter 200)# ip vrf forwarding vrf-20 TOR1(config-if-OverlayRouter 200)# ip address 200.1.4.1/24 TOR1(config-if-OverlayRouter 200)#exit TOR1(config)# vxlan 10 TOR1(config-vxlan)# extend-vlan 10 TOR1(config-vxlan)# router-interface OverlayRouter 10 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# vxlan 20 TOR1(config-vxlan)# extend-vlan 20 TOR1(config-vxlan)# router-interface OverlayRouter 20 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxl an)# exit TOR1(config)# vxlan 100 TOR1(config-vxlan) # symmetric TOR1(config-vxlan)# router-interface OverlayRouter 100 TOR1(config)# vxlan 30 TOR1(config-vxlan)# extend-vlan 30 TOR1(config-vxlan)# router-interface OverlayRouter 30 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxl an)# exit TOR1(config)# vxlan 200 TOR1(config-vxlan) # symmetric TOR1(config-vxlan)# router-interface OverlayRouter 200 TOR1(config-vxlan)# exit TOR1(config)# router bgp 64512 TOR1(config-router)# neighbor 2.2.2.2 remote-as 64512 TOR1(config-router)# neighbor 2.2.2.2 update-source loopback 1 TOR1(config-router)# neighbor 3.3.3.3 remote-as 64512 TOR1(config-router)# neighbor 3.3.3.3 update-source loopback 1 TOR1(config-router)# neighbor 4.4.4.4 remote-as 64512 TOR1(config-router)# neighbor 4.4.4.4 update-source loopback 1 TOR1(config-router)# address-family l2vpn evpn TOR1(config-router-af)# neighbor 2.2.2.2 activate TOR1(config-router-af)# neighbor 3.3.3.3 activate TOR1(config-router-af)# neighbor 4.4.4.4 activate TOR1(config-router-af)#advertiseipv4unicast TOR1(config-router-af)# exit TOR1(config-router)# address-family ipv4 vrf vrf-10 TOR1(config-router-af)# redistribute connected TOR1(config-router-af)# exit TOR1(config-router)# address-family ipv4 vrf vrf-20 TOR1(config-router-af)# redistribute connected TOR1(config-router-af)# exit TOR1(config-router)# exit TOR1(config)# evpn TOR1(config-evpn)# vni 10 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 20 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 30 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 100 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# route-target import 1000:1000 TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 200 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# route-target import 2000:2000 TOR1(config-evpn-vni)# exit TOR2 TOR2 TOR2# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR2(config)# interface Loopback 1 TOR2(config-if-Loopback1)#ipaddress2.2.2.2/32 TOR2(config-if- Loopback 1)# exit TOR2(config)# vtep TOR2(config-vtep)# source loopback 1 TOR2(config-vtep)# arp suppress enable TOR2(config-vtep)# exit TOR2(config)# fabric anycast-gateway-mac 0011.2233.2016 TOR2(config)# ip vrf vrf-10 TOR2(config-vrf)# rd 10:10 TOR2(config-vrf)# route-target export 1000:1000 TOR2(config-vrf)# exit TOR2(config) # ip vrf vrf-20 TOR2(config- vrf)# rd 20:20 TOR2(config-vrf)# route-target export 2000:2000 TOR2(config-vrf)# exit TOR2(config)# int overlayrouter 10 TOR2(config-if-OverlayRouter 10)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR2(config-if-OverlayRouter 10)# anycast-gateway TOR2(config-if-OverlayRo uter 10)# exit TOR2(config)# int overlayrouter 20 TOR2(config-if-OverlayRouter 20)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 20)# ip address 20.1.1.1/24 TOR2(config-if-OverlayRouter 20)# anycast-gateway TOR2(config-if-OverlayRo uter 20)# exit TOR2(config)# int overlayrouter 100 TOR2(config-if-OverlayRouter 100)# ip vrf forwarding vrf-10 TOR2(config-if-OverlayRouter 100)# ip address 100.1.3.1/24 TOR2(config-if-OverlayRouter 100)#exit TOR2(config)# int overlayrouter 30 TOR2(config-if-OverlayRouter 30)# ip vrf forwarding vrf-20 TOR2(config-if-OverlayRouter 30)# ip address 30.1.1.1/24 TOR2(config-if-OverlayRouter 30)# anycast-gateway TOR2(config-if-OverlayRouter 30)# exit TOR2(config)# vxlan 30 TOR2(config-vxlan)# router-interface OverlayRouter 30 TOR2(config)# int overlayrouter 200 TOR2(config-if-OverlayRouter 200)# ip vrf forwarding vrf-20 TOR2(config-if-OverlayRouter 200)# ip address 200.1.3.1/24 TOR2(config-if-OverlayRouter 200)#exit TOR2(config)# vxlan 10 TOR2(config-vxlan)# extend-vlan 10 TOR2(config-vxlan)# router-interface OverlayRouter 10 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# vxlan 20 TOR2(config-vxlan)# extend-vlan 20 TOR2(config-vxlan)# router-interface OverlayRouter 20 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxl an)# exit TOR2(config)# vxlan 100 TOR2(config-vxlan) # symmetric TOR2(config-vxlan)# router-interface OverlayRouter 100 TOR2(config-vxlan)# exit TOR2(config)# vxlan 30 TOR2(config-vxlan)# extend-vlan 30 TOR2(config-vxlan)# router-interface OverlayRouter 30 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxl an)# exit TOR2(config)# vxlan 200 TOR2(config-vxlan) # symmetric TOR2(config-vxlan)# router-interface OverlayRouter 200 TOR2(config-vxlan)# exit TOR2(config)# router bgp 64512 TOR2(config-router)# neighbor 1.1.1.1 remote-as 64512 TOR2(config-router)# neighbor 1.1.1.1 update-source loopback 1 TOR2(config-router)# neighbor 3.3.3.3 remote-as 64512 TOR2(config-router)# neighbor 3.3.3.3 update-source loopback 1 TOR2(config-router)# neighbor 4.4.4.4 remote-as 64512 TOR2(config-router)# neighbor 4.4.4.4 update-source loopback 1 TOR2(config-router)# address-family l2vpn evpn TOR2(config-router-af)# neighbor 1.1.1.1 activate TOR2(config-router-af)# neighbor 3.3.3.3 activate TOR2(config-router-af)# neighbor 4.4.4.4 activate TOR2(config-router-af)#advertise ipv4unicast TOR2(config-router-af)# exit TOR2(config-router)# address-family ipv4 vrf vrf-10 TOR2(config-router-af)# redistribute connected TOR2(config-router-af)# exit TOR2(config-router)# address-family ipv4 vrf vrf-20 TOR2(config-router-af)# redistribute connected TOR2(config-router-af)# exit TOR2(config-rout er)# exit TOR2(config)# evpn TOR2(config-evp n)# vni 10 TOR2(config-evpn- vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evp n)# vni 20 TOR2(config-evpn- vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 30 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 100 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# route-target import 1000:1000 TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 200 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# route-target import 2000:2000 TOR2(config-evpn-vni)# exit Border 1 Border1 Border1# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. Border1(config)# interface Loopback 1 Border1(config-if- Loopback 1)# ip address 3.3.3.3/32 Border1(config-if- Loopback 1)# exit Border1(config)# vtep Border1(config-vtep)# source loopback 1 Border1(config-vtep)# arp suppress enable Border1(config-vtep)# exit Border1(config)# fabric anycast-gateway-mac 0011.2233.2016 Border1(config)# ip vrf vrf-10 Border1(config-vrf)# rd10:10 Border1(config-vrf)# route-target export 1000:1000 Border1(config-vrf)# route-target import 3000:3000 Border1(config-vrf)# exit Border1(config)# ip vrf vrf-20 Border1(config-vrf)# rd 20:20 Border1(config-vrf)# route-target export 2000:2000 Border1(config-vrf)# route-target import 3000:3000 Border1(config-vrf)# exit Border1(config)# ip vrf vrf-30 Border1(config-vrf)# rd 30:30 Border1(config-vrf)# route-target export 3000:3000 Border1(config-vrf)# route-target import 1000:1000 Border1(config-vrf)# route-target import 2000:2000 Border1(config-vrf)# exit Border1(config)# int overlayrouter 90 Border1(config-if-OverlayRouter 90)# ip vrf forwarding vrf-30 Border1(config-if-OverlayRouter 90)# ip address 90.1.1.1/24 Border1(config-if-OverlayRouter 90)# anycast-gateway Border1(config-if-OverlayRouter 90)# exit Border1(config)# int overlayrouter 100 Border1(config-if-OverlayRouter 100)# ip vrf forwarding vrf-10 Border1(config-if-OverlayRouter 100)# ip address 100.1.1.1/24 Border1(config-if-OverlayRouter 100)#exit Border1(config)# int overlayrouter 200 Border1(config-if-OverlayRouter 200)# ip vrf forwarding vrf-20 Border1(config-if-OverlayRouter 200)# ip address 200.1.1.1/24 Border1(config-if-OverlayRouter 200)#exit Border1(config)# vxlan 90 Border1(config-vxlan)# extend-vlan 90 Border1(config-vxlan)# router-interface OverlayRouter 90 Border1(config-vxlan)# arp suppress enable Border1(config-vxlan)# exit Border1(config)# vxlan 100 Border1(config-vxlan) # symmetric Border1(config-vxlan)# router-interface OverlayRouter 100 Border1(config-vxlan)# exit Border1(config)# vxlan 200 Border1(config-vxlan) # symmetric Border1(config-vxlan)# router-interface OverlayRouter 200 Border1(config-vxlan)# exit Border1(config)# router bgp 64512 Border1(config-router)# neighbor 1.1.1.1 remote-as 64512 Border1(config-router)# neighbor 1.1.1.1 update-source loopback 1 Border1(config-router)# neighbor 2.2.2.2 remote-as 64512 Border1(config-router)# neighbor 2.2.2.2 update-source loopback 1 Border1(config-router)# neighbor 4.4.4.4 remote-as 64512 Border1(config-router)# neighbor 4.4.4.4 update-source loopback 1 Border1(config-router)# address-family l2vpn evpn Border1(config-router-af)# neighbor 1.1.1.1 activate Border1(config-router-af)# neighbor 2.2.2.2 activate Border1(config-router-af)# neighbor 4.4.4.4 activate Border1(config-router-af)# advertise ipv4 unicast Border1(config-router-af)# exit Border1(config-router)# address-family ipv4 vrf vrf-10 Border1(config-router-af)# exit Border1(config-router)# address-family ipv4 vrf vrf-20 Border1(config-router-af)# exit Border1(config-router)# address-family ipv4 vrf vrf-30 Border1(config-router-af)# redistribute static Border1(config-router-af)# exit Border1(config-router)# exit Border1(config)# evpn Border1(config-evpn)# vni 100 Border1(config-evpn-vni)# rd auto Border1(config-evpn-vni)# route-target both auto Border1(config-evpn-vni)# route-target import 3000:3000 Border1(config-evpn-vni)# exit Border1(config-evpn)# vni 200 Border1(config-evpn-vni)# rd auto Border1(config-evpn-vni)# route-target both auto Border1(config-evpn-vni)# route-target import 3000:3000 Border1(config-evpn-vni)# exit Border 2 Border2 Border2# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. Border2(config)# interface Loopback 1 Border2(config-if- Loopback 1)# ip address 4.4.4.4/32 Border2(config-if- Loopback 1)# exit Border2(config)# vtep Border2(config-vtep)# source loopback 1 Border2(config-vtep)# arp suppress enable Border2(config-vtep)# exit Border2(config)# fabric anycast-gateway-mac 0011.2233.2016 Border2(config)# ip vrf vrf-10 Border2(config-vrf)# rd10:10 Border2(config-vrf)# route-target export 1000:1000 Border2(config-vrf)# route-target import 3000:3000 Border2(config-vrf)# exit Border2(config)# ip vrf vrf-20 Border2(config-vrf)# rd 20:20 Border2(config-vrf)# route-target export 2000:2000 Border2(config-vrf)# route-target import 3000:3000 Border2(config-vrf)# exit Border2(config)# ip vrf vrf-30 Border2(config-vrf)# rd 30:30 Border2(config-vrf)# route-target export 3000:3000 Border2(config-vrf)# route-target import 1000:1000 Border2(config-vrf)# route-target import 2000:2000 Border2(config-vrf)# exit Border2(config)# int overlayrouter 90 Border2(config-if-OverlayRouter 90)# ip vrf forwarding vrf-30 Border2(config-if-OverlayRouter 90)# ip address 90.1.2.1/24 Border2(config-if-OverlayRouter 90)# anycast-gateway Border2(config-if-OverlayRouter 90)# exit Border2(config)# int overlayrouter 100 Border2(config-if-OverlayRouter 100)# ip vrf forwarding vrf-10 Border2(config-if-OverlayRouter100)#ipaddress 100.1.2.1/24 Border2(config-if-OverlayRouter 100)#exit Border2(config)# int overlayrouter 200 Border2(config-if-OverlayRouter 200)# ip vrf forwarding vrf-20 Border2(config-if-OverlayRouter 200)# ip address 200.1.2.1/24 Border2(config-if-OverlayRouter 200)#exit Border2(config)# vxlan 90 Border2(config-vxlan)# extend-vlan 90 Border2(config-vxlan)# router-interface OverlayRouter 90 Border2(config-vxlan)# arp suppress enable Border2(config-vxlan)# exit Border2(config)# vxlan 100 Border2(config-vxlan) # symmetric Border2(config-vxlan)# router-interface OverlayRouter 100 Border2(config-vxlan)# exit Border2(config)# vxlan 200 Border2(config-vxlan) # symmetric Border2(config-vxlan)# router-interface OverlayRouter 200 Border2(config-vxlan)# exit Border2(config)# router bgp 64512 Border2(config-router)# neighbor 1.1.1.1 remote-as 64512 Border2(config-router)# neighbor 1.1.1.1 update-source loopback 1 Border2(config-router)# neighbor 2.2.2.2 remote-as 64512 Border2(config-router)# neighbor 2.2.2.2 update-source loopback 1 Border2(config-router)# neighbor 3.3.3.3 remote-as 64512 Border2(config-router)# neighbor 3.3.3.3 update-source loopback 1 Border2(config-router)# address-family l2vpn evpn Border2(config-router-af)# neighbor 1.1.1.1 activate Border2(config-router-af)# neighbor 2.2.2.2 activate Border2(config-router-af)# neighbor 3.3.3.3 activate Border2(config-router-af)# advertise ipv4 unicast Border2(config-router-af)# exit Border2(config-router)# address-family ipv4 vrf vrf-10 Border2(config-router-af)# exit Border2(config-router)# address-family ipv4 vrf vrf-20 Border2(config-router-af)# exit Border2(config-router)# address-family ipv4 vrf vrf-30 Border2(config-router-af)# redistribute static Border2(config-router-af)# exit Border2(config-rout er)# exit Border2(config)# evpn Border2(config-evp n)# vni 100 Border2(config-evpn -vni)# rd auto Border2(config-evpn-vni)# route-target both auto Border2(config-evpn-vni)# route-target import 3000:3000 Border2(config-evpn-vni)# exit Border2(config-evpn)# vni 200 Border2(config-evpn-vni)# rd auto Border2(config-evpn-vni)# route-target both auto Border2(config-evpn-vni)# route-target import 3000:3000 Border2(config-evpn-vni)# exit Verification Verify that HOST-1, HOST-2, and HOST-4 can ping each other. Verify that HOST-3 and HOST-6 can ping each other. Verify that HOST-1, HOST-2, and HOST-4 cannot ping HOST-3 and HOST-6. Verify that the virtual machines can be migrated between the HOSTson the same VXLANand can access the network normally after migration without modifying the configuration. Border1# sh vxlan VXLAN Total Count: 3 VXLAN Capacity : 8000 VXLAN 90 Symmetric property : FALSE RouterInterface : overlayrouter 90 (anycast) ExtendVLAN 90 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6146 3.3.3.3 2.2.2.2 dynamic VXLAN 100 Symmetric property : TRUE RouterInterface : overlayrouter 100 (non-anycast) ExtendVLAN : - VTEP Adjacency Count: 1 Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6146 3.3.3.3 2.2.2.2 dynamic VXLAN 200 Symmetric property : TRUE RouterInterface : overlayrouter 200 (non-anycast) ExtendVLAN : - VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6146 3.3.3.3 2.2.2.2 dynamic 4.3 Configuring an EVPN Distributed Network to Be Compatible with Non-EVPN VTEP Devices Configuration Effect Enable the control plane learning function to implement VXLAN tunnel learning, MAC address learning, and route learning via control plane protocols, thereby finally implementing VXLAN bridging, VXLAN routing, and data communication between VXLANs and between a VXLAN and an external network. Support functions such as anycast gateways, symmetric VXLAN instances, ARP suppression, and IPv6 ND suppression in EVPN control plane mode. Non-EVPN VTEP devices establish tunnels with EVPN-supported VTEP devices to implement the VXLAN bridging and forwarding function. Notes VXLAN instances require the support from existing unicast routes on the network. Therefore, an IPv4 unicast routing protocol such as OSPF must be configured on the network devices. VXLAN needs the MP-BGP-EVPN protocol to implement VXLAN tunnel learning, MAC address learning, and route learning. Therefore, BGP must be configured on EVPN-supported network devices. Configuration Steps Configuringa Loopback Interface Associated with the Local End Mandatory for devices supporting the EVPN address family. Configurethe IPaddressof a loopback interfaceasthe IPaddress ofthe localVTEP.One devicecan be associated withonly one loopback interface and the IP address of the loopback interface serves as the VXLAN VTEP IP address. If the L3 egress isan overlay router interface during static route configuration, the next-hop IP address cannot be settotheVTEP IP address. Configuring a Virtual MAC Address for Anycast Gateways Optional. When anycast gateways are required, a unified virtual MAC address must be configured and used as the MAC address of the anycastgateways. The anycastfunction can beenabledontheVXLAN overlay router interfaces of the localdevice only after the virtual MAC address is configured. Configuring ARP Suppression Optional. After ARP suppression is enabled, the VTEP device responds to ARP requests from hosts as a proxy, to reduce flooded ARP data on the network. ARP suppression is generally enabled on distributed gateways in distributed deployment scenarios. Configuring ARP Proxy Optional. After ARP suppression is enabled on a VTEP device, you can enable the ARP proxy function on an overlay router interface. After ARP proxy is enabled, the VTEP device responds toARP requests from hosts as a proxy and the MAC address used for proxy response is the gateway MAC address configured on the VTEP device. In this way, the MAC address in the ARP request responses are the MAC address of the VTEP device, and the traffic between hosts in the same VNI is forwarded at L3. ARP proxy can be enabled only on VXLAN gateways and is generally enabled on distributed gateways in distributed deployment scenarios. Configuring IPv6 ND Suppression Optional. After IPv6 ND suppression is enabled, the VTEP device responds to NS multicast packets from hosts as a proxy, to reduce flooded NS multicast packets on the network. IPv6 ND suppression is generally enabled on distributed gateways in distributed deployment scenarios. Configuring the EVPN Protocol Packet Control Function In symmetric EVPN deployment scenarios, the EVPN protocol packet control function can be configured on TOR switches to reduce the traffic of EVPN packets. Currently, the EVPN protocol packet control function includes the following: Extracting MAC entries from EVPN MAC-IP type-2 routes (ARP entries) on a L2-VPNVXLAN instance Extracting MAC entries from EVPN MAC-IPv6 type-2 routes (IPv6 ND entries) on a L2-VNIVXLAN instance Banning synchronization of the local MAC address to the remote VTEP through EVPN messageson an L2-VNIVXLAN instance Banning delivery of the MAC addresses remotely synchronized through EVPN messages to the local MAC address tableonan L2-VNI VXLANinstance Stopping an L2-VNIVXLAN instance from generating EVPN type-2 routes Creating aVXLAN Instance Mandatory. Associating the VXLAN Instance with an Overlay Router Interface Mandatory for VXLAN gateways. The device supports the VXLAN routing function and can serve asa VXLAN IP gateway only after the VXLAN is associated with an overlay routerinterface. Associating the VXLAN Instance with aVLAN Mandatory for VXLAN devices directly connected to hosts. Packets of aVLAN are encapsulated into VXLAN packets for forwarding only after the VLAN is associated with aVXLAN instance. After a VLAN is associated with aVXLAN, all packets of the VLAN will be encapsulated into VXLAN packets. Therefore, an SVI on the device cannot be used as the IP gateway of the VLAN. Configuring Overlay Tunnels Mandatory for a VTEP not supporting EVPN and devices that directly communicate with the VTEP. The type of tunnels mutually established by two VTEP devices must be the same. The tunnels are those delivered by the SDN controller, statically configured on the CLI, or auto-discovered by EVPN. Configuring the Source and Destination IP Addresses for Overlay Tunnels Mandatory for a VTEP not supporting EVPN and devices that directly communicate with the VTEP. Associating the VXLAN Instance with the Overlay Tunnels Mandatory for a VTEP not supporting EVPN and devices that directly communicate with the VTEP. This command is used to statically specify VXLAN tunnels. Configuring Storm Control for the VXLAN Instance Optional. This function is required only when the storm rate needs to be limited for a VXLAN instance. Configuring the VXLAN UDP Destination Port Optional. The VXLAN UDP destination port used by early devices may not be port 4789. You can run this command to achieve compatibility. In addition, you can also run this command to specify the VXLAN UDP destination port. The VXLAN UDP destination port 4789 designated by IANA is used by default. Configuring Symmetric Instances Optional. Symmetric instances need to be configured only in symmetric scenarios. Only one symmetric instance can be configured in each VRF instance. After asymmetric instance is configured ina VRF instance, L3 forwarding in other asymmetric instances is switched to the symmetric instance. Configuring VXLAN Static Routes Optional. Configure VXLAN static routes for VXLAN instances if required. Configuring the Synchronization of MAC Entries Whose Egresses Are Static Tunnels Optional. Configure this function when MAC entries with the egress of static tunnels need to be synchronized externally. Configuring the Synchronization of ARP Entries Whose Egresses Are Static Tunnels Optional. Configure this function when ARP entries with the egress of static tunnels need to be synchronized externally. Verification The device can establish VXLAN tunnels and obtain VXLAN MAC entries and VXLAN routing entries via EVPN control plane learning. The tunnels and entries can be those delivered by the SDN controller or statically configured on the CLI. They implement the inter-VTEP communication. Run the following commands for verification. Run the show vxlan vni-number command to check whether the local and remote VXLAN devices learn the peer VTEP neighbor relationships. Run the show vxlan mac to check whether the VXLAN MAC addresses are learned. Run the show arp command to check whether the ARP entry of the VXLAN IP gateway is learned. Run the show ipv6 neighbors command to check whether all local/remote IPv6 ND entries are learned. Run the show vxlan udp-port command to display the VXLAN UDP destination port. Related Commands Configuring a Loopback Interface Associated with the Local End Command source loopback loopback-port-id Parameter Description loopback-port-id: Indicates the ID of the loopback interface. Command Mode VTEP configuration mode Usage Guide The local VETP IP address is the IP address of the configured loopback interface. Configuring a Virtual MAC Address for Anycast Gateways Command fabric anycast-gateway-mac mac-addr Parameter Description mac-addr: Indicates the MAC address in the format of xxxx.xxxx.xxxx. Command Mode Global configuration mode Usage Guide All gateways on which the anycast function is enabled use this MAC address as the gateway MAC address. The virtual MAC address for anycast gateways cannot beset to the local MAC address orthe MAC address of any device on the overlay network. Configuring Remote ARP Packet Learning Command remote arp learn enable Parameter Description N/A Command Mode VTEP configuration mode Usage Guide Enable or disable the remote ARP packet learning function globally. After this function is enabled, the VXLAN gateways will learn ARP entries from the VXLAN-encapsulated ARP packets received from VXLAN tunnels. Configuring Remote IPv6 ND Protocol Packet Learning Command remote nd learn enable Parameter N/A Description Command Mode VTEP configuration mode Usage Guide Enable or disable the remote IPv6 ND packet learning function globally. After this function is enabled, the device can learn IPv6 ND entries from the VXLAN-encapsulated IPv6 NS packets received from VXLAN tunnels. Configuring Global ARP Suppression Command arp suppress enable Parameter Description N/A Command Mode VTEP configuration mode Usage Guide Enable or disable the global ARP suppression function. After ARP suppression is enabled, the switch responds toARP requestsfrom hosts asa proxy. TheVNI-based ARP suppression maybealso supported, depending on the product type. Youcan configure global ARP suppression orVNI-based ARP suppression based on the actual application scenario. Configuring VNI-based ARP Suppression Command arp suppress enable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide Enable or disable VNI-based ARP suppression. After ARP suppression is enabled, the switch responds to ARP requests from hosts as a proxy. The global ARP suppression may be also supported, depending on the product type. You can configure global ARP suppression or VNI-based ARP suppression based on the actual application scenario. Configuring ARP Proxy Command route-in-vni Parameter Description N/A Command Mode Overlay router interface configuration mode Usage Guide After the intra-VNI routing function (ARP proxy) is enabled on an overlay router interface, the VTEP device uses its gateway MAC address to respond to all ARP requests from hosts in theVNI, towhich the overlay router interface belongs, when serving as an ARP proxy. In this way, the communication traffic between hosts in the same VNI is forwarded through VXLAN routes. Configuring Global IPv6 ND Suppression Command nd suppress enable Parameter Description N/A Command Mode VTEP configuration mode Usage Guide Enable or disable the global IPv6 ND suppression function. After IPv6 ND suppression is enabled, the device responds to IPv6 NS multicast packets from hosts as a proxy. The VNI-based IPv6 ND suppression may be also supported, depending on the product type. You can configure global IPv6 ND suppression or VNI-based IPv6 ND suppression based on the actual application scenario. Configuring VNI-based IPv6 ND Suppression Command nd suppress enable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide Enable or disable the VNI-based IPv6 ND suppression function. After IPv6 ND suppression is enabled, the device responds to IPv6 NS multicastpacketsfrom hostsasa proxy. Theglobal IPv6 NDsuppression maybealsosupported, depending on the product type. Youcan configure global IPv6 ND suppression or VNI-based IPv6 ND suppression based on the actual application scenario. Extracting MAC Entries from EVPN MAC-IP Type-2 Routes (ARP Entries) Command evpn arp mac-learning enable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide After this command is configured, the device parses one ARP entry and one MAC entry from a MAC-IP type-2 route synchronized from the VXLAN-EVPN neighbor. This command is disabled by default and the device parsesoneARP entry but no MAC entry from a MAC-IP type-2 route synchronized from the VXLAN-EVPN neighbor. This command is configured on aVXLAN instance and affects only the EVPN entry parsing of theVXLAN instance. Other VXLAN instances, for which this command is not configured, are not affected. This command can be used in combination with theevpn mac advertise disable command. After they are executed, the network-wide VXLAN-EVPN neighbors synchronize only MAC-IP type-2 routes but no MAC-only type-2 routes. All devices parse and extract MAC entries from MAC-IP type-2 routes. In symmetric deployment scenarios, this command is configured on L3-VNI VXLAN instances (that is, symmetric instances). Extracting MAC Entries from EVPN MAC-IPv6 Type-2 Routes (IPv6 ND Entries) Command evpn nd mac-learning enable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide After this command is configured, the device parses one IPv6 ND entry and one MACentry from a MAC-IPv6 type-2 route (IPv6 ND entry) synchronized from the VXLAN-EVPN neighbor. This command is disabled by default and the device parses one IPv6 ND entry but no MAC entry from a MAC-IPv6 type-2 route synchronized from the VXLAN-EVPN neighbor. This command is configured on aVXLAN instance and affects onlytheEVPN entry parsing of theVXLAN instance. Other VXLAN instances, for which this command is not configured, are not affected. This command can be used in combinationwiththeevpn mac advertise disable command. Afterthey are executed, the network-wide VXLAN-EVPN neighbors synchronize only MAC-IPv6 type-2 routes but no MAC-only type-2 routes. All devices parse and extract MAC entries from MAC-IPv6 type-2 routes. This command is configured on L2-VNI VXLAN instances. Configuring an L2-VNI VXLAN Instance Not to Synchronize the Local MAC Address to the Remote VTEP Through EVPN Messages Command evpn mac advertise disable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide This command is not configured on a device by default. The device generates one MAC-only type-2 route through the VXLAN-EVPN protocol based on a locally learned MAC entry, and synchronizes the type-2 route to the EVPN neighbor (that is, remoteVTEP). Then, the remote VTEP can learn the MAC entry from the MAC-only type-2 route. After this command is configured, the device does not generate VXLAN-EVPN MAC-only type-2 routes based on MAC entries, and therefore, it will not advertise MAC-only type-2 routes to the EVPN neighbor. This command is configured on aVXLAN instance and affects only whether the VXLAN instance generates MAC-only type-2 routes. Other VXLAN instances, for which this command is not configured, can still generate MAC-only type-2 routes. This command can be used in combinationwith theevpn arp mac-learning enable and evpn nd mac-learning enable commands. After they are executed, the network-wide VXLAN-EVPN neighbors synchronize only MAC-IP type-2 routes but no MAC-only type-2 routes.All devicesparse and extract MAC entriesfrom MAC-IP or MAC-IPv6 type-2 routes. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances with the symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. Configuring an L2-VNI VXLAN Instance Not to Deliver MAC Addresses Remotely Synchronized Through EVPN Messages to the Local MAC Address Table Command evpn mac inactive Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide After this command is configured, the device does not learn MAC entries from VXLAN-EVPN type-2 routes (MAC-IP or MAC-only type-2 routes) synchronized from neighbors. This command isnot configuredona device by default. The device learns MAC entriesfromVXLAN-EVPN type-2 routes synchronized from neighbors. This command is configured on aVXLAN instance and affectsonlywhethertheVXLAN instance learns MACentriesfrom VXLAN-EVPN type-2 routes. Other VXLAN instances, for which this command is not configured, can still learn MAC entries. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances withthe symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. Configuring an L2-VNI VXLAN Instance Not to Generate EVPN Type-2 Routes Command evpn rt-2 advertise disable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide This command is not configured on a device by default. The device generates one MAC-only type-2 route through the VXLAN-EVPN protocol based on a locally learned MAC entry, and synchronizes the type-2 route to the EVPN neighbor (that is, remote VTEP). Then, the remote VTEP learns the MAC entry from the MAC-only type-2 route. In addition, the devicegenerates one MAC-IPtype-2 route through the VXLAN-EVPN protocol basedona locally learned ARP entry and synchronizes thetype-2 routetotheEVPN neighbor.Then, the remoteVTEP learns theARP entry and host route fromthe MAC-IP type-2 route. The device generates one MAC-IPv6 type-2 route through the VXLAN-EVPN protocol basedon a locallylearned IPv6 NDentry, and synchronizes thetype-2 routetotheEVPN neighbor.Then, the remoteVTEP learns the IPv6 ND entry and host route from the MAC-IPv6 type-2 route. After this command is configured, the MAC entries, ARP entries, and IPv6 ND entries of the device are not used to generate VXLAN-EVPN type-2 routes and therefore, no type-2 route is advertised to the EVPN neighbor. Thiscommand isconfigured onaVXLAN instanceand affectsonly whether theVXLAN instancegeneratestype-2 routes. Other VXLAN instances, for which this command is not configured, can still generate type-2 routes. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances with the symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. Creating an Overlay Router Interface Command interface OverlayRouter port-id Parameter Description port-id: Indicates the ID of an overlay router interface. Command Mode Global configuration mode Usage Guide This interface serves as the VXLAN IP gateway in the VXLAN routing environment. It is similar to an SVI interface in a VLAN. Configuring an IP Address for the Overlay Router Interface Command ip address ip-address mask Parameter Description ip-address: Indicates the IP address of the overlay router interface. mask: Indicates the subnet mask. Command Mode Interface configuration mode Usage Guide This IP address serves as the VXLAN IP gateway address in the VXLAN routing environment. It is similar to the IP address of an SVI in a VLAN. Configuring an IPv6 Address for the Overlay Router Interface Command ipv6 address ip-address mask Parameter Description ip-address: Indicates the IPv6 address of the overlay router interface. mask: Indicates the subnet mask. Command Mode Overlay router interface configuration mode Usage Guide This IPv6 address serves as the VXLAN IPv6 gateway address in the VXLAN routing environment. It is similar to the IP address of an SVI in a VLAN. Associating the Overlay Router Interface with a VRF Instance Command vrf forwarding table name Parameter Description table name: Indicates the VRF instance, with which the overlay router interface is associated. Command Mode Interface configuration mode Usage Guide This command is used to associate with a VRF instance in the VXLAN routing environment and is used for VXLAN L3 routing isolation. Creating a VXLAN Instance or Entering the VXLAN Configuration Mode Command vxlan vni-number Parameter Description vni-number: Indicates the VNI. The value ranges from 1 to 16,777,215. Command Mode Global configuration mode Usage Guide N/A Configuring a Symmetric Instance Command symmetric Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide No symmetric instance is configured by default. A symmetric instance is used to manage L3 forwarding entries of all asymmetric instances in the VRF instance associated with the symmetric instance. Associating the VXLAN Instance with the Overlay Router Interface Command router-interface interface-name Parameter Description interface-name: Indicates the name of the overlay router interface. Command Mode VXLAN configuration mode Usage Guide The overlay router interfaces betweenVXLANs cannot conflict witheachother and differentVXLANs cannot associate with the same overlay router interface. Creating an Overlay Tunnel Interface Command interface OverlayTunnel port-id Parameter port-id: Indicates the ID of an overlay tunnel interface. Description Command Mode Global configuration mode Usage Guide This interface is used to statically create an overlay tunnel. You can run the tunnel-interface command to associate it with a VXLAN. Configuring a Tunnel Source IP Address for the Overlay Tunnel Interface Command tunnel source ip-address Parameter Description ip-address: Indicates the tunnel source IP address. Command Mode Overlay tunnel interface configuration mode Usage Guide This command is used to specify the source IP address of an overlay tunnel. When packets are encapsulated and forwarded, the outer source IP address of the packets is the source IP address of an overlay tunnel. Configuring a Tunnel Destination IP Address for the Overlay Tunnel Interface Command tunnel destination ip-address Parameter Description ip-address: Indicates the tunnel destination IP address. Command Mode Overlay tunnel interface configuration mode Usage Guide This command is used to specify the destination IP address of an overlay tunnel. When packets are encapsulated and forwarded, the outer destination IP address of the packets is the destination IP address of an overlay tunnel. The tunnel destination IP address is unique globally. Different overlay tunnels cannot share the same destination IP address. Otherwise, a configuration conflict will occur. Associating the VXLAN Instance with the Overlay Tunnel Interface Command tunnel-interface interface-name Parameter Description interface-name: Indicates the name of an overlay tunnel interface. Command Mode VXLAN configuration mode Usage Guide This command is used to statically specify a VXLAN VTEP. Configuring the VXLAN UDP Destination Port Command vxlan udp-port port-number Parameter Description port-number: Indicates the UDP destination port ID. The value ranges from 0 to65535 and the default value is 4789. Command Mode Global configuration mode Usage Guide The VXLAN UDP destination port cannot be set to a commonly used UDP port. Configuring Storm Control for the VXLAN Instance Command storm-control {broadcast | multicast | unicast} [kbps-value | pps pps-value] Parameter Description kbps-value: Indicates the rate limit value, in kbit/s. pps-value: Indicates the rate limit value, in packets/second. Command Mode VXLAN configuration mode Usage Guide Configure this function when the storm rate needs to be limited based on the VNI. Configuring the Synchronization of MAC Entries Whose Egresses Are Static Tunnels Command evpn mac advertise enable Parameter Description N/A Command Mode Overlay tunnel interface configuration mode Usage Guide Configure this function when MAC entries with the egress of static tunnels need to be synchronized externally. Configuring the Synchronization of ARP Entries Whose Egresses Are Static Tunnels Command evpn macip advertise enable Parameter Description N/A Command Mode Overlay tunnel interface configuration mode Usage Guide Configure this function when ARP entries with the egress of static tunnels need to be synchronized externally. Configuration Example Only configuration related to the VXLAN is described below. Only IPv4 configuration is used as an example below and the IPv6 scenario configuration is largely the same as the IPv4 scenario configuration. Detailed configuration of a full mesh network: Figure 4-8 image.png Note: Blue lines in the figure indicate the VXLAN tunnels that the manually configured VTEP-1 establishes with other VTEPs. Configuration Steps Configure an IPv4 unicast routing protocol (such as OSPF) on core switches, TOR switches, and VTEP to ensure that unicast routes are reachable. Configure the BGP-EVPN routing protocol on the CORE and TOR switches to ensure that the four switches establish BGP neighbor relationships and support the EVPN protocol family. Configure EVI for BGP-EVPN on the core and TOR switches. For details, see the BGP-EVPN Configuration Guide. Configure VXLANs on the virtual servers and specify the gateway address for the virtual machines. (Omitted) Associate the VTEP with the loopback interface on TOR-1 and TOR-2 for the establishment of tunnels. Create VXLAN instances on TOR-1, TOR-2, and VTEP-1 and associate the VXLAN instances with VLANs. Configure the same anycast gateway MAC address on TOR-1 and TOR-2 so that the VXLAN anycast gateways on the TOR switches use the same virtual MAC address. Create overlay router interfaces on TOR-1 and TOR-2 and configure the VXLAN gateway IP address. Configure different VRF instances for the overlay router interfaces and determine their respective tenants. Note that the overlay router interface configuration on TOR-1 and TOR-2 must be the same. That is, on all devices, the IP address and mask configured for the overlay router interfaces associated with the same VXLAN instance must be the same and such overlay router interfaces belong to the same tenant (VRF instance). In addition, all overlay router interfaces must be configured as anycast gateways. Associate VXLAN instances with overlay router interfaces on TOR-1 and TOR-2 to implement VXLAN routing. Create VXLAN overlay tunnels on TOR-1, TOR-2, and VTEP-1 and configure the SIP and DIP. (Optional) Configure ARP suppression on TOR-1 and TOR-2 to reduce ARP packets flowing into the VXLAN. HOST The detailed configuration of the servers is omitted here. Configure the IP address and gateway according to the figure above. CORE VXLAN does not need to be configured on the core switches. The OSPF and BGP network configurations are omitted here. TOR-1 TOR1# configure terminal Enter configuration commands, one per line. End with CNTL/Z. TOR1(config)# route-map dc TOR1(config-route-map)# match route-type evpn-type-2 TOR1(config-route-map)# set next-hop 1.1.1.1 TOR1(config-route-map)# exit TOR1(config)# interface Loopback 1 TOR1(config-if-Loopback1)#ip address1.1.1.1/32 TOR1(config-if- Loopback 1)# exit TOR1(config)# vtep TOR1(config-vtep)# source loopback 1 TOR1(config-vtep)# arp suppress enable TOR1(config-vtep)# vxlan outside center vtep-ip 3.3.3.3 TOR1(config-vtep)# exit TOR1(config)# fabric anycast-gateway-mac 0000.1234.5678 TOR1(config)# int overlayrouter 10 TOR1(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR1(config-if-OverlayRouter 10)# anycast-gateway TOR1(config-if-OverlayRouter10)#route-in-vni //Optional.Itneedstobeusedincombinationwiththearp suppress enable command. TOR1(config-if-OverlayRouter 10)# exit TOR1(config)# int overlayrouter 20 TOR1(config-if-OverlayRouter 20)# ip address 10.1.2.1/24 TOR1(config-if-OverlayRouter 20)# anycast-gateway TOR1(config-if-OverlayRouter 20)# route-in-vni TOR1(config-if-OverlayRouter 20)# exit TOR1(config)# int overlaytunnel 1 TOR1(config-if-OverlayTunnel 1)# tunnel source 1.1.1.1 TOR1(config-if-OverlayTunnel 1)# tunnel destination 3.3.3.3 TOR1(config-if-OverlayTunnel 1)# evpn mac advertise enable TOR1(config-if-OverlayTunnel 1)# evpn macip advertise enable TOR1(config-if-OverlayTunnel 1)#exit TOR1(config)# vxlan 10 TOR1(config-vxlan)# extend-vlan 10 TOR1(config-vxlan)# router-interface OverlayRouter 10 TOR1(config-vxlan)# tunnel-interface overlaytunnel 1 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxlan)# exit TOR1(config)# vxlan 20 TOR1(config-vxlan)# extend-vlan 20 TOR1(config-vxlan)# router-interface OverlayRouter 20 TOR1(config-vxlan)# arp suppress enable TOR1(config-vxl an)# exit TOR1(config)# router bgp 100 TOR1(config-router)# neighbor 2.2.2.2 remote-as 100 TOR1(config-router)# neighbor 2.2.2.2 update-source loopback 1 TOR1(config-router)# address-family l2vpn evpn TOR1(config-router-af)# neighbor 2.2.2.2 activate TOR1(config-router-af)# neighbor 2.2.2.2 route-map dc out TOR1(config-router-af)# exit TOR1(config-router)# exit TOR1(config)# evpn TOR1(config-evpn)# vni 10 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR1(config-evpn)# vni 20 TOR1(config-evpn-vni)# rd auto TOR1(config-evpn-vni)# route-target both auto TOR1(config-evpn-vni)# exit TOR2 TOR2# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. TOR2(config)# interface Loopback 1 TOR2(config-if-Loopback1)#ipaddress2.2.2.2/32 TOR2(config-if- Loopback 1)# exit TOR2(config)# vtep TOR2(config-vtep)# source loopback 1 TOR2(config-vtep)# arp suppress enable TOR2(config-vtep)# exit TOR2(config)# fabric anycast-gateway-mac 0000.1234.5678 TOR2(config)# int overlayrouter 10 TOR2(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR2(config-if-OverlayRouter 10)# anycast-gateway TOR2(config-if-OverlayRouter10)#route-in-vni //Optional.Itneedstobeusedincombinationwiththearpsuppress enable command. TOR2(config-if-OverlayRouter 10)# exit TOR2(config)# int overlayrouter 20 TOR2(config-if-OverlayRouter 20)# ip address 10.1.2.1/24 TOR2(config-if-OverlayRouter 20)# anycast-gateway TOR2(config-if-OverlayRouter20)#route-in-vni //Optional.Itneedstobeusedincombinationwiththearpsuppress enable command. TOR2(config-if-OverlayRouter 20)# exit TOR2(config)# vxlan 10 TOR2(config-vxlan)# extend-vlan 10 TOR2(config-vxlan)# router-interface OverlayRouter 10 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# vxlan 20 TOR2(config-vxlan)# extend-vlan 20 TOR2(config-vxlan)# router-interface OverlayRouter 20 TOR2(config-vxlan)# arp suppress enable TOR2(config-vxlan)# exit TOR2(config)# router bgp 100 TOR2(config-router)# neighbor 1.1.1.1 remote-as 100 TOR2(config-router)# neighbor 1.1.1.1 update-source loopback 1 TOR2(config-router)# address-family l2vpn evpn TOR2(config-router-af)# neighbor 1.1.1.1 activate TOR2(config-router-af)# exit TOR2(config-router)# exit TOR2(config)# evpn TOR2(config-evpn)# vni 10 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit TOR2(config-evpn)# vni 20 TOR2(config-evpn-vni)# rd auto TOR2(config-evpn-vni)# route-target both auto TOR2(config-evpn-vni)# exit VTEP-1 VTEP-1 VTEP1# configure terminal Enterconfigurationcommands,oneperline. EndwithCNTL/Z. VTEP1(config)# interface Loopback 1 VTEP1(config-if- Loopback 1)# ip address 3.3.3.3/32 VTEP1(config-if- Loopback 1)# exit VTEP1(config)# int overlaytunnel 1 VTEP1(config-if-OverlayTunnel 1)# tunnel source 3.3.3.3 VTEP1(config-if-OverlayTunnel 1)# tunnel destination 1.1.1.1 VTEP1(config-if-OverlayTunnel 1)# exit VTEP1(config)# vxlan 10 VTEP1(config-vxlan)# extend-vlan 10 VTEP1(config-vxlan)# tunnel-interface overlaytunnel 1 Verification Verify that HOST-1, HOST-2, HOST-3, HOST-4, and HOST-5 can ping each other. Verify that virtual machines can be migrated between hosts in the same VXLAN and can access the network normally after migration, with no need to modify the configuration. TOR1# sho vxlan VXLAN Total Count: 2 VXLANCapacity : 8000 VXLAN 10 Symmetric property : FALSE RouterInterface : overlayrouter 10 (anycast) ExtendVLAN 10 VTEP Adjacency Count: 2 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 1 1.1.1.1 3.3.3.3 static OverlayTunnel 6145 1.1.1.1 2.2.2.2 dynamic VXLAN 20 Symmetric property : FALSE RouterInterface : overlayrouter 20 (anycast) Extend VLAN 20 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 6145 1.1.1.1 2.2.2.2 dynamic TOR2# sho vxlan VXLAN Total Count: 2 VXLANCapacity : 8000 VXLAN 10 Symmetric property : FALSE RouterInterface : overlayrouter 10 (anycast) Extend VLAN 10 VTEP Adjacency Count: 2 VTEP Adjacency List : Interface SourceIP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 1 2.2.2.2 3.3.3.3 static OverlayTunnel 6145 2.2.2.2 1.1.1.1 dynamic VXLAN 20 Symmetric property : FALSE RouterInterface : overlayrouter 20 (anycast) Extend VLAN 20 Symmetric property : FALSE Router Interface : overlayrouter 20 (anycast) Extend VLAN 20 VTEP Adjacency Count: 1 VTEP Adjacency List : Interface Source IP Destination IP Type OverlayTunnel 6145 2.2.2.2 1.1.1.1 dynamic VTEP1# sho vxlan VXLAN Total Count: 1 VXLAN Capacity : 8000 VXLAN 10 Symmetricproperty :FALSE RouterInterface : - ExtendVLAN 10 VTEP Adjacency Count: 2 VTEP Adjacency List : Interface Source IP Destination IP Type ---------------------- --------------- --------------- ------- OverlayTunnel 1 3.3.3.3 1.1.1.1 static OverlayTunnel 2 3.3.3.3 2.2.2.2 static 4.4 Configuring L2 Sub interfaces to Accessa VXLAN Configuration Effect Configure hosts to access aVXLAN through L2 subinterfaces. L2 subinterfaces can access aVXLAN inVLAN encapsulation or untagged encapsulation mode. Notes If the main interface is a trunk interface, the subinterfaces are not recommended to access a VXLAN in untagged encapsulation mode due to chip limitations. If the untagged encapsulation mode is configured for the subinterfaces, tagged packets are transferred to the logic of subinterfaces using the untagged packaging mode. The extend-VLAN configured for a VXLAN instance is like the VLAN or untagged encapsulation rule configured for a subinterface. When a subinterface is available, the encapsulation rule of the subinterface has a higher priority than the extend-VLAN configured for a VXLAN instance. Configuration Steps Configuring the VXLAN Encapsulation Rule for a Subinterface Mandatory. Configure theVXLAN encapsulation rule for a specified subinterface. Configuring the VLAN and Untagged Encapsulation Rules for a Subinterface Mandatory. L2 subinterfaces can access aVXLAN inVLAN encapsulation or untagged encapsulation mode. Creating a VXLAN Instance Mandatory. Associating the VXLAN Instance with an Overlay Router Interface Mandatory forVXLAN gateways. Thedevicesupports theVXLANroutingfunctionandcanserveasaVXLANIPgatewayonlyaftertheVXLANis associatedwith an overlay routerinterface. Verification L2 subinterfaces can access the VXLAN. Run the following commands for verification. Run the show vxlan vni-number command to check the local configuration of theVXLAN. Run the show vxlan mac to check whether theVXLAN MAC addresses are learned. Run the show arp command to check whether the ARP entry of theVXLAN IP gateway is learned. Run the show ipv6 neighbors command to check whether all local/remote IPv6 ND entries are learned. Run the show running command to display the subinterface configuration. Related Commands Configuring the VXLAN Encapsulation Rule for a Subinterface Command encapsulation vxlan vni-number Parameter Description vni-number: Indicates the VNI. The value ranges from 1 to 16,777,215. Command Mode L2 subinterface configuration mode Usage Guide 1. Only one VXLAN encapsulation rule can be configured for one subinterface. 2. You can preconfigure VXLAN encapsulation rules when noVXLAN instance exists. Configuring the VLAN and Untagged Encapsulation Rules for a Subinterface Command encapsulation dot1q {untag | s-vid vlan-id} Parameter Description {untag | s-vid vlan-id}: SpecifiesVLANencapsulation oruntagged encapsulationfora port.TheVLAN ID rangesfrom1to 4094 when VLAN encapsulation is adopted. Command Mode L2 subinterface configuration mode Usage Guide If the main interface is a trunk interface, the subinterfaces are not recommended to access a VXLAN in untagged encapsulation mode due to chip limitations. If the untagged encapsulation mode is configured for the subinterfaces, tagged packets are transferred to the logic of subinterfaces using the untagged packaging mode. Creating an Overlay Router Interface Command interface OverlayRouter port-id Parameter Description port-id: Indicates the ID of an overlay router interface. Command Mode Global configuration mode Usage Guide This interface serves as the VXLAN IP gateway in the VXLAN routing environment. It is similar to an SVI interface in a VLAN. Configuring an IP Address for the Overlay Router Interface Command ip address ip-address mask Parameter Description ip-address: Indicates the IP address of the overlay router interface. mask: Indicates the subnet mask. Command Mode Interface configuration mode Usage Guide This IP address serves as the VXLAN IP gateway address in the VXLAN routing environment. It is similar to the IP address of an SVI in a VLAN. Configuring an IPv6 Address for the Overlay Router Interface Command ipv6 address ip-address mask Parameter Description ip-address: Indicates the IPv6 address of the overlay router interface. mask: Indicates the subnet mask. Command Mode Overlay router interface configuration mode Usage Guide This IPv6 address serves as the VXLAN IPv6 gateway address in the VXLAN routing environment. It is similar to the IP address of an SVI in a VLAN. Associating the Overlay Router Interface with a VRF Instance Command vrf forwarding table name Parameter Description table name: Indicates the VRF instance, with which the overlay router interface is associated. Command Mode Interface configuration mode Usage Guide This command is used to associate with a VRF instance in the VXLAN routing environment and is used for VXLAN L3 routing isolation. Creating a VXLAN Instance or Entering the VXLAN Configuration Mode Command vxlan vni-number Parameter Description vni-number: Indicates the VNI. The value ranges from 1 to 16,777,215. Command Mode Global configuration mode Usage Guide N/A Associating the VXLAN Instance with the Overlay Router Interface Command router-interface interface-name Parameter Description interface-name: Indicates the name of the overlay router interface. Command Mode VXLAN configuration mode Usage Guide The overlay router interfaces betweenVXLANs cannot conflict witheachother and differentVXLANs cannot associate with the same overlay router interface. Configuration Example Only configuration related to theVXLAN is described below. Only IPv4 configuration is used as an example below and the IPv6 scenario configuration is largely the same as the IPv4 scenario configuration. The recommended configuration is as follows: Figure 4-9 page_10_img_1_192ebb47.jpeg Configuration Steps Configure a virtual server, virtual machine, and physical server (omitted). Create an L2 subinterface on the TOR switch, configure VXLAN encapsulation and VLAN or untagged encapsulation rule for the subinterface. Create an overlay routerinterfaceon theTORswitch and configure theVXLAN gateway IPaddress. Configure theVXLAN instance to associate with the overlay router interface on the TOR switch to implement VXLAN routing. TOR TOR TOR# configure terminal TOR(config)#vlan 10 TOR(config-vlan)#exit TOR(config)#vlan 20 TOR(config-vlan)#exit TOR(config)#interface TFGigabitEthernet 0/1 TOR(config-if-TFGigabitEthernet 0/1)#switchport mode trunk TOR(config-if-TFGigabitEthernet 0/1)#exit TOR(config)#interface TFGigabitEthernet 0/1.1 TOR(config-subif-TFGigabitEthernet 0/1.1)#encapsulation dot1q s-vid 10 TOR(config-subif-TFGigabitEthernet 0/1.1)#encapsulation vxlan 10 TOR(config-subif-TFGigabitEthernet 0/1.1)#exit TOR(config)#interface TFGigabitEthernet 0/1.2 TOR(config-subif-TFGigabitEthernet 0/1.2)#encapsulation dot1q s-vid 20 TOR(config-subif-TFGigabitEthernet 0/1.2)#encapsulation vxlan 20 TOR(config-subif-TFGigabitEthernet 0/1.2)#exit TOR(config)#interface TFGigabitEthernet 0/2 TOR(config-if-TFGigabitEthernet 0/2)#switchport mode access vlan 10 TOR(config-if-TFGigabitEthernet 0/2)#exit TOR(config)#interface TFGigabitEthernet 0/2.1 TOR(config-subif-TFGigabitEthernet 0/2.1)#encapsulation dot1q untag TOR(config-subif-TFGigabitEthernet 0/2.1)#encapsulation vxlan 10 TOR(config-subif-TFGigabitEthernet 0/2.1)#exit TOR(config)# interface overlayrouter 10 TOR(config-if-OverlayRouter 10)# ip address 10.1.1.1/24 TOR(config-if-OverlayRouter 10)# exit TOR(config)# vxlan 10 TOR(config-vxlan)# router-interface OverlayRouter 10 TOR(config-vxlan)# exit TOR(config)# interface overlayrouter 20 TOR(config-if-OverlayRouter 20)# ip address 20.1.1.1/24 TOR(config-if-OverlayRouter 20)# exit TOR(config)# vxlan 20 TOR(config-vxlan)# extend-vlan 20 TOR(config-vxlan)# router-interface OverlayRouter 20 TOR1(config-vxlan)# exit Verification Verify that the virtual machine and physical machine can ping each other. TOR# sho vxlan VXLAN Total Count: 2 VXLANCapacity : 8000 VXLAN 10 Symmetric property : FALSE RouterInterface :overlayrouter 10 (non-anycast) ExtendVLAN : - VTEP Adjacency Count: 0 VXLAN 20 Symmetric property : FALSE RouterInterface :overlayrouter 20 (non-anycast) ExtendVLAN : - VTEP Adjacency Count: 0 TOR#sh running-config !vlan range 1,10,20 ! interface TFGigabitEthernet 0/1 switchport mode trunk ! interface TFGigabitEthernet 0/1.1 encapsulation dot1q s-vid 10 encapsulation vxlan 10 ! interface TFGigabitEthernet 0/1.2 encapsulation dot1q s-vid 20 encapsulation vxlan 20 ! interface TFGigabitEthernet 0/2 switchporttrunknativevlan10 ! interface TFGigabitEthernet 0/2.1 encapsulation dot1q untag encapsulation vxlan 10 Banning synchronization of the localMAC address tothe remoteVTEP throughEVPN messages on an L2-VNIVXLAN instance BanningdeliveryoftheMACaddresses remotelysynchronized throughEVPNmessages tothelocalMAC addresstableon an L2-VNI VXLANinstance Stopping an L2-VNIVXLAN instance from generating EVPN type-2 routes Configuring an L2-VNI VXLAN Instance Not to Synchronize the Local MAC Address to the Remote VTEP Through EVPN Messages Command evpn mac advertise disable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide This command is not configured on a device by default. The device generates one MAC-only type-2 route through the VXLAN-EVPN protocol based on a locally learned MAC entry, and synchronizes the type-2 route to the EVPN neighbor (that is, remote VTEP). Then, the remote VTEP can learn the MAC entry from the MAC-only type-2 route. After this command is configured, the device does not generate VXLAN-EVPN MAC-only type-2 routes based on MAC entries, and therefore, it will not advertise MAC-only type-2 routes to the EVPN neighbor. This command is configured on aVXLAN instance and affects only whether the VXLAN instance generates MAC-only type-2 routes. Other VXLAN instances, for which this command is not configured, can still generate MAC-only type-2 routes. This command can be used in combination with theevpn arp mac-learning enable and evpn nd mac-learning enable commands. After they are executed, the network-wide VXLAN-EVPN neighbors synchronize only MAC-IP type-2 routes but no MAC-only type-2 routes.All devicesparse and extract MAC entries from MAC-IP or MAC-IPv6 type-2 routes. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances with the symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. Configuring an L2-VNI VXLAN Instance Not to Deliver MAC Addresses Remotely Synchronized Through EVPN Messages to the Local MAC Address Table Command evpn mac inactive Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide After this command is configured, the device does not learn MAC entries from VXLAN-EVPN type-2 routes (MAC-IP or MAC-only type-2 routes) synchronized from neighbors. This command isnot configuredona device by default. The device learns MAC entries fromVXLAN-EVPN type-2 routes synchronized from neighbors. This command is configured on aVXLAN instance and affectsonly whether theVXLAN instance learns MACentries from VXLAN-EVPN type-2 routes. Other VXLAN instances, for which this command is not configured, can still learn MAC entries. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances with the symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. Configuring an L2-VNI VXLAN Instance Not to Generate EVPN Type-2 Routes Command evpn rt-2 advertise disable Parameter Description N/A Command Mode VXLAN configuration mode Usage Guide This command is not configured on a device by default. The device generates one MAC-only type-2 route through the VXLAN-EVPN protocol based on a locally learned MAC entry, and synchronizes the type-2 route to the EVPN neighbor (that is, remote VTEP). Then, the remote VTEP learns the MAC entry from the MAC-only type-2 route. In addition, the devicegenerates one MAC-IPtype-2 route through the VXLAN-EVPN protocol basedona locally learned ARP entry and synchronizes thetype-2 routetotheEVPN neighbor.Then, the remoteVTEP learns theARP entry and host route fromthe MAC-IP type-2 route. The device generates one MAC-IPv6 type-2 route through the VXLAN-EVPN protocol basedon a locallylearned IPv6 NDentry, and synchronizes thetype-2 routetotheEVPN neighbor.Then, the remoteVTEP learns the IPv6 ND entry and host route from the MAC-IPv6 type-2 route. After this command is configured, the MAC entries, ARP entries, and IPv6 ND entries of the device are not used to generate VXLAN-EVPN type-2 routes and therefore, no type-2 route is advertised to the EVPN neighbor. Thiscommand isconfigured onaVXLAN instanceand affectsonly whether theVXLAN instancegeneratestype-2 routes. Other VXLAN instances, for which this command is not configured, can still generate type-2 routes. Note: This command can be configured only on L2-VNIVXLAN instances (that is, VXLAN instances with the symmetric command not configured). It is unavailable on L3-VNI VXLAN instances. 5. Monitoring Displaying Description Command Displays the VXLAN configuration and status of the device. show vxlan vni-number Displays the MAC addresses learned by the device. show vxlan mac [vni vni-number] [address mac-address] Displays the VXLAN ARP entries learned by the device. show arp Displays the VXLAN IPv6 ND entries learned by the device. show ipv6 neighbors Displays the global configurations of the device, such as the VTEP IP address and anycast MAC address. show vxlan global Displays the ARP suppression status of the device. show vxlan arp suppress Displays the VXLAN UDP destination port of the device. show vxlan udp-port

Security Hardening Guide for N5860 and N8560 and NC8200 Series Switches

2026-02-03 - Security Hardening Guide for N5860 and N8560 and NC8200 Series Switches Overview This document provides a comprehensive security hardening guide for the FS N5860 and N8560 and NC8200 Series Switches. It outlines common security risks, recommended best practices, and detailed configuration steps to strengthen the device management plane, control plane, and data plane. By following the guidelines in this document, users can improve system security, reduce potential attack surfaces, and ensure stable and reliable network operations in enterprise and data center environments. 1 Security Hardening Overview Network devices are essentially computer systems operating in an open and uncontrolled network environment. Due to network complexity, openness, and inherent software vulnerabilities, both the device operating system and the services it carries are exposed to various security threats. These threats can generally be classified into passive attacks and active attacks. Passive Attacks Passive attacks refer to situations where attackers obtain information transmitted over the network through eavesdropping techniques without interfering with data transmission. Common security issues include: Information leakage: Information is accessed or obtained by unauthorized individuals, entities, or processes. For example, attackers may capture plaintext packets transmitted or stored in the network. Unauthorized access: Attackers exploit configuration vulnerabilities or misuse system debugging mechanisms to forcibly access the system or obtain information beyond their authorized privileges. Active Attacks Active attacks involve tampering with or forging data transmitted over the network, thereby disrupting normal communications. Common security issues include: Compromise of information integrity: Information is modified or destroyed during transmission, processing, or storage. For example, packets may be maliciously altered by a man-in-the-middle attacker. Denial of Service (DoS) attacks: Attackers exhaust network or system resources, causing temporary or complete service disruption. For example, sending a large number of malicious requests may overload the device CPU. Vulnerability exploitation: Attackers exploit software vulnerabilities on devices, potentially leading to system instability or crashes. Passive attacks do not involve modification of transmitted data and are therefore difficult to detect. Countermeasures primarily rely on encryption technologies to protect data confidentiality. In contrast, active attacks are easier to detect. In addition to preventive measures such as encryption, detection mechanisms can be used to identify and block such attacks. To mitigate the above threats, appropriate and effective security hardening strategies must be implemented. Security hardening refers to identifying potential risks faced by a system and applying effective control measures to minimize those risks, thereby achieving an optimal security level within defined performance, time, and cost constraints. Security hardening considerations span the entire lifecycle of a product, from development to deployment and operation. To mitigate security risks caused by software vulnerabilities, security requirements are integrated into the software development process, including the elimination of hidden backdoors, mandatory security scanning using professional tools (such as Nessus), and proactive vulnerability tracking for open-source components. During deployment and operation, configuring appropriate security hardening policies is essential to enhance defense capabilities and reduce security risks. 2 Security Hardening Principles Configuring security hardening measures enhances a device's resilience against risks, but simultaneously impacts device performance, memory resources, and deployment costs. Implementing more security hardening policies does not necessarily yield greater benefits for networks and services. Therefore, it is essential to thoroughly understand the impact of each policy on networks and services, comprehensively evaluate security risks against configuration costs, and select an appropriate security hardening solution. When selecting security hardening strategies, adhere to the following principles: Business Priority Principle When security hardening conflicts with business operational performance, ensure uninterrupted business operations first. Security hardening objectives must align with business goals. Design Before Implementation To avoid unpredictable outcomes from direct deployment, security hardening strategies should be implemented only after completing the design phase. Designers must clearly understand both the current network risks and the expected post-hardening network state. Design Based on Risk Severity When designing a hardening plan, strategies should be considered in descending order of risk severity. Implementation Based on Business Impact from Smallest to Largest When implementing the security hardening plan, deploy hardening strategies sequentially based on their impact on business operations, starting from the smallest to the largest, while observing and confirming the affected business operations. Comprehensive Approach Principle Networks are complex systems composed of devices, protocols, and packets. Any vulnerability can serve as an attacker's entry point. Therefore, security hardening focused solely on specific network components or security objectives cannot effectively enhance overall network security. When designing security hardening solutions, comprehensively consider confidentiality, integrity, and availability by analyzing factors such as device node types, network topology, supported protocols, and actual configurations. 3 Security Architecture 3.1 Management Plane The device management plane can interpret and execute configuration commands issued by network administrators for various device functions. Network administrators can access and configure devices through the management channels provided by the management plane. To ensure the device operating system functions properly, the management plane provides the following security mechanisms: Secure management protocols, including SSH and SNMPv3 Multiple authentication schemes, including password authentication and AAA authentication Silent authentication failure Restricting access to IPs via ACLs Disabling unnecessary services Changing service port numbers Supporting advanced encryption algorithms Security Hardening Guidelines File Signature Verification Among the above security mechanisms: Protocols like SSH and SNMPv3 ensure communication security; Authentication mechanisms and silent authentication failure prevent malicious user logins; ACLs, disabling unnecessary services, and changing service port numbers reduce the device's exposure to attacks; File signature verification ensures the operating system files from tampering; applying advanced encryption algorithms makes ciphertext resistant to brute-force attacks. 3.2 Control Plane The control plane manages network protocols, maintaining network information and routing entries. To ensure protocol integrity, it implements these security mechanisms: ARP packet inspection Authentication for common routing protocols Security for common multicast protocols NTP security mechanisms MSTP Attack Prevention Specifically, ARP packet inspection discards invalid ARP requests. Common routing protocol authentication covers OSPF, OSPFv3, RIP, IS-IS, and BGP protocols to prevent attack packets from disrupting protocol operations. Common multicast protocol security mechanisms provide Layer 3 and Layer 2 multicast safeguards, including PIM neighbor filtering, MSDP MD5 authentication, and IGMP Snooping group policies NTP security mechanisms include NTP packet encryption, port modification, and ACL access control to ensure secure NTP packet transmission and prevent malicious time tampering. MSTP anti-attack strategies encompass BPDU protection, BPDU packet interception, TC packet attack prevention, root protection, and loop protection to prevent loops or oscillations caused by MSTP protocol anomalies. 3.3 Forwarding Plane The forwarding plane processes and forwards data across all device interfaces. To ensure reliable data forwarding, it provides the following security mechanisms: Access Control Lists (ACLs) Traffic Suppression and Storm Control Port Protection Port Isolation 4 Control Plane Security 4.1 Security Hardening Strategies Port-Based Attack Protection Port-based attack protection is a defense mechanism against DoS attacks that operates at the port level. It prevents malicious protocol traffic on an attacked port from consuming excessive bandwidth and CPU resources, which could otherwise block legitimate protocol packets from being forwarded to the CPU and cause service interruption on other ports. By default, the device enables port-based attack protection for common control and user protocols, including ARP, ICMP, DHCP, IGMP, OSPF, and BGP. When a protocol-based attack occurs, the system automatically confines the impact to the affected port, minimizing the influence on other ports and ensuring overall network stability. User-Based Rate Limiting User-based rate limiting identifies users based on their MAC address or IP address and applies traffic rate limits to specific protocol packets, such as ARP, ICMP, DHCP, DHCPv6, UC-ROUTE, ND, and TCP SYN. This ensures that when a single user is involved in a DoS attack—either as the attack source or the target—the impact is limited to that user only, while other users remain unaffected. The core mechanism behind user-based rate limiting is the NFPP (Network Foundation Protection Policy) feature. By default, user-based rate limiting is enabled on the device to provide continuous protection. Attack Traceability Attack traceability allows the system to independently rate-limit or directly drop traffic originating from a specific attacking user or a specific attacking port. NFPP analyzes and collects statistics on selected packets sent to the CPU and applies predefined thresholds to different traffic types. When the packet rate exceeds the configured threshold, the traffic is identified as malicious. Based on the extracted attack source—either a user or an interface—the system applies targeted rate limiting or packet dropping to the offending traffic. At the same time, logs and alarms are generated to notify administrators, enabling them to take further actions to protect the device and maintain network security. Attack traceability is also implemented through the NFPP (Network Foundation Protection Policy) mechanism. 4.1.1 Configuring Port Attack Prevention This feature is only supported on select switch models from our company. It is disabled by default. In scenarios requiring enhanced security protection, this feature can be enabled to apply individual rate-limiting penalties to attack protocols or ports. This prevents interference with packet transmission for other protocols on the same port or packets destined for other ports. (1) Enable port attack prevention functionality and associated alerting capabilities. Hostname(config)# cpu-protect auto-port-defend enable Hostname(config)# cpu-protect auto-port-defend alarmenable (2) Enable the CPP automatic protection feature and configure BGP and OSPF as protocols requiring enhanced protection. Hostname(config)#cpu-protect auto-defend enable Hostname(config)# cpu-protect auto-defend type bgp enable Hostname(config)#cpu-protect auto-defend type ospf enable (3) Check the port attack prevention enable status. Hostname# show cpu-protect auto-port-defend summary auto-port-defend enable auto-port-defend alarm enable auto-port-defend monitor-period 300s auto-port-defendlimit-threshold 0% Interface Status Blacklist Attack (4) Check the enable status of the CPP automatic protection feature. Hostname# show cpu-protect auto-defend summary 4.1.2 User-Level Rate Limiting User-level rate limiting identifies users based on their MAC or IP address and applies rate limiting to specific protocol packets (ARP/ICMP/DHCP/DHCPv6/UC-ROUTE/ND/TCP-SYN) to limit speed. This ensures that if a single user is subjected to a DoS attack, only that user is affected, while other users remain unaffected. The core component of user-level rate limiting is the NFPP (Network Foundation Protection Policy) feature. By default, user-level rate limiting is enabled. Configure user-level bandwidth throttling Improper configuration of user-level throttling thresholds and attack thresholds may impact network services. If adjustments are required, contact our technical support engineers for assistance. NFPP provides features such as arp-guard, icmp-guard, dhcp-guard, dhcpv6-guard, ip-guard, nd-guard, and tcp-syn-guard, all of which support user-level rate limiting. Taking arp-guard as an example, you can modify the rate limit threshold and attack threshold for individual users as follows: (1) Modify the rate limit threshold and attack threshold for a single user. Hostname> enable Hostname(config)# nfpp Hostname(config-nfpp)# arp-guard rate-limit per-src-ip 50 Hostname(config-nfpp)# arp-guard rate-limit per-src-mac 50 Hostname(config-nfpp)# arp-guard attack-threshold per-src-ip 1000 Hostname(config-nfpp)# arp-guard attack-threshold per-src-mac 1000 (2) Check the speed limit waterline and attack waterline. Hostname> enable Hostname# show nfpp arp-guard summary (Format of column Rate-limit and Attack-threshold is per-src-ip/per-src-mac/per-port.) Interface Status Isolate-period Rate-limit Attack-threshold Scan-threshold Global Enable 0 50/50/1000 1000/1000/2500 100 Maximum count of monitored hosts: 20000 Monitor period: 600s 4.1.3 Attack Traceback Attack traceback enables individual rate limiting or direct packet discard for traffic originating from a single attacking user or a single attacking port. NFPP analyzes and statistics specific packets sent to the CPU, setting thresholds for each packet type. When packet rates exceed these thresholds, the system identifies the traffic as an attack. Based on the attack source user or interface identified from the packet, the system individually throttles or drops packets from that specific user or port. It also alerts administrators via logs and alarms, enabling them to take further measures to protect device security. The core component of attack tracing is the NFPP (Network Foundation Protection Policy) feature. Configuring Attack Tracing Improper adjustment of attack tracing's rate-limiting threshold and attack threshold may impact network services. If adjustments are required, we recommend contacting our technical support engineers for assistance. The attack tracing feature enables automatic detection and protection against attack sources. Pre-deployment significantly enhances operational security in live networks. During attacks, it isolates threats at their source, minimizing disruption to normal services. Most of our switching equipment has attack tracing enabled by default, requiring no special deployment. For ICMP packet attack tracing configuration with automatic protection: Set the throttling threshold to 100 pps. Set the attack threshold to 500 pps. When the same ICM P packet exceeds 100 pps per second but remains below 500 pps, packet throughput is throttled to under 100 pps. When the same ICMP packet exceeds 500 pps per second, it is identified as an attack, and all packets from that user are blocked, exceed 100 pps but remain below 500 pps per second, the packet rate is throttled to within 100 pps. If the same ICMP packet exceeds 500 pps per second, an attack is detected, and all ICMP packets from that user are immediately discarded. (1) Modify the per-user rate limit threshold and attack threshold for nfpp's icmp-guard, along with the per-port rate limit threshold and attack threshold, and enable isolation functionality. Hostname> enable Hostname(config)# nfpp Hostname(config-nfpp)# icmp-guard rate-limit per-src-ip 2000 Hostname(config-nfpp)# icmp-guard attack-threshold per-src-ip 2500 Hostname(config-nfpp)# icmp-guard rate-limit per-port 4000 Hostname(config-nfpp)# icmp-guard attack-threshold per-port 4500 Hostname(config-nfpp)# icmp-guard isolate-period 600 (2) View the ICMP Guard rate limiting threshold and attack threshold for nfpp, along with the isolation feature's enabled status. Hostname> enable Hostname# show nfpp icmp-guard summary (Format of column Rate-limit and Attack-threshold is per-src-ip/per-src-mac/per-port.) Interface Status Isolate-period Rate-limit Attack-threshold Global Enable 600 2000/-/4000 2500/-/4500 Maximum count of monitored hosts: 20000 Monitor period: 600s (3) View the attack ports or attack user information detected by nfpp's ICMP Guard. Hostname> enable Hostname# show nfpp icmp-guard host If col_filter 1 shows '*', it means "hardware do not isolate host". VLAN interface IP address remain-time(s) Total: 0 host In addition to ICMP Guard, NFPP also includes ARP Guard, DHCP Guard, DHCPv6 Guard, IP Guard, ND Guard, and TCP SYN Guard, enabling attack attribution and automatic protection for other specified protocol packets. 4.2 Routing Protocol Security 4.2.1 OSPFv2 1. Security Threats Fake Packet Attacks The primary attack method targeting OSPFv2 on networks involves forging packets. Attackers may fabricate packet information through the following means: Modifying packet aging time to the maximum value, causing all devices to flood the packet. Publishing LSA packets with serial numbers set to the maximum or near-maximum value. Manipulating the timing of neighbor devices' encryption sequence number reset during reboots to alter sequence numbers. Altering the neighbor list within Hello messages. Injection of incorrect routing information OSPFv2 accepts all packets from legitimate devices. Therefore, illegal or incorrect routing information carried in OSPFv2 packets may be used to attack the switch. Such information can cause routing database calculation errors and lead to network failures. 2. Security Hardening Strategies OSPFv2 Packet Authentication OSPFv2 supports packet authentication functionality. Packets sent after authentication configuration will carry authentication information. Matching authentication mode and password are prerequisites for establishing neighbors normally. Only authenticated packets can be accepted. OSPFv2 packet authentication prevents unauthorized routers accessing the network and hosts forging OSPFv2 packets from participating in the OSPFv2 protocol process, ensuring the stability and intrusion resistance of the OSPFv2 protocol. OSPFv2 supports three authentication modes. Packet verification is enabled only when the authentication mode matches the configured password: Type 0: Authentication not required. When OSPFv2 authentication is not enabled via configuration, packets carry authentication type 0. Type 1: Plaintext authentication mode. Type 2: Encrypted authentication mode. OSPFv2 packet authentication supports area authentication and interface authentication. By default, both area and interface authentication are disabled. Configure at least one of these options. Area authentication enhances OSPFv2 area security, while interface authentication improves OSPFv2 neighbor communication security. When both are configured, the interface-level authentication settings take precedence. Configure OSPFv2 area authentication using one of the following methods: Configure area plaintext authentication Configure area MD5 authentication Configure area keychain authentication Configure OSPFv2 interface authentication using one of the following methods: Configure interface plaintext authentication Configure interface MD5 authentication Configure interface keychain authentication 3. Configure regional plaintext authentication (1) Configure the authentication mode of area 0 in the OSPFv2 process 1 as plaintext authentication (both 1 and 0 are examples) Hostname> enable Hostname# configure terminal Hostname(config)# router ospf 1 Hostname(config-router)# area 0 authentication Hostname(config-router)# exit (2) Configure the plaintext authentication key on the interface GigabitEthernet 0/1 that joins this area as areaauth (both GigabitEthernet 0/1 and ospfauth are examples). Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ip ospf authentication-key areaauth Hostname(config-if-GigabitEthernet 0/1)# end (3) Review the OSPFv2 information summary. If the output contains the field “Area has simple password authentication,” it indicates that plaintext authentication is enabled for the area. Hostname# show ip ospf | begin Area 0 Area 0 (BACKBONE) (Inactive) Number of interfaces in this area is 0(0) Number of fully adjacent neighbors in this area is 0 Area has simple password authentication SPF algorithm last executed 00:22:53.345 ago SPF algorithm executed 3 times iSPF algorithm executed 0 times Number of LSA 0. Checksum 0x000000 4. Configuring MD5 Authentication for an OSPF Area (1) Configure the authentication mode of Area 0 in OSPFv2 process 1 as message-digest (MD5) authentication. (All parameters in this example are for demonstration purposes only.) Hostname> enable Hostname# configure terminal Hostname(config)# router ospf 1 Hostname(config-router)# area 0 authentication message-digest (2) Configure MD5 authentication for OSPFv2 packets on a Layer 3 Ethernet interface. The authentication key ID is set to 1, and the corresponding key string is areaauth. (All parameters in this example are for demonstration purposes only.) Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ip ospf message-digest-key 1 md5 areaauth 5. Configure Area Keychain Authentication (1) Create a Keychain for OSPF Area Authentication In this example, a keychain named ospfkey is created. When the key ID is 1, the corresponding authentication key is areaauth (ospfkey, 1, and areaauth are example values). Hostname> enable Hostname# configure terminal Hostname(config)# key chain ospfkey Hostname(config-keychain)# key 1 Hostname(config-keychain-key)# key-string areaauth Hostname(config-router)# end (2) Enable Area Keychain Authentication OSPFv2 supports Keychain-based authentication in either plain-text or MD5 mode. The following examples demonstrate both options. Choose one according to your security requirements. Configure plain-text Keychain authentication for Area 0 in OSPF process 1 (The OSPF process ID 1, Area 0, and Keychain name ospfkey are examples.) Hostname# configure terminal Hostname(config)# router ospf 1 Hostname(config-router)# area 0 authentication keychain ospfkey Configure MD5 Keychain authentication for Area 0 in OSPF process 1 (The OSPF process ID 1, Area 0, and Keychain name ospfkey are examples.) Hostname(config)# router ospf 1 Hostname(config-router)# area 0 authentication keychain ospfkey Hostname(config-router)# area 0 authentication message-digest 6. Configure Interface Plain-Text Authentication (1) Enable plain-text authentication on an OSPF-enabled Layer 3 interface On the Layer 3 Ethernet interface GigabitEthernet 0/1, which already has OSPFv2 enabled, configure the interface to use plain-text authentication with the key intauth (GigabitEthernet 0/1 and intauth are example values). Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ip ospf authentication Hostname(config-if-GigabitEthernet 0/1)# ip ospf authentication-key intauth 7. Configure Interface MD5 Authentication (1) Enable MD5 authentication on an OSPF-enabled Layer 3 interface On the Layer 3 Ethernet interface GigabitEthernet 0/1, configure MD5-based authentication. When the key ID is 1, the authentication key is intauth, encrypted using the MD5 algorithm (all parameters are examples). Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ip ospf authentication message-digest Hostname(config-if-GigabitEthernet 0/1)# ip ospf message-digest-key 1 md5 intauth 8. Configure Interface Keychain Authentication (1) Enable Keychain authentication on an OSPF-enabled interface On the Layer 3 Ethernet interface GigabitEthernet 0/1, configure the authentication mode as Keychain-based authentication, using the Keychain named ospfkey (GigabitEthernet 0/1 and ospfkey are example values). Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ip ospf authentication keychain ospfkey Hostname(config-if-GigabitEthernet 0/1)# exit (2) Configure the Keychain used by the interface Create a Keychain named ospfkey. When the key ID is 1, the corresponding authentication key is intauth (ospfkey, 1, and intauth are example values). Hostname(config)# key chain ospfkey Hostname(config-keychain)# key 1 Hostname(config-keychain-key)# key-string intauth Hostname(config-keychain-key)# end 9. Operational Recommendation Based on typical service deployment scenarios, it is recommended to use MD5 authentication to enhance the security of the OSPF protocol and prevent unauthorized OSPF neighbor establishment. 4.2.2 OSPFv3 1. Security Threats Spoofed Packet Attacks In operational networks, the primary attack vector against OSPFv3 is spoofed OSPF packets. An attacker may forge OSPFv3 packets using several techniques, including: Modifying the LSA age to the maximum age, causing all devices to flood the LSA. Advertising LSAs with maximum or near-maximum sequence numbers. Exploiting the moment when a neighboring device restarts and resets its encryption sequence state to manipulate sequence numbers. Tampering with the neighbor list in Hello packets. Injection of Incorrect Routing Information OSPFv3 accepts all packets received from legitimate neighbors. If OSPFv3 packets carry malicious or incorrect routing information, the switch may process invalid LSAs, leading to routing database inconsistencies and potential network outages. 2. Security Hardening Strategies OSPFv3 Packet Authentication OSPFv3 supports packet authentication. Once authentication is configured, all transmitted packets carry authentication information, and only authenticated packets are accepted. Authentication prevents unauthorized routers and hosts from participating in the OSPFv3 process and forging OSPFv3 packets, thereby improving protocol stability and resistance to attacks. OSPFv3 supports the following authentication modes. Authentication succeeds only when both the mode and password match: No authentication (default) MD5 authentication SHA1 authentication OSPFv3 packet authentication can be applied at either the area level or the interface level. By default, both are disabled. With area authentication, all devices within the same area must use identical authentication parameters. Interface authentication is configured between adjacent devices and takes precedence over area authentication. When both are configured, interface authentication overrides area authentication. Authentication can be configured using at least one of the following approaches: Area authentication Interface authentication OSPFv3 Encrypted Authentication OSPFv3 also supports encrypted authentication, which encrypts OSPFv3 packets in addition to authenticating them. Only packets that are successfully decrypted and authenticated are accepted; otherwise, OSPF neighbors cannot be established. Supported encryption algorithms include: DES 3DES AES-CBC Encrypted authentication can be applied at either the area level or the interface level, and both are disabled by default. Area encryption requires all devices in the same area to use identical parameters. Interface encryption applies between neighbors and has higher priority than area encryption. Encrypted authentication can be configured using: Area-level encryption authentication Interface-level encryption authentication 3. Configure Area Authentication (1) Configure MD5 authentication for Area 1 in OSPFv3 process 1 The Security Parameter Index (SPI) is 300, and the authentication password is aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa (all values are examples). Hostname> enable Hostname# configure terminal Hostname(config)# ipv6 router ospf 1 Hostname(config-router)# area 1 authentication ipsec spi 300 md5 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 4. Configure Interface Authentication (1) Enable MD5 authentication on an OSPFv3-enabled Layer 3 interface On GigabitEthernet 0/1, configure OSPFv3 MD5 authentication with SPI 300 and password aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa (example values). Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ipv6 ospf authentication ipsec spi 300 md5 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 5. Configure Area Encrypted Authentication (1) Enable encrypted authentication for Area 1 SPI: 300 Encryption: DES Encryption key: 0123456789abcdef Authentication: MD5 Authentication password: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Hostname> enable Hostname# configure terminal Hostname(config)# ipv6 router ospf 1 Hostname(config-router)# area 1 encryption ipsec spi 300 esp des 0123456789abcdef md5 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 6. Configure Interface Encrypted Authentication (1) Enable encrypted authentication on an OSPFv3-enabled interface Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ipv6 ospf encryption ipsec spi 300 esp des 0123456789abcdef md5 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 7. Operational Recommendation Based on service deployment requirements, MD5 authentication is recommended to enhance OSPFv3 security. 4.2.3 BGP4 / BGP4+ 1. Security Threats Spoofed Packet Attacks Attackers may capture legitimate TCP packets carrying BGP messages and craft forged packets. Although authentication can detect and discard such packets, excessive forged traffic may still impact processing performance and network stability. 2. Security Hardening Strategies BGP4/BGP4+ MD5 Authentication BGP/BGP4+ MD5 authentication is an encryption method implemented to meet network security requirements. It encrypts BGP messages by adding an authentication field within the TCP segment carrying the BGP message. When a local device receives a TCP segment from a remote device containing a BGP message, it will discard the segment if the authentication password does not match, thereby providing self-protection. For configuration details, please refer to Configuring BGP4/BGP4+ MD5 Authentication. BGP GTSM Security Check The BGP GTSM (Generalized TTL Security Mechanism) security check verifies whether the TTL value in the IP header is within a pre-specified range. If the TTL value falls outside this range, the packet is considered invalid, and the device will discard it directly. For configuration details, please refer to Configuring BGP GTSM Security Check. AS_PATH Count Limit Configuring an AS_PATH count limit in BGP allows the device to check whether the number of ASes in the received route’s AS_PATH attribute exceeds a specified maximum. If the count exceeds the limit, the route will be discarded to prevent attacks using maliciously crafted packets with overly long AS_PATH attributes. For configuration details, please refer to Configuring AS_PATH Count Limit. 3. Configure BGP4/BGP 4+ MD5 Authentication (1) Configure BGP process 65530 with neighbor 1.1.1.1 and remote AS 1 (65530, 1.1.1.1, and 1 are examples). Hostname> enable Hostname# configure terminal Hostname(config)# router bgp 65530 Hostname(config-router)# neighbor 1.1.1.1 remote-as 1 (2) Configure MD5 authentication for neighbor 1.1.1.1 using plaintext password test (1.1.1.1, plaintext, and test are examples). Hostname(config-router)# neighbor 1.1.1.1 password 0 test (3) Verify whether the configuration is effective. Hostname(config)# sh running-config router bgp inc password 4. Configure BGP GTSM Security Check (1) Configure BGP process 65530 with neighbor 1.1.1.1 and remote AS 1 (65530, 1.1.1.1, and 1 are examples). Hostname> enable Hostname# configure terminal Hostname(config)# router bgp 65530 Hostname(config-router)# neighbor 1.1.1.1 remote-as 1 (2) Configure GTSM security check for neighbor 1.1.1.1, limiting the maximum number of hops from the local device to this neighbor to 6 (1.1.1.1 and 6 are examples). Hostname(config-router)# neighbor 1.1.1.1 ttl-security hops 6 (3) Verify whether the configuration is effective. Hostname(config)# sh running-config router bgp inc ttl-security 5. Configure AS_PATH Count Limit (1) Configure BGP process 65530 with neighbor 1.1.1.1 and remote AS 1 (65530, 1.1.1.1, and 1 are examples). Hostname> enable Hostname# configure terminal Hostname(config)# router bgp 65530 Hostname(config-router)# neighbor 1.1.1.1 remote-as 1 (2) Configure a limit of 300 AS numbers in the AS_PATH attribute for BGP routes (300 is an example). Hostname(config-router)# bgp maxas-limit 300 (3) Verify whether the configuration is effective. Hostname(config)# sh running-config router bgp inc maxas-limit 6. Operational Recommendation Based on deployment requirements, MD5 authentication is recommended to enhance TCP session and BGP protocol security. 4.3 Multicast Protocols 4.3.1 Layer 3 Multicast 1. Security Threats Malicious Hello Messages Multicast devices establish PIM neighbor relationships through Hello messages. When a large number of malicious Hello messages exist, it may cause an excessive number of PIM neighbors, leading to high memory usage or CPU utilization on the router, which can compromise the security of multicast services. 2. Security Hardening Strategies PIM Neighbor Filtering When a large number of malicious Hello messages are detected, ACLs can be configured on interfaces to allow only specified Hello messages to pass, effectively filtering out malicious Hello messages. For detailed configuration, please refer to Configuring PIM Neighbor Filtering. MSDP MD5 Authentication Security can be enhanced by configuring MD5 authentication on MSDP peers. Both sides of the peer connection must enable MD5 authentication with the same password. After enabling this feature, MSDP messages sent from the sending peer are encrypted with MD5 and transmitted via TCP to the receiving peer. The receiving peer decrypts the MSDP message using the shared MD5 key and processes it through the MSDP module only if authentication is successful. This prevents malicious MSDP messages from being processed. For detailed configuration, please refer to Configuring MSDP MD5 Authentication. 3. Configure PIM Neighbor Filtering (1) Create standard IP ACL 1 and add a rule to permit packets with source IP addresses from 192.168.1.64 to 192.168.1.127 (ACL 1, 192.168.1.64, and 192.168.1.127 are examples). Hostname> enable Hostname# configure terminal Hostname(config)# access-list 1 permit 192.168.1.64 0.0.0.63 (2) Apply ACL 1 to interface GigabitEthernet 0/1 (GigabitEthernet 0/1 is an example). Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ip pim neighbor-filter 1 (3) Verify that the filtering rule is effective. Hostname(config)# show running-config | include pim ip pim neighbor-filter 1 4. Configure MSDP MD5 Authentication (1) Create an MSDP peer and establish a TCP connection through Loopback 0 (Loopback 0 and 192.168.5.1 are examples). Hostname> enable Hostname# configure terminal Hostname(config)# interface loopback 0 Hostname(config-if-Loopback 0)# exit Hostname(config)# ip msdp peer 192.168.5.1 connect-source loopback 0 (2) Configure the MD5 password on the peer 192.168.5.1 (peer 192.168.5.1 and password aaa are examples). Hostname(config)# ip msdp password peer 192.168.5.1 aaa 5. Configuration and Maintenance Recommendation Based on your deployment, it is recommended to enable MD5 authentication for MSDP to enhance the security of TCP connections. 4.3.2 Lay 2 Multicast 1. Security Threats Denial-of-Service (DoS) Attacks Malicious users may join invalid multicast group channels by altering group addresses, causing a large number of invalid entries on the device. This consumes system resources and prevents legitimate users from successfully accessing multicast services. 2. Security Hardening Strategies Restrict Multicast Group Range By configuring multicast group policies and applying them on interfaces, the allowed range of multicast groups for users can be specified. This prevents DoS attacks caused by joining excessive invalid multicast groups. 3. Configure Multicast Group Policy (1) Create a group policy Profile 1 (Profile 1 is an example). Hostname> enable Hostname# configure terminal Hostname(config)# ip igmp profile 1 (2) Set the multicast group address range for Profile 1 to 224.2.2.2–224.2.2.244 (Profile 1, 224.2.2.2, and 224.2.2.244 are examples). Hostname (config-profile)# range 224.2.2.2 224.2.2.244 (3) Configure the filtering action for the group addresses in Profile 1. Select one of the following options as an example: #Permit packets within the multicast group address range. Hostname(config-profile)# permit #Deny packets within the multicast group address range. Hostname(config-profile)# deny (4) Apply group policy Profile 1 to interface GigabitEthernet 0/1 (GigabitEthernet 0/1 is an example). Hostname(config-profile)# exit Hostname(config)# interface gigabitEthernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# ip igmp snooping filter 1 4. Configuration and Maintenance Recommendation Based on your deployment, it is recommended to configure a valid multicast group address range for IPTV services to prevent excessive joining of invalid multicast groups. 4.4 NTP Security 4.4.1 Security Threats Unknown Service Packets NTP packets are transmitted across the network to synchronize device time. These packets may be maliciously tampered with. By crafting unknown service packets, attackers can prevent successful time synchronization or cause incorrect time settings, affecting device time stability. 4.4.2 Security Hardening Strategies NTP Packet Authentication NTP authentication is an encryption method implemented to meet network security requirements. It encrypts NTP packets by adding authentication fields. When a local device receives an NTP packet from a remote device, it discards the packet if the authentication key does not match, providing self-protection. Enable ACLs NTP supports ACL-based access control. By associating ACLs, a controllable security measure is provided for NTP. 4.4.3 Configure NTP Authentication Configure the NTP authentication ID as 6, set the global authentication key ntpauth using HMAC-SHA256, and designate it as the global trusted key. Then enable the authentication mechanism (6 and ntpauth are examples). Hostname> enable Hostname# configure terminal Hostname(config)# ntp authentication-key 6 hmac-sha256 ntpauth Hostname(config)# ntp trusted-key 6 Hostname(config)# ntp authenticate 4.4.4 Configure ACL-Based Access Configure the ACL to allow only the device 192.168.1.1 to request time and perform control queries on the local NTP service (1 and 192.168.1.1 are examples). Hostname> enable Hostname# configure terminal Hostname(config)# access-list 1 permit 192.168.1.1 Hostname(config)# ntp access-group peer 1 4.4.5 Configuration and Maintenance Recommendation ACLs must be configured separately using ACL commands, and filtering rules must be set for them to take effect. 4.5 MSTP Attack Prevention 4.5.1 Security Threats BPDU Attack Packets Sending malicious BPDU packets can cause changes in the spanning tree topology, potentially creating loops in the network. TC Packet Attacks TC packet attacks can frequently trigger deletion of MAC address table entries, causing network instability. Link Disruption Attacks By disrupting links, ports may fail to receive normal BPDU packets, which can also lead to potential loops. 4.5.2 Security Hardening Strategies BPDU Guard To prevent BPDU attacks from causing abnormal spanning tree topology changes, BPDU Guard can be enabled on specific ports. Ports with BPDU Guard enabled will enter an error state if a BPDU packet is received. For detailed configuration, see Configuring BPDU Guard. By default, BPDU Guard is disabled on interfaces. BPDU Packet Filtering To prevent abnormal BPDU packets from affecting the spanning tree topology, BPDU packet filtering can be configured on interfaces to drop such packets. For detailed configuration, see Configuring BPDU Packet Filtering. By default, BPDU filtering is disabled on interfaces. TC Packet Attack Prevention a. When TC Protection is enabled globally, after receiving a TC packet, only one deletion operation is performed within a certain time (typically 4 seconds), avoiding frequent deletion of MAC and ARP table entries. b. When TC Guard is enabled on a port, TC packets received on or generated by that port are blocked from propagating to other ports, effectively limiting TC attacks and maintaining network stability. c. TC Filtering ignores TC packets on a port but still allows normal topology changes to be processed. For detailed configuration, see Configuring TC Attack Prevention. ROOT Guard To prevent a root bridge from losing its position due to misconfiguration or malicious attacks, ROOT Guard can be configured on designated ports. For detailed configuration, see Configuring ROOT Guard. Loop Protection To prevent potential loops caused by ports (root, master, or alternate ports) not receiving BPDU packets from the designated bridge, Loop Protection can be configured on these ports to improve device stability. For detailed configuration, see Configuring Loop Protection. 4.5.3 Configure BPDU Guard (1) Enable BPDU Guard on Layer 2 Ethernet interface GigabitEthernet 0/1. Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# spanning-tree bpduguard enable (2) Verify that BPDU Guard is effective on GigabitEthernet 0/1. Hostname> enable Hostname# show running-config interface gigabitEthernet 0/1 spanning-tree bpduguard enable 4.5.4 Configure BPDU Packet Filtering (1) Enable BPDU packet filtering on Layer 2 Ethernet interface GigabitEthernet 0/1. Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# spanning-tree bpdufilter enable (2) Verify that BPDU packet filtering is effective on GigabitEthernet 0/1. Hostname> enable Hostname# show running-config interface gigabitEthernet 0/1 spanning-tree bpdufilter enable 4.5.5 Configure TC Packet Attack Prevention (1) Enable TC Protection in global configuration mode. Hostname> enable Hostname# configure terminal Hostname(config)# spanning-tree tc-protection (2) Enable TC Guard on Layer 2 Ethernet interface GigabitEthernet 0/1. Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# spanning-tree tc-guard (3) Enable TC packet filtering on GigabitEthernet 0/1. Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# spanning-tree tc-guar (4) Verify that TC Guard is effective on GigabitEthernet 0/1. Hostname> enable Hostname# show running-config interface gigabitEthernet 0/1 spanning-tree tc-guard (5) Verify that TC packet filtering is effective on GigabitEthernet 0/1. Hostname> enable Hostname# show running-config interface gigabitEthernet 0/1 spanning-tree ignore tc 4.5.6 Configure ROOT Guard (1) Enable ROOT Guard on Layer 2 Ethernet interface GigabitEthernet 0/1. Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# spanning-tree guard root (2) Verify that ROOT Guard is effective on GigabitEthernet 0/1. Hostname> enable Hostname# show running-config interface gigabitEthernet 0/1 spanning-tree guard root 4.5.7 Configure Loop Protection (1) Enable Loop Protection on Layer 2 Ethernet interface GigabitEthernet 0/1. Hostname> enable Hostname# configure terminal Hostname(config)# interface gigabitethernet 0/1 Hostname(config-if-GigabitEthernet 0/1)# spanning-tree guard loop (2) Verify that Loop Protection is effective on GigabitEthernet 0/1. Hostname> enable Hostname# show running-config interface gigabitEthernet 0/1 spanning-tree guard loop 5 Forwarding Plane Security 5.1 Local Attack Prevention 5.1.1 Security Threats Malicious Forwarding Plane Packets In the network, besides many malicious packets targeting the device CPU, there are also attacks targeting the forwarding plane. Some types of packets are copied to the CPU after entering a port, while the original packet continues to be forwarded. If such packets are malicious attack packets, forwarding them can cause attacks on other devices. 5.1.2 Security Hardening Strategies Forwarding Plane Packet Rate Limiting For packets that are both sent to the CPU and forwarded, attack detection is performed. When identified as attack traffic, rate limiting can be applied both when sending to the CPU and during forwarding. Forwarding Plane Packet Isolation For packets that are both sent to the CPU and forwarded, attack detection is performed. When identified as attack traffic, the traffic can be isolated separately, without affecting the forwarding or CPU processing of normal traffic. 5.1.3 Configure Forwarding Plane Rate Limiting Improper configuration of forwarding plane rate limiting may affect network services. If adjustments are needed, it is recommended to contact our technical support engineers. NFPP provides features such as arp-guard, icmp-guard,dhcp-guard, dhcpv6-guard, ip-guard, nd-guard和tcp-syn-guard. Only some products support forwarding plane rate limiting. Taking arp-guard as an example, modify the forwarding plane rate limiting configuration as follows: (1) Enable forwarding plane rate limiting in arp-guard configuration. Hostname> enable Hostname(config)# nfpp Hostname(config-nfpp)# arp-guard ratelimit-forwarding enable (2) Verify changes in packet forwarding rates: Hostname> enable Hostname# show interfaces counters rate up Interface Sampling Time Input Rate Input Rate Output Rate Output Rate (bits/sec) (packets/sec) (bits/sec) (packets/sec) 5.1.4 Configure Forwarding Plane Isolation Improper configuration of forwarding plane isolation may affect network services. If adjustments are needed, it is recommended to contact our technical support engineers. NFPP provides features such as arp-guard, icmp-guard,dhcp-guard, dhcpv6-guard, ip-guard, nd-guard和tcp-syn-guard. Only some products support forwarding plane isolation. Taking arp-guard as an example, modify the forwarding plane isolation configuration as follows: (1) Enable forwarding plane isolation in arp-guard configuration. Hostname> enable Hostname(config)# nfpp Hostname(config-nfpp)# arp-guard isolate-forwarding enable (2) Verify changes in packet forwarding rates: Hostname> enable Hostname# show interfaces counters rate up Interface Sampling Time Input Rate Input Rate Output Rate Output Rate (bits/sec) (packets/sec) (bits/sec) (packets/sec) 6 Management Plane Security 6.1 Default Accounts and Passwords 6.1.1 Security Threats If the device has default accounts and passwords, attackers may obtain them through other means and perform unauthorized operations on the device. 6.1.2 Security Hardening Strategy The device does not support default accounts and passwords. When a user logs in through the console for the first time, they are forced to set an administrative password, and no default account or password is created 6.2 Account and Password Security 6.2.1 Security Threats Brute-force password attacks When connecting to the device, authentication prompts for username and password appear. Brute-force attacks occur when an attacker, without knowing the correct credentials, guesses a password range and attempts all possible passwords to gain access. 6.2.2 Security Hardening Strategies Configure password policy Enhances password complexity to increase difficulty of brute-force attacks: Minimum password length: default minimum is 8 characters. Strong password check: at least three of the following required by default — numbers, uppercase letters, lowercase letters, special characters. Weak password enforcement: default enabled; if the password does not meet requirements, user is forced to change it. Password lifetime: default unlimited; once expired, users are prompted to reset password. Restrict password reuse: default allows reuse; can restrict to last 5 passwords. Encrypted password storage: passwords stored in encrypted form by default. Configure authentication failure silencing Console: lock for 10 seconds on failure. User: max 3 failures for privilege escalation; reset counter every 10 minutes. Telnet: max 6 failures; reset every 5 minutes; IP silence wake-up period 10 minutes. SSH: max 6 failures; reset every 5 minutes; silence wake-up 10 minutes. SNMP: max 3 failures; lock 10 minutes after consecutive failures. FTP: max 3 failures; lock 10 minutes after consecutive failures. AAA: max 3 failures; lock 10 minutes after consecutive failures. 6.2.3 Configure Password Lifetime Set password lifetime to 90 days. Hostname> enable Hostname# configure terminal Hostname(config)# password policy life-cycle 90 6.2.4 Configure Password Reuse Restriction Restrict reuse of the last 5 passwords. Hostname> enable Hostname# configure terminal Hostname(config)# password policy no-repeat-times 5 6.2.5 Configuration Maintenance Recommendation The device’s default behavior aligns with recommended security settings; it is advised not to disable related security configurations. 6.3 Console Login 6.3.1 Security Threats Device exposure: Without console security, attackers can access the device through the physical port. System privilege compromise: Attackers may attempt to guess usernames and passwords via console. 6.3.2 Security Hardening Strategy Configure local user authentication for console access. Configure AAA authentication; console supports password or AAA authentication for first-time setup. 6.3.3 Configure Local User Authentication Hostname> enable Hostname# configure terminal Hostname(config)# line console 0 Hostname(config-line)# login local Hostname(config-line)# exit Hostname(config)# username test privilege 15 password 0 pw15 6.3.4 Configure AAA Authentication Hostname> enable Hostname# configure terminal Hostname(config)# aaa new-model Hostname(config)# aaa authentication login list local Hostname(config)# line console 0 Hostname(config-line)# login authentication list 6.3.5 Configuration Maintenance Recommendation Ensure proper console authentication using local or AAA credentials. Password-only authentication is not secure. 6.4 Telnet Login 6.4.1 Security Threats Brute-Force Password Attacks An attacker repeatedly attempts to establish a connection. When the device prompts for authentication, the attacker performs brute-force attempts to pass authentication and obtain access privileges. Denial-of-Service (DoS) Attacks The Telnet Server supports a limited number of concurrent users. When the maximum number of logged-in users is reached, additional users cannot log in. This may be caused by normal usage, or it may be the result of an attacker intentionally exhausting available sessions. Port Scanning The attacker scans and listens to the device’s network ports in an attempt to capture user interaction traffic. Because Telnet transmits data in clear text, device information may be intercepted and stolen. 6.4.2 Security Hardening Strategies Configure Authentication Methods The Telnet Server supports username/password authentication. Only users who pass authentication are allowed to log in to the device and access the command-line interface (CLI). Disable the Telnet Service When the Telnet Server is enabled, the device exposes the Telnet service and can be easily discovered by attackers through scanning. When the Telnet Server is not required, it should be disabled. By default, the Telnet Server service is disabled. Configure ACL-Based Access Control In global configuration mode, ACL filtering rules can be applied to individual VTY lines to control which client IP addresses are permitted to log in. Additionally, an ACL can be associated directly with the Telnet server to restrict login access based on client IP addresses. 6.4.3 Configure Authentication Method (1) Configure the authentication method by applying parameters from an authentication list. Hostname> enable Hostname# configure terminal Hostname(config)# line vty 0 Hostname(config-line)# login authentication list 6.4.4 Configure Disable the Telnet Service (1) Disable the Telnet service. Hostname> enable Hostname# configure terminal Hostname(config)# no enable service telnet-server 6.4.5 Configure ACL-Based Access Control (1) Create a standard IP ACL and add rules to permit traffic from source IP addresses ranging from 192.168.1.64 to 192.168.1.127 (all parameters are examples). Hostname> enable Hostname# configure terminal Hostname(config)# ip access-list 1 permit 192.168.1.64 0.0.0.63 (2) Apply ACL-based filtering rules to restrict outbound traffic using access list abc (interfaces 0, access list name abc, and direction out are examples). (If Step 3 is configured, this step can be omitted.) Hostname> enable Hostname# configure terminal Hostname(config)# line vty 0 Hostname(config-line)# access-class abc out (3) Apply ACL-based filtering rules (access-list 1) to restrict client IP addresses allowed to access the Telnet Server. Hostname> enable Hostname# configure terminal Hostname(config)# ip telnet access-class 1 6.4.6 Configuration Maintenance Recommendation Use dedicated management IPs. Use local or AAA authentication for VTY. Configure ACLs to restrict Telnet access. 6.5 SSH Login 6.5.1 Security Threats Brute-Force Password Attacks When connecting to the device, the system prompts for username and password authentication. A brute-force password attack refers to an attacker attempting to determine the correct credentials without knowing them in advance by estimating a possible password range based on password policies, and then systematically testing passwords within that range until the password is compromised. Denial-of-Service (DoS) Attacks The SSH server supports a limited number of concurrent login users. Once the maximum number of sessions is reached, additional users cannot log in. An attacker can exploit this limitation to launch a denial-of-service attack against the device’s SSH service. 6.5.2 Security Hardening Strategies Configure Login Authentication Users are required to authenticate when logging into the device. The device supports password-based authentication and public key authentication. Only users who successfully pass authentication are permitted to log in and access the command-line interface (CLI). Disable SSHv1 By default, SSHv1 is disabled. After the SSH service is enabled, only SSHv2 is supported. The algorithms supported by the SSHv1 protocol are known to have security vulnerabilities and provide a lower level of security. It is recommended to use SSHv2 when connecting to the device. Configure ACL-Based Access Control The device supports associating ACL filtering rules with the SSH service, allowing only clients within a specified IP address range to log in. Configure the SSH Login Port By default, the SSH server listens on port 22, which is easily scanned and targeted by attackers. The SSH server port can be changed to a non-well-known port to reduce the likelihood of scanning and attack attempts. 6.5.3 Configure Login Authentication (1) #Enable local login on VTY 0–35 Hostname> enable Hostname# configure terminal Hostname(config)# line vty 0 35 Hostname(config-line)# login local 6.5.4 Disable SSH Service (1) Disable SSH Hostname> enable Hostname# configure terminal Hostname(config)# no enable service ssh-server (2) Check SSH service status Hostname> enable Hostname# show service snmp-agent : disabled ssh-server : enabled telnet-server : enabled 6.5.5 Configure ACL-based Access (1) Create standard IP ACL 1, allow 192.168.1.64–192.168.1.127 Hostname> enable Hostname# configure terminal Hostname(config)# access-list 1 permit 192.168.1.64 0.0.0.63 (2) Associate ACL 1 to SSH access Hostname> enable Hostname# configure terminal Hostname(config)# ip ssh access-class 1 6.5.6 Configure SSH Login Port (1) Set SSH port to 1300 Hostname> enable Hostname# configure terminal Hostname(config)# ip ssh port 1300 (2) Check SSH port Hostname> enable Hostname# configure terminal Hostname# show ip ssh SSH Enable - version 2.0 SSH Port: 22 SSH Cipher Mode: ctr,gcm SSH HMAC Algorithm: sha2-256,sha2-512 Authentication timeout: 120 secs Authentication retries: 3 SSH SCP Server: disabled SSH dh-exchange min-len: 2048 SSH SFTP Server: disabled SSH ip-block: enabled 6.5.7 Configuration Maintenance Recommendation Use dedicated management IPs. Configure VTY authentication with local or AAA. Restrict SSH access via ACL. 6.6 NETCONF Management 6.6.1 Security Threats The operation of the NETCONF protocol relies on SSH to provide a secure transport channel, enabling interaction between the NETCONF client and the NETCONF server. Therefore, the security threats faced by NETCONF are essentially the same as those associated with the SSH protocol. The main security threats include the following: Brute-Force Password Attacks An attacker attempts to establish a connection. When the device prompts for authentication, the attacker performs brute-force attempts to bypass authentication and obtain access privileges. Denial-of-Service (DoS) Attacks The NETCONF server supports a limited number of concurrent login users. Once the maximum number of sessions is reached, additional users cannot log in. An attacker can exploit this limitation to launch a denial-of-service attack against the device’s NETCONF service. Authentication Disabled by Default Authentication is disabled by default, allowing users to log in with full operational privileges. 6.6.2 Security Hardening Strategies Enable Authentication Users must be authenticated when logging in to the device. The device supports password-based authentication and public key authentication. Only authenticated users are allowed to log in and access the command-line interface (CLI). Enable ACLs ACL filtering rules can be associated with the NETCONF service to allow login access only from client IP addresses within a specified range. Change the Port Number By default, the NETCONF server listens on port 830, which is easily scanned and targeted by attackers. The NETCONF server port can be changed to a non-well-known port to reduce the likelihood of scanning and attack attempts. Enable NETCONF ACM Configuration By default, NETCONF access control is not enabled, which may allow users with lower privileges to perform unauthorized operations. NETCONF Access Control Model (ACM) can be configured so that only authorized users are permitted to issue NETCONF requests. 6.6.3 Configure Login Authentication Enable local login on VTY 0–35 Hostname> enable Hostname# configure terminal Hostname(config)# line vty 0 35 Hostname(config-line)# login local Hostname(config-line)# exit 6.6.4 Configure ACL-based Access (1) Create standard IP ACL 1, allow 192.168.1.64–192.168.1.127 Hostname> enable Hostname# configure terminal Hostname(config)# access-list 1 permit 192.168.1.64 0.0.0.63 (2) Associate ACL 1 to NETCONF Hostname> enable Hostname# configure terminal Hostname(config)# netconf access-class ipv4 1 6.6.5 Configure NETCONF Port Change Set NETCONF server port to 5000 Hostname> enable Hostname# configure terminal Hostname(config)# netconf port 5000 6.6.6 NETCONF ACM Configuration Configure ACM: user "test" has no operational permissions Hostname> enable Hostname# configure terminal Hostname(config)#netconf acm Hostname(config-nacm)#user-group-name test Hostname(config-nacm-user-group)#user-name Hostname(config-nacm-user-group)#rule-list test fs Hostname(config-nacm-rule-list)#user-group test Hostname(config-nacm-rule-list)#rule fs action deny 6.6.7 Configuration Maintenance Recommendation Use dedicated management IPs. Configure VTY authentication local/AAA. Restrict NETCONF access via ACL. Configure NETCONF ACM permissions. 6.7 gRPC Management 6.7.1 Security Threats The gRPC protocol operates over HTTP/HTTPS as its transport channel. The main security threats faced by gRPC include the following: Brute-Force Password Attacks An attacker attempts to establish a connection. When the device prompts for authentication, the attacker performs brute-force attempts to bypass authentication and obtain access privileges. Denial-of-Service (DoS) Attacks The gRPC server has limited service capacity. When device performance reaches its limit, some sessions may fail to receive telemetry data. This condition may be caused by normal usage or may be intentionally triggered by an attacker. Port Scannin An attacker scans and listens to the device’s network ports in an attempt to capture user interaction traffic. If encrypted transport is not configured, gRPC uses HTTP for clear-text communication, which may result in device information being intercepted and stolen. Authentication Disabled by Default Authentication is disabled by default, allowing users to log in with full operational privileges. 6.7.2 Security Hardening Strategies Enable Authentication Users must be authenticated when logging in to the device. The device supports username/password authentication. Only authenticated users are allowed to log in to the device. Enable ACLs ACL filtering rules can be associated with the gRPC service to allow login access only from client IP addresses within a specified range. Configure HTTPS Transport By default, gRPC uses HTTP as the transport protocol. It can be configured to use HTTPS instead, enabling encryption of transmitted data and preventing data interception. 6.7.3 Configure Login Authentication gRPC local or AAA login authentication Hostname> enable Hostname# configure terminal Hostname(config)#grpc Hostname (config-grpc)#authen login local Configure the gRPC login authentication use AAA server authentication Hostname> enable Hostname# configure terminal Hostname(config)# aaa new-model Hostname(config)# aaa authentication login aaa-auth-list group tacacs+ Hostname(config)# grpc Hostname(config-grpc)# authen login authentication aaa-auth-list 6.7.4 Configure ACL-based Access (1) Create the standard IP ACL 1, allow 192.168.1.64–192.168.1.127 Hostname> enable Hostname# configure terminal Hostname(config)# access-list 1 permit 192.168.1.64 0.0.0.63 (2) Associate ACL 1 to gRPC Hostname> enable Hostname# configure terminal Hostname(config)#grpc Hostname(config-grpc)#server access-class ipv4 1 6.7.5 Configure HTTPS Transport Configure gRPC server certificate Hostname> enable Hostname# configure terminal Hostname(config)#grpc Hostname(config-grpc)#server certificate load pem test.ca private-key test.key 6.7.6 Configuration Maintenance Recommendation Use dedicated management IPs. Configure authentication (local/AAA). Restrict gRPC access via ACL. Enable gRPC transport encryption. 6.8 SNMP Management 6.8.1 Security Threats An attacker forges the source IP address of packets to obtain the privileges of an authorized user, thereby performing unauthorized management operations. An attacker intercepts communication between the management station and the SNMP agent to obtain information such as usernames, passwords, and community strings, resulting in unauthorized access. An attacker intercepts SNMP messages and performs reordering, delaying, or replaying of messages, disrupting normal operations until unauthorized access is obtained. 6.8.2 Security Hardening Strategies Disable SNMPv1/v2c By default, SNMPv1 and SNMPv2c are disabled. Data transmitted using SNMPv1/v2c cannot be encrypted and provides a low level of security. Configure SNMPv3 SNMPv3 introduces support for the User-based Security Model (USM) and provides authentication using MD5, SHA, and SHA2 algorithms, as well as encryption using DES and AES algorithms. By authenticating and encrypting management traffic, SNMPv3 addresses security risks such as message spoofing, tampering, and information disclosure. SNMPv3 is enabled by default. Because SNMPv3 offers higher security than SNMPv1/v2c, it is recommended to use SNMPv3 for device management. Enable Encrypted Storage For SNMPv3 users, authentication and encryption passwords are stored in encrypted form, effectively preventing password leakage from configuration files. For SNMPv1/v2c users, community strings and user passwords are stored in encrypted form, effectively preventing leakage of community strings and user credentials from configuration files. Enable ACLs ACL filtering rules can be associated with the SNMP service to allow access only from client IP addresses within a specified range. Change the Port Number By default, the SNMP server listens on port 161, which is easily scanned and targeted by attackers. The SNMP server port can be changed to a non-well-known port to reduce the likelihood of scanning and attack attempts. 6.8.3 Disable SNMPv1/v2c (1) Disable SNMPv1/v2c Hostname> enable Hostname# configure terminal Hostname(config)# no snmp-server enable version v1 Hostname(config)# no snmp-server enable version v2c 6.8.4 Configure SNMPv3 (1) Create ACL "snmp-acl", deny 192.168.1.1, allow 192.168.2.1 Hostname> enable Hostname# configure terminal Hostname(config)# ip access-list extended snmp-acl Hostname(config-ext-nacl)# deny udp 192.168.1.1 255.255.255.0 any Hostname(config-ext-nacl)# permit udp 192.168.2.1 255.255.255.0 any Hostname(config-ext-nacl)# exit (2) Create MIB view "view1", limit to subtree 1.3 Hostname(config)# snmp-server view view1 1.3 include (3) Create SNMP group "v3group", version v3, security level priv, read/write view1 Hostname(config)# snmp-server group v3group v3 auth read view1 write view1 (4) Create SNMP user "v3user", in group v3group, version v3, SHA auth, password authsnmp, AES128 encryption, password authsnmp Hostname(config)# snmp-server user v3user v3group v3 auth sha authsnmp priv aes128 authsnmp (5) Associate SNMP server with ACL "snmp-acl" Hostname> enable Hostname# configure terminal Hostname(config)# snmp-server ip access-class snmp-acl 6.8.5 Configuration Maintenance Recommendation Use dedicated management IPs. Configure VTY authentication local/AAA. Restrict SNMP access via ACL.

FSOS 12.5(1)B0508S13 Release Notes for N5860-48SC and N8560-48BC and N8560-64C and NC8200-4TD Switches

2026-01-27 - FSOS 12.5(1)B0508S13 Release Notes for N5860-48SC and N8560-48BC and N8560-64C and NC8200-4TD Switches Version Information Version Number N8560_FSOS 12.5(1)B0508S13 Products N8560-48BC、NC8200-4TD、N5860-48SC、N8560-64C Version Type Official Version Applicable Customers Data Center Customers Release Date 2025/5/7 Baseline Version N8560_FSOS 12.5(1)B0508S10 Use the show version command to view the version number, an example is as follows: FS#show version detail、 System description : FS Data Center Switch(N8560-48BC) By FS.COM Inc System start time : 2025-04-24 17:26:17 System uptime : 0:16:52:38 System hardware version : 2.34 System software version : N8560_FSOS 12.5(1)B0508S13 System patch number : NA System software number : M21523704232025 System serial number : G1S41C6001311 System boot version : 1.3.40(Master) 1.3.40(Slave) System rboot version : 1.2.13 System core version : 5.4.241-Cavium-Octeon+ System cpu partition : 3 Module information: Slot 1/0 : N8560-48BC System uptime : 0:16:52:38 Hardware version : 2.34 Boot version : 1.3.40(Master) 1.3.40(Slave) Rboot version : 1.2.13 Software version : N8560_FSOS 12.5(1)B0508S13 Software number : M21523704232025 Serial number : G1S41C6001311 Slot 2/0 : N8560-48BC System uptime : 0:14:42:13 Hardware version : 2.34 Boot version : 1.3.40(Master) 1.3.40(Slave) Rboot version : 1.2.13 Software version : N8560_FSOS 12.5(1)B0508S13 Software number : M21523704232025 Serial number : G1SC27B001055 FS#show version detail System description : FS Data Center Switch(N8560-64C) By FS.COM Inc System start time : 2025-04-25 09:19:54 System uptime : 0:00:09:57 System hardware version : 1.00 System software version : N8560_FSOS 12.5(1)B0508S13 System patch number : NA System software number : M21523704232025 System serial number : G1MS20N00006B System boot version : 1.3.36(Master) 1.3.36(Slave) System rboot version : 1.2.15 System core version : 5.4.241-Cavium-Octeon+ System cpu partition : 3 Module information: Slot 1/0 : N8560-64C System uptime : 0:00:09:57 Hardware version : 1.00 Boot version : 1.3.36(Master) 1.3.36(Slave) Rboot version : 1.2.15 Software version : N8560_FSOS 12.5(1)B0508S13 Software number : M21523704232025 Serial number : G1MS20N00006B Slot 2/0 : N8560-64C System uptime : 0:00:09:52 Hardware version : 1.00 Boot version : 1.3.36(Master) 1.3.36(Slave) Rboot version : 1.2.15 Software version : N8560_FSOS 12.5(1)B0508S13 Software number : M21523704232025 Serial number : G1ML21D000979 Release History Release Date Version Type 2023-08-11 N8560_FSOS 12.5(1)B0505 Initial Official Release 2023-08-11 N8560_FSOS 12.5(1)B0506 Official Release 2024-05-13 N8560_FSOS 12.5(1)B0508 Official Release 2025-01-23 N8560_FSOS 12.5(1)B0508S10 Official Release New Features Feature Module Requirement Source (Optional) Feature Description First Supported Version BFD NA Supports BFD over MLAG functionality N8560_FSOS 12.5(1)B0508 FRR NA Supports ISIS ECMP FRR functionality N8560_FSOS 12.5(1)B0508 Local QoS value marking functionality NA Marks local QoS values at the ingress using MQC, allowing modifications such as DSCP marking, rate limiting, filtering, and statistics at the egress based on ingress information. N8560_FSOS 12.5(1)B0506 ACL NA Adds support for matching ACL and printing packet features to CLI at ingress/egress. N8560_FSOS 12.5(1)B0506 Fault Isolation NA The FSUI component monitors the file system for read-only status and will reset the device for self-healing. N8560_FSOS 12.5(1)B0506 Storm suppression NA Supports storm control violation actions for multicast and broadcast PPS (packets per second) on violation ports. N8560_FSOS 12.5(1)B0506 PBR v6 NA Supports policy-based routing to specify the next hop across VRFs. N8560_FSOS 12.5(1)B0506 Sloved Problems 12.5(1)B0508S13 Issue ID Problem Description Fixed Version 1334969 Fixed an issue where, in a VSU environment, when the active device powered down and the standby device took over, there was a probability that STP states of cross-VSU AP member ports were set to BLOCK, resulting in traffic forwarding disruption. N8560_FSOS 12.5(1)B0508S13 1336721 Fixed an issue where the subnet broadcast address could be pinged. N8560_FSOS 12.5(1)B0508S13 1353316 Fixed an issue where the physical port LED status was inconsistent with the software-reported LINK state during device operation. N8560_FSOS 12.5(1)B0508S13 1154843 Fixed an issue where continuously shutting down and bringing up ECMP member ports after batch creation of full-capacity ECMP groups (the current available ECMP group resources can be checked via show route-res usage all) could repeatedly trigger ECMP group resource allocation and occupation, with a probability of causing high SDA memory usage. N8560_FSOS 12.5(1)B0508S10 1290111 Fixed an issue where, after shutting down an MLAG downlink port, unicast traffic destined for a VXLAN MAC entry whose egress was a shutdown VAP could not be forwarded. N8560_FSOS 12.5(1)B0508S10 1290112 Fixed an issue where ECMP route traffic interruption could occur when the ECMP route egress changed or when elastic hashing was enabled or disabled. N8560_FSOS 12.5(1)B0508S10 1290116 Fixed an issue where, during a business process upgrade involving function patches, there was a small probability that temporary threads exited after creation, causing function patch removal to fail with the error message "Remove package error!". N8560_FSOS 12.5(1)B0508S10 1197591 Fixed an issue where SSH authentication using the ed25519 public key algorithm could fail. N8560_FSOS 12.5(1)B0508S10 12.5(1)B0508S10 Issue ID Problem Description Fixed Version 1154843 Resolve the issue where bulk creation of full-capacity ECMP groups (the current available ECMP group resource size can be viewed using show route-res usage all) and continuous shutdown/no shutdown of ECMP member ports lead to continuous ECMP group resource allocation, which may result in high SDA memory usage. N8560_FSOS 12.5(1)B0508S10 1290111 Resolve the issue where shutting down the MLAG downstream port and hitting the VXLAN MAC exit causes unicast traffic of the shutdown VAP to fail forwarding. N8560_FSOS 12.5(1)B0508S10 1290112 Resolve the issue where changes in the ECMP routing exit or enabling/disabling elastic hashing causes ECMP route disruption. N8560_FSOS 12.5(1)B0508S10 1290116 Resolve the issue where upgrading business process function patches may occasionally fail to uninstall the patch, resulting in the error message "Remove package error!" due to the temporary creation of a thread that exits. N8560_FSOS 12.5(1)B0508S10 1197591 Resolve the issue where authentication fails when using the ED25519 public key algorithm for SSH. N8560_FSOS 12.5(1)B0508S10 12.5(1)B0508 Issue ID Problem Description Fixed Version 1154843 Resolve the issue where NETCONF fails to transcode when the device contains illegal characters. In this case, NETCONF skips transcoding and sends the data directly to the controller, which causes a disconnection and reconnection. N8560_FSOS 12.5(1)B0508 1109939 Supports enabling and disabling the gRPC server functionality through configuration commands. N8560_FSOS 12.5(1)B0508 1126197 Resolve the occasional BFD protocol flapping issue in VXLAN scenarios when there is a high volume of VXLAN broadcast traffic. N8560_FSOS 12.5(1)B0508 12.5(1)B0506 Issue ID Problem Description Fixed Version 1040164 Unable to view PS information - TIPC link down, and syslog does not print any logs. N8560_FSOS 12.5(1)B0506 1039712 The order of the clock timezone configuration in the show running-config is inconsistent after upgrading from 11X to 12X. N8560_FSOS 12.5(1)B0506 1018363 The DHCPv6 client obtains an IPv6 address with a 64-bit prefix from the server. If the gateway prefix is not 64 bits, the device generates a default route. This can cause communication issues across different subnets. N8560_FSOS 12.5(1)B0506 1017825 At the customer's site, the integration of Arista multi-level MLAG and VSTP does not support configuring the revision. Otherwise, it may cause the two devices to send BPDU packets with different cost values. N8560_FSOS 12.5(1)B0506 1036352 Under PBR traffic, the statistics from show ip pbr statistics / show ipv6 pbr statistics command return 0. N8560_FSOS 12.5(1)B0506 Known Issues Bug ID Issue Description Workaround First Detected Version 1050908 In the ZAM zero-configuration startup scenario, if the customer's script contains issues (unable to correctly configure "no ip address"), the lease on the DHCP server may remain after a reboot. The issue can be resolved by correctly revising the script. N8560_FSOS 12.5(1)B0508 1108966 On devices with the patch already installed, performing the ZAM process will not allow the installation of a new patch without first uninstalling the old patch. The ZAM process should be performed after uninstalling the patch. N8560_FSOS 12.5(1)B0508 1103759 When using Checkpoint to back up and rollback configurations, if several of the 6 configured IPFIX entries are deleted and then restored, the order of the restored configuration may differ from the original. There is no workaround, and only the order is different. N8560_FSOS 12.5(1)B0508 1037893 When integrating with a partner's IPSec algorithm, some algorithms may fail to establish encryption and decryption interoperability. The command/configuration manual has listed the IPSec encryption algorithms that are supported for integration with the partner. N8560_FSOS 12.5(1)B0508 1051233 In a high-capacity BGP environment, when the BGP message rate exceeds the default CPP threshold (3000 pps), CPP packet loss occurs, leading to BGP instability. In a high-capacity BGP scenario, increasing the BGP CPP threshold can help avoid this issue. It is recommended to set it to the maximum value. N8560_FSOS 12.5(1)B0508 1096015 In an MLAG+VSTP scenario, restarting the MSTP process may cause temporary packet loss. - N8560_FSOS 12.5(1)B0508 1042361 When using Checkpoint to back up configurations and perform a rollback to restore a Layer 2 port to a Layer 3 port, the configuration may fail to restore. The configuration that was not restored will prompt the user, requiring manual restoration. N8560_FSOS 12.5(1)B0508 1090243 During the one-click VSU upgrade process using the upgrade auto command, if the standby unit is rebooting and the syslog shows "VSU-5-DTM_AUTO_UPGRADE: Upgrading the system, wait a moment please," manually executing a new upgrade command will cause the one-click upgrade to fail. When the upgrade auto command is used to perform a one-click VSU upgrade, and the standby unit is still in the reboot process, with the VSU upgrade not yet complete and log messages indicating the upgrade is still in progress, do not execute a new upgrade command. N8560_FSOS 12.5(1)B0508 1138568 When the FP resources are at full capacity, enabling the VXLAN-QoS feature while exceeding the FP capacity limit will generate a syslog message. The configuration will not roll back. After exceeding the capacity limit, a syslog message will be printed, indicating the need to delete some FP rules. The configuration will not roll back automatically. N8560_FSOS 12.5(1)B0508 1108385 In a full capacity scenario, the idle FP reserved slice resources cannot be used by the ACL. When the TCAM resources are full, delete other unused configurations to release resources. N8560_FSOS 12.5(1)B0508 1048968 When using TFTP to transfer and specify the USB path, if the USB drive is removed, the configuration will not roll back. Continuing the transfer at this point will fail. When using TFTP with the USB path, if the USB drive is removed, you will need to re-specify the TFTP path. N8560_FSOS 12.5(1)B0508 1162266 When two devices are grouped in MLAG with a non-direct VAP, executing vap shut/no shut and pl shut/no shut will cause a brief burst of multiple packets on the devices. It is not recommended to configure a non-direct VAP. N8560_FSOS 12.5(1)B0508 1135051 In 11.x, the default load balancing mode is src-dst-mac, while in 12.x, it is the enhanced load balancing mode. If the enhanced load balancing is configured in 11.x and the system is upgraded to 12.x, the show running-config command will not display the enhanced load balancing configuration (as the default configuration will not be shown). - N8560_FSOS 12.5(1)B0508 1077447 The GTSM feature is not supported for Link-local neighbors, although the command is still present. In this version, the configuration for GTSM will be disabled. If there was a previous configuration for GTSM on Link-local neighbors, it will be lost after the upgrade, but the functionality will not be affected. - N8560_FSOS 12.5(1)B0508 1039201 [11.X to 12.X Upgrade/Downgrade] In the 11.X upgrade, the configuration order for MTU forwarding 9000 has changed. The implementation in 12.x is more reasonable. N8560_FSOS 12.5(1)B0506 1039202 [11.X to 12.X Upgrade/Downgrade] In the 11.X upgrade, the configuration order related to AAA has changed. In 11.x, the RADIUS server group is attached to the PARSE_ADD_CFG_TOP_CMD node in the CLI, and the global server configuration is attached to the PARSE_ADD_CFG_CMD node. Commands under the TOP node are applied first, so during configuration loading, the server group configuration is applied before the global server configuration. In practice, the configuration under the server group depends on the global server configuration. If the server group is configured before the global server, a log will prompt that the global server must be configured. This causes a warning log during boot-time configuration loading. Once the global server is configured, the functionality works as expected. In the initial version of 12.x, the server group is attached to the PARSE_ADD_CFG_CMD node, and the global server configuration is attached to the PARSE_ADD_CFG_TOP_CMD node. This resolves the boot-time warning log issue. There is no functional difference, and the implementation in 12.x is more reasonable. N8560_FSOS 12.5(1)B0506 1036119 [11.X to 12.X Upgrade/Downgrade] After upgrading from 11.X, the default Layer 3 MTU under an interface equals the link-layer MTU. When the link-layer MTU is modified, the default Layer 3 MTU changes accordingly. When mtu 4000, ip mtu 4000, and ipv6 mtu 4000 are configured simultaneously under an interface, the show running-config output differs between 11.X and 12.X. - N8560_FSOS 12.5(1)B0506 981320 The device does not support link-local addresses other than FE80:0:0:0. - N8560_FSOS 12.5(1)B0506 Unsupported Line Cards The NC8200-4TD does not support the NC8200-16 module. 964728 After three consecutive login failures, the account will be automatically locked, and the IP will be prevented from logging in again for 10 minutes. Do not exceed three consecutive failures, or wait 10 minutes after a failure. N8560_FSOS 12.5(1)B0505 920024 Changes in default behavior for management protocols: Telnet is disabled by default. SNMP is disabled by default; when enabled, SNMPv3 is supported by default (SNMPv1 and SNMPv2 are not supported by default and need to be manually enabled). Password strings are encrypted by default. Strong password check is enabled by default. - N8560_FSOS 12.5(1)B0505 Default Behavior Change Changes in default behavior after upgrading from 11.X to 12.X: QOS configuration: qos-queue compatible enable //12.X's default value depends on whether the capability is 0 to 7 or 1 to 8. AAA configuration: aaa slave-login allow //12.X's backup login functionality has changed. The console port on the backup device now logs into the primary device. This command is now obsolete. Disable visualization: no visual path detection enable //12.X does not support PFV. Key encryption: service password-encryption // Security policy: This configuration will be displayed whether it is explicitly set or not. Disable telnet login service: no enable service telnet-server // Telnet is disabled by default, as per network device security policies. - N8560_FSOS 12.5(1)B0505 916174 The CPP command output order in 11.X follows the configuration order, while in 12.X, it follows dictionary order, resulting in inconsistent sorting. External Interface Change Notice: The current CPP configuration display in 12.X is an optimization over 11.X, and no workaround is needed. N8560_FSOS 12.5(1)B0505 837022 By default, the dh_group_exchange_sha1 key exchange algorithm is not supported. The algorithm can be supported by configuring it with the ip ssh-server key-exchange dh_group_exchange_sha1 command. However, this algorithm does not comply with national standards and is not recommended for use. N8560_FSOS 12.5(1)B0505 916400 In 12.X, the libproxy-cli module commands from 11.X have been split into different modules. As a result, it is not possible to guarantee the same command order across modules as in 11.X. External Interface Change Notice, involving command scope: EFMP: config ecmp cluster enable LSM: 1）config mtu forwarding <64-9216> 2）interface posit-lamp enable TCPIP: config arp-passby optimization enable dcn_app_intf: interface transceiver xxx ptm_cli: 1）interface mac-loopback 2）interface mac-loopback remove-itag N8560_FSOS 12.5(1)B0505 909474 The value range for BGP commands has been changed to align with BFD. If a user upgrades from 11.X to 12.X, it may result in configuration loss. External Interface Change Notice. N8560_FSOS 12.5(1)B0505 847744 When the device is configured with TCAM mode, downgrading from 12.X to 11.X requires the device to restart once for the TCAM mode to take effect. An additional restart will make it effective. N8560_FSOS 12.5(1)B0505 845454 After the SDK upgrade, FP does not support matching the packet fragment ID. External Interface Change Notice: [Product Difference] Due to the limitation of SDK 6.5.18 after the upgrade, IFA and IPID are not supported, and FP does not support matching the packet fragment ID. N8560_FSOS 12.5(1)B0505 853495 CPP Protocol Difference – SDN Protocol Compatibility Issues Between 11.X and 12.X Upgrades/Downgrades 11xSdn Sdn_of_fetch Sdn_of_copy Sdn_of_trap12X merged into trap display N8560_FSOS 12.5(1)B0505 853344 CPP Protocol Difference – BFD Protocol Compatibility Issues Between 11.X and 12.X Upgrades/Downgrades In 11.X, show cpu protect displays bfd, micro-bfd, and micro-bfd-v6; in 12.X, it only displays bfd. N8560_FSOS 12.5(1)B0505 853497 CPP Protocol Differences: MPLS Compatibility Between 11.X and 12.X [Product Difference] 12.x introduces a separate IPv6 type due to chip-level distinction (e.g., JR). 11.x JR-based devices also differentiate between IPv4 and IPv6. N8560_FSOS 12.5(1)B0505 918437 [Product Difference] When upgrading directly from 11.x to 12.x, static MAC configurations for VXLAN become disordered. - N8560_FSOS 12.5(1)B0505 Compatibility Notes Hardware Support Notes The following are the hardware models and version numbers supported by this version. Hardware Type Hardware Model Hardware Version Number Description (Optional) Switch N8560-48BC 1.x Host Switch NC8200-4TD 1.x Host Switch N5860-48SC 1.x Host Switch N8560-64C 1.x Host Expansion Card NC8200-8C 1.x Expansion Card Expansion Card NC8200-24BC 1.x Expansion Card Power Supply GW-CRPS550N2C 1.x The AC power supply model for N8560-48BC and N5860-48SC. Power Supply U1A-D10550-DRB-Z 1.x The DC power supply model for N8560-48BC and N5860-48SC. Power Supply U1D-D10800-DRB 1.x The DC 800W power supply model for the N8560-48BC, N5860-48SC, and N8560-64C. Power Supply GW-CRPS550N2RC 1.x The AC power supply model for the N5860-48BC and N5860-48SC, Front-to-Back. Power Supply GW-CRPS800N2C 1.x NC8200-4TD、N8560-64C 800W Fan AFM-NC4TD-FB 1.x NC8200-4TD Front-to-Back airflow fan module. Fan AFM-N48SC-FB 1.x, Fan model for N5860-48SC and N8560-48BC, Front-to-Back Fan M1EFAN IV-R 1.x Fan model for N5860-48SC、N5860-48BC, Back-to-Front Fan AFM-N64C-FB 1.x Fan model for N8560-64C, Front-to-Back Upgrade Instructions Upgrade File File Name N8560_FSOS12.5(1)B0508S13_install.bin File Description System Installation Package (For 12.x Upgrade Only) File Size 148413428 bytes Applicable Products N8560-48BC、NC8200-4TD、N5860-48SC、N8560-64C MD5 Value 00ba3a1878302088909b2d18c73f989d Software Version N8560_FSOS 12.5(1)B0508S13 Upgrade Requirements (Upgrade Notes) 5.2.1. Pre-upgrade Checks Perform the following checks before upgrading to help prevent upgrade failures: If service interruptions are allowed during the upgrade, it is recommended to stop any unrelated services to improve the reliability of the upgrade. Check memory and CPU utilization. Avoid upgrading when CPU utilization is high (e.g., above 80%) or memory usage is high (e.g., above 70%). Upgrading under such conditions may lead to system instability, resulting in transmission or upgrade failures. Prepare a rollback package. If you need to prevent issues such as system malfunctions (e.g., trial versions) after the upgrade, and the selected upgrade method does not support rollback, you should prepare the upgrade package of the previous version in advance. Check the validity of the upgrade package. Compare the actual MD5 of the current upgrade package with the one listed in the release notes, or verify if the upgrade package type matches the device that is being upgraded. The installation package dedicated to upgrading from 11.x, (upgrade_from_11.x)N8560_FSOS12.5(1)B0508_install.bin, will cause a restart during the upgrade process. After the restart, the device will begin automatically upgrading to 12.x. During this process, ensure there is no power interruption or restart, and refrain from performing any other operations until the device automatically restarts and enters the 12.x CLI interface. During the upgrade or downgrade process, please pay close attention to the output prompts. If any failure occurs, save the log information and contact our technical support. During the upgrade or downgrade process, you may encounter prompts indicating that a restart is not allowed. If such prompts appear, please do not power off, reset the system, or unplug any modules. During the upgrade or downgrade process, please pay close attention to the output prompt information. If a failure occurs, save the log information and contact our technical support. During the upgrade or downgrade process, you may encounter prompts indicating that a restart is not allowed. If such a prompt appears, do not power off, reset the system, or remove/insert any modules. During the upgrade process, ensure that you do not power off, restart, or perform any other actions until the system automatically reboots to the 12.X CLI interface (estimated time: around 1h 30mins, depending on the actual situation). Failure to do so may result in upgrade failure or device malfunction. Before downgrading from 12.x to 11.x, if the device has a configured password, please remove the password or delete the configuration before downgrading. Otherwise, issues may occur after downgrading (as 11.x and 12.x handle password parsing differently), such as missing username configurations. After upgrading from 11.x to 12.x, the order of some configurations may change. During the downgrade process, ensure that the system is not powered off or rebooted, and avoid making any other operations. Once the version update is complete, enter the CLI interface. Do not perform any write operations while the system is in the intermediate version. After the upgrade or downgrade is complete, please run the "show version" command to check the current device version and confirm the success of the upgrade. 5.2.2. Upgrade Operation Instructions Upgrade via USB Drive Insert the USB drive into the device and use the upgrade command to complete the installation package upgrade. An example of the upgrade process is as follows: Hostname# upgrade usb0: /xxx/hulkos.bin Installation Package Local Upgrade Steps Under the condition that the network communication between the device and the PC is normal, use the copy oob_tftp command to transfer the installation package to the device, and then use the upgrade command to complete the installation package upgrade. Assuming the PC's address is 1.1.1.3, the upgrade example is as follows: FS# config FS(config)# interface mgmt 0 FS(config-if-Mgmt 0)# ip address 1.1.1.1/24 FS(i config-if-Mgmt 0)# end FS# copy oob_tftp://1.1.1.3/hulkos.bin flash:hulkos.bin FS# upgrade flash:hulkos.bin Network Upgrade Steps for Installation Package Ensure network connectivity between the device and the PC. Use the upgrade download oob_tftp command to complete the upgrade. Assuming the PC's IP address is 1.1.1.3, an upgrade example is as follows: FS# config FS(config)# interface mgmt 0 FS(config-if-Mgmt 0)# ip address 1.1.1.1/24 FS(config-if-Mgmt 0)# end FS# upgrade download oob_tftp://1.1.1.3/ hulkos.bin Upgrade Steps Under rboot Mode First, press Ctrl + C during bootup to enter rboot. Then, in the rboot menu, select Tftp utilities -> Upgrade main program to proceed with the upgrade. Connect the PC to the device’s MGMT port, set the device’s management IP address to 1.1.1.2. Assuming the PC’s address is 1.1.1.3, an upgrade example is as follows: ====== Rboot Menu (Ctrl+Z to upper level) ====== TOP menu items. ************************************************ 0. Tftp utilities. 1. X/Y/ZModem utilities. 2. Run main. 3. SetMac utilities. 4. Scattered utilities. ************************************************ Press a key to run the command: 0 ====== Rboot Menu (Ctrl+Z to upper level) ====== Tftp utilities. ************************************************ 0. Upgrade uboot/bios program. 1. Upgrade rboot program. 2. Upgrade main program. 3. Upgrade the entire device by distribute package. 4. Burn the total FlashROM by this downloaded file. ************************************************ Press a key to run the command: 2 Plz enter the Local IP [192.168.64.31]:1.1.1.2 Plz enter the Remote IP [192.168.64.1]:1.1.1.3 Plz enter the Filename [fsos.bin]:fsos.bin If the upgrade file is already stored locally, select X/Y/ZModem utilities -> Local utilities -> Upgrade main program in the rboot menu and enter the file path to proceed with the upgrade. ====== Rboot Menu (Ctrl+Z to upper level) ====== TOP menu items. ************************************************ 0. Tftp utilities. 1. X/Y/ZModem utilities. 2. Run main. 3. SetMac utilities. 4. Scattered utilities. ************************************************ Press a key to run the command: 1 ====== Rboot Menu (Ctrl+Z to upper level) ====== X/Y/ZModem utilities. ************************************************ 0. XModem utilities. 1. YModem utilities. 2. ZModem utilities. 3. Local utilities. ************************************************ Press a key to run the command: 3 ====== Rboot Menu (Ctrl+Z to upper level) ====== Local utilities. ************************************************ 0. Upgrade uboot program. 1. Upgrade rboot program. 2. Upgrade main program. 3. Upgrade the entire device by distribute package. 4. Burn the total FlashROM by this downloaded file. ************************************************ Press a key to run the command: 2 Plz enter the Filepath [fsos.bin]:/xxx/fsos.bin 5.2.3. Post-Upgrade Checklist After the upgrade, check the following system status items to verify whether the upgrade was successful: Verify upgrade success. In rboot, the upgrade status can be determined by checking the logs during the upgrade process. Plz enter the Filepath [fsos.bin]: /xxx/fsos.bin Checking file, keep power on and wait please ... Determined to upgrade? [Y/N]: y Rootfs found: rootfs.sqsh Uboot found: Rboot found: Extract package.... Mount rootfs file system ... Install rootfs files to Flash ... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Install main success. Upgrade main(or uboot/rboot) finish. Success (or upgrade nothing due to version is same). For upgrades performed using the upgrade command, you can determine if the upgrade was successful by checking the logs during the upgrade process and using the show upgrade status command. FS#upgrade flash: fsos.bin < The terminal is locked by upgrade module > Upgrade start !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!10% < you can press Ctrl+C to unlock terminal > !!!!!!!!!!!!100% Upgrade success [Slot 0] Device type : N8560-48BC Status : success < The terminal is unlocked by upgrade module > FS#show upgrade status upgrade global status: INIT [Slot 0] Device type : N8560-48BC Status : success Verify New Version You can check the system's displayed version and upgrade status by using the show upgrade history and show upgrade status commands. The upgrade history can confirm the success of the upgrade by checking the upgrade time and package name. FS# show upgrade status upgrade global status: INIT [Slot 0] Device type : N8560-48BC Status : ready FS#show upgrade history Upgrade History Information: Time : 2000-01-04 18:38:37 Method : LOCAL Package Name : fsos-factory.bin Package Type : MAIN Time : 2019-09-25 10:35:04 Method : LOCAL Package Name : fsos.bin Package Type : MAIN Package Type: MAIN FS# show version detail …Refer to the version information in Chapter 1.… Supporting Documents Document Name Document Description Release Notes for N8560 Series Switch FSOS 12.5(1)B0508S13 Version This document (i.e., this article) provides a detailed introduction to the N8560_FSOS 12.5(1)B0508S13 version, including: version information, hardware support, feature support, known issues, version change logs, and upgrade instructions.

FSOS 12.5(1)B0508S13 Software Image for N5860-48SC and N8560-48BC and N8560-64C and NC8200-4TD Switches

2026-01-19 - For details, please click the attachment icon below to view or download for a good reading experience or resources.

2025-12-11 - For details, please click the attachment icon below to view or download for a good reading experience or resources.

2025-10-30 - For details, please click the attachment icon below to view or download for a good reading experience or resources.

N5860 and N8560 and NC8200 Series Switches CLI Reference Guide

2025-10-28 - For details, please click the attachment icon below to view or download for a good reading experience or resources.

2025-09-06 - For details, please click the attachment icon below to view or download for a good reading experience or resources.

2025-08-08 - Safety Precautions for FS Switches Mainly Applicable to FS Commercial Switches Preface Audience This document is for network engineers responsible for installing and maintaining FS switches. Experience with network equipment installation and maintenance is required. 1 Precautions for Safe Use 1.1 General Safety Keep the chassis clean and dust-free. Do not place the equipment in walking areas. Do not wear loose clothes, ornaments, or any other things that may be hooked by the chassis during installation and maintenance. Cut off all power supplies and unplug all power cords before installing or dismantling the chassis. Prevent the switch from being frequently handled. Cut off all power supplies and unplug all power cords before moving or handling the switch. Keep balance and prevent personal injuries when handling the switch. Do not hold the handle of the power module or the fan module when moving the switch. Otherwise, it may cause equipment damage or even personal injury. Remove the fan modules and the power modules before handling the equipment. Install the equipment where it is not likely to be moved. 1.2 Installation Environment Requirements FS switches (excluding industrial ethernet switches) must be installed indoors. In order to ensure normal operation and prolonged service life, the installation site must meet the following requirements. 1.2.1 Cabinet Installation Before installing the FS switches in a cabinet, make sure that the cabinet meets the following requirements: Install the switch in an open cabinet. If the switch is installed in a closed area, ensure that it has a good ventilation system. Confirm that the cabinet is sturdy enough to support the weight of the FS switches and accessories. Maintain a clearance around the chassis in the cabinet for heat dissipation. The cabinet is properly grounded. 1.2.2 Ventilation Keep a minimum clearance of 200mm (7.87 in.) around the FS switches for air circulation. After various cables are connected, bundle the cables or route them over the cable management bracket to avoid blocking air inlets. 1.2.3 Temperature and Humidity To ensure the normal operation and prolonged service life of the FS switches, maintain an appropriate temperature and humidity in the equipment room. The equipment room with too high or too low temperature and humidity for a long time may damage the switches. In an environment with high relative humidity, the insulating material may have poor insulation or even leak electricity. Sometimes also prone to changes in the mechanical properties of materials, metal parts rust and other phenomena. In an environment with low relative humidity, static electricity is prone to occur and damage the internal circuits of the equipment. Too high temperatures can accelerate the aging of insulation materials, greatly reducing the reliability of the switch and severely affecting its service life. The ambient temperature and humidity of the equipment are measured at the point that is 1.5m (59.06 in.) above the floor and 0.4m (15.75 in.) before the rack when there is no protective plate in the front or at the back of the rack. 1.3 Electrical Safety 1.3.1 Grounding A proper grounding system is the basis for stable and reliable operation of FS switches and is indispensable for preventing lightning strikes and resisting interference. Carefully check the grounding conditions at the installation site according to the grounding specifications, and complete grounding properly based on the actual situation. Safety Grounding The switch using AC or high-voltage DC power supply must be grounded by using the green and yellow grounding cable. Otherwise, when the high voltage circuit inside the switch has a grounding fault, an electric shock may occur. The building should provide a protective ground connection to ensure that switches are connected to the protective ground. Lightning Grounding The lightning protection system of the facility is a separate system consisting of a lightning rod, a lower conductor and a connector connected to the grounding system. This grounding system is usually shared with the ground used as the power supply reference ground and the yellow-green safety ground. Lightning discharge grounding is only for facilities, equipment does not have this requirement. For lightning protection, see Requirements for Lightning Proof Grounding. EMC Grounding Grounding requirements for EMC design purposes include: shield grounding, filter grounding, noise and interference suppression, and level referencing. These form the combined grounding requirements. Grounding resistance is required to be less than 1 Ω. There is one ground terminal on the back of the FS switch chassis, which is indicated by a conspicuous warning label. Figure 1 EMC Grounding image.png 1.4 Battery Safety 1.4.1 Basic Requirements Observe local regulations and specifications during electrical operations. Only personnel with relevant qualifications can perform such operations. Check whether there are potential risks in the work area. For example, check whether the power supply is grounded, whether the grounding is reliable, and whether the ground is wet. Learn about the position of the indoor emergency power switch before installation. Cut off all power when an accident occurs. Do not maintain the equipment that is powered-on alone. Check the equipment carefully before shutting down the power supply. Do not place the equipment in a wet position, and keep the chassis away from liquid. Irregular and incorrect electrical operation may cause accidents such as fire or electric shock, and lead to serious and fatal injuries to the human body and equipment. Direct or indirect contact with high voltage and mains electricity through wet objects may pose a fatal risk. If the FS switch system has more than one input power source, be sure to disconnect all power cords before shutting down the system. If a power supply system is equipped with a leakage protector (also referred to as "leakage current switch" or "leakage current breaker"), the rated leakage action current of each leakage protector is greater than twice of the theoretical maximum leakage current of all the power supplies in the system (For example, if a system is equipped with 16 identical power supplies, the leakage current of each power supply is equal to or less than 1.75mA, and the leakage current of the system totals 28mA. A leakage protector with 30 mA rated action current supports less than 9 power supplies (that is, Action current of the leakage protector/2/Maximum leakage current of each power supply = 30/2/1.75 ≈8.57). In other words, the leakage protector with 30mA rated action current supports no more than 8 power supplies. In this case, the 16 power supplies in the system require at least two leakage protectors with 30mA rated action current and each leakage protector supports 8 power supplies). If power supplies in a system differ in models, the rated leakage action current of each leakage protector divided by two is greater than the sum of maximum leakage currents of all the power supplies. The rated leakage non-action current of a leakage protector shall be 50% of the leakage action current (Take a leakage protector with 30mA rated leakage action current as an example. The rated leakage non-action current shall be 15mA. When the leakage current is below 15mA, the protector shall not act. Otherwise, misoperation may easily occur due to high sensitivity and thus the leakage protector trips, devices are powered off, and services are interrupted). To guarantee personal safety, the rated leakage action current of each leakage protector in the system must be equal to or less than 30mA (human body safety current is 30mA). When twice of the total leakage current of the system is greater than 30mA, the system must be equipped with two or more leakage protectors. 1.4.2 Requirements for Rechargeable Batteries If a rechargeable battery is used, pay attention to the following precautions: If discoloration, deformation, overheating, or any other abnormality occurs, replace the battery before continuing with usage, charging or storage. Tighten battery cables or copper bars using the torque specified in the battery documentation. Insecure connection of battery bolts may cause excessive voltage drop or even overcurrent leading to battery overheating. If the battery temperature exceeds 60°C (140 °F), check for and promptly handle any leakage. If the electrolyte overflows, take proper measures promptly. When removing or moving the battery with spilled electrolyte, be careful with the electrolyte that can cause potential injury. If any electrolyte spills, use NaHCO(3) or Na(2)CO(3) to neutralize and absorb it. After batteries are installed, ensure that the fuse or circuit breaker is disconnected before powering the system. This avoids battery damage caused by power discharge in case of long-term power-off. Improper usage of lead-acid batteries will cause the release of flammable gas. Ensure that batteries are kept in a well-ventilated area and take preventive measures against fire. The battery should not be exposed to high temperature environments or around heat generating equipment such as sunlight, heaters, microwave ovens, ovens or water heaters. Battery overheating may cause an explosion. 1.4.3 Requirements for Non-Rechargeable Batteries If the equipment uses a dry battery or non-rechargeable lithium battery, consider the following: If discoloration, deformation, overheating, or any other abnormality occurs, replace the battery before continuing with usage or storage. Do not attempt to replace non-removable, built-in batteries. Doing so may damage the batteries or the equipment. Batteries must be replaced by an authorized service center. Do not throw the battery into the fire. Otherwise, the battery will catch fire and explode. 1.5 Radiation Safety 1.5.1 Electromagnetic Field Exposure Various interference sources, whether from outside the equipment or application system, or from within, are capacitive coupling, inductive coupling, electromagnetic wave radiation and other conductive ways to produce effects on the equipment. Electromagnetic interference is divided into two categories: radiation interference and conducted interference, which is determined by the type of propagation path. When the energy emitted by a device, usually RF energy, reaches a sensitive device through space, it is called radiated interference. The interference source can be both a part of the interfered system and a completely electrically isolated unit. Conducted interference results from the electromagnetic wire or signal cable connection between the source and the sensitive component, along the cable the interference conducts from one unit to another. Conducted interference often affects the power supply of the equipment, but can be controlled by a filter. Radiated interference may affect any signal path in the equipment, and is difficult to shield. Take effective measures for the power system to prevent interference from the electric grid. Keep the running position of the switch as far as possible from the grounding device of the power equipment or the anti-lightning grounding device. Keep the device away from high-power radio transmitters, radar transmitting station, and high-frequency large-current device. Take measures to isolate static electricity. 1.5.2 Laser Safety Among the modules supported by the FS switches, many optical transceivers are Class I laser products. Precautions: When an optical transceiver works, ensure that the port has been connected with a fiber or covered by a dust cap to keep out dust and prevent it from burning your eyes. Do not look into any optical port. Do not approach or stare at any fiber port under any circumstances, as this may cause permanent damage to your eyes. 1.6 Hardware Maintenance 1.6.1 Expansion Modules Maintenance In the event of a failure and the need to replace an expansion module, the expansion module must be installed and disassembled in accordance with the instructions for operation. 1.6.2 Cooling System Maintenance If a fan module fails, an alarm will be generated. Replace the faulty fan module. Tighten the captive screws. 1.6.3 Power Supply Maintenance When a power module is faulty, unplug the power cord, replace the power module, and plug the power cord again. 1.6.4 Replacing Lithium Battery The built-in lithium batteries can support the real time clock of the FS switches without external power supply. To replace lithium batteries, please contact FS technical support personnel. The technical support personnel will select lithium batteries of the same specifications for replacement. 1.6.5 Replacing Fuses To replace fuses, please contact FS technical support personnel. The technical support personnel will select fuses of the same specifications for replacement. 2 Environmental Requirements for Device Operation 2.1 Environmental Requirements for an Equipment Room Ensure that the installation environment complies with equipment specifications, including voltage, temperature, humidity, altitude, degree of pollution, overvoltage category, and waterproofing and dustproofing classification. Avoid flammable, explosive gas or smog environments. Keep the installation site free of acidic, alkaline or other corrosive gases. Keep the equipment away from sources of heat or fire, such as the electric heater, microwave oven, oven, water heater, fireplace, candle or other heat generators. Heat may cause the equipment to catch fire or its housing to melt. Do not obscure or cover running equipment with flammable materials such as paper or fabric. This hampers heat dissipation and can cause the equipment to catch fire or its housing to melt. This equipment (or system) must be installed or used in restricted areas. Do not block air vents when the equipment is running. Maintain air vents away from the wall or other objects as required in the operation guide. 2.2.1 Requirements for Selecting a Site for an Equipment Room Communication equipment should be in a good operating environment. When designing a project, consider the communication network planning and technical requirements of the equipment. Also consider hydrographic, geological, seismic, power supply, and transportation factors. Construction, structure, heating and ventilation, power supply, lighting and fire-proof construction of the equipment room should be designed by specialized construction designers to suit the environmental requirements of devices. The equipment room should also follow local regulations concerning the industrial construction, environmental protection, fire safety, and civil air defense. Construction must conform to government standards, regulations, and other requirements. The equipment room should be located in a place free from high temperature, dust, toxic gases, explosive materials, or unstable voltage. Keep the equipment room away from significant vibrations or loud noises, as well as power transformer stations. The specific requirements for selecting a site for an equipment room are as follows: The equipment room should be at least 5 km away from heavy pollution sources, such as the smelter works, coal mine, and thermal power plant. The equipment room should be at least 3.7 km away from medium pollution sources, such as the chemical factory, rubber factory, and electroplating factory. The equipment room should be at least 2 km away from light pollution sources, such as the food factory and leather plant. If these pollution sources are unavoidable, the equipment room should be located on the windward side of the pollution sources perennially with advanced protection. Do not build the equipment room in the proximity of livestock farms. Otherwise, the equipment room should be located on the windward side of the pollution source perennially. The previous livestock house or fertilizer warehouse couldn't be used as the equipment room. The equipment room should be away from the residential area. Otherwise, the equipment room should meet the construction standard in terms of noise. Keep the door and the window closed to make the equipment room sealed. The steel door is recommended for soundproofing. Make sure there are no cracks or holes on the wall and floor. If there are cable entries on the wall or window, take proper sealing measures. Ensure that the wall is flat, wear-resistant, and dust-free, which should be up to the standard for flame retarding, soundproofing, heat absorption, dust reduction, and electromagnetic shielding. Make sure that the air vents of the equipment room are away from the sewage pipe, septic tank, and sewage treatment tank. Keep the equipment room under positive pressure to prevent corrosive gas from entering the equipment room to corrode components and PCBs. Keep the equipment room away from industrial boilers and heating boilers. The equipment room had better be on the second floor or above. Otherwise, the equipment room floor should be 600mm higher than the highest flood level ever recorded. The equipment room should be at least 3.7 km away from the sea or salt lake. Otherwise, the equipment room must be sealed, with air conditioner installed for temperature control. Saline soil can not be used for construction. Otherwise, you should select devices with advanced protection against severe environments. The equipment room should be firm enough to withstand severe weather conditions such as wind storms and heavy rain as well as away from dust. If dust is unavoidable, keep the door and window away from the pollution source. Keep the air conditioner from blowing wind straight toward the equipment or blowing water drops from the window or air vent toward the equipment. Sulfur-containing materials are forbidden. 2.1.2 Equipment Room Layout An equipment room usually contains mobile switching equipment, telecommunications equipment, power supply equipment, and other auxiliary equipment. To ensure easy maintenance and management, place the equipment in different rooms. Figure 2 shows the layout of the equipment room. Figure 2 Layout of the Equipment Room image.png The general layout principles of the equipment room are as follows: It should meet requirements for laying out and maintaining communication cables and power cables. It should reduce the cabling distance, which facilitates cable maintenance, reduces potential communication faults, and maximizes efficiency. 2.1.3 Construction Requirements for the Equipment Room Table 1 describes the construction requirements for the equipment room. Figure 3 Internal Partition Wall Inside the Equipment Room image.png 2.1.4 Equipment Room Environment Dust on devices may cause electrostatic discharge and result in poor contact for connectors or metal connection points. This problem can shorten the life span of devices and cause faults. The equipment room must be free from explosive, conductive, magnetically-permeable, and corrosive dust. Table 2 lists the requirement for dust concentration in the equipment room. Take the following measures to meet the requirements: Use dustproof materials for ground, wall, and ceiling construction. Use screens on the door and windows facing outside. The outer windows should be dust-proof. Clean the equipment room and clean devices' air filters monthly. Wear shoe covers and ESD clothing before entering the equipment room. 2.2 Requirements for Corrosive Gases The room should be free from dusts and corrosive gases, such as SO(2), H(2)S, and NH(3). Table 3 lists the requirements for the corrosive gas concentration. Take the following measures to meet the requirements: Avoid constructing a room near a place where the corrosive gas concentration is high, such as a chemical plant. Ensure the air intake vent of the room is in the prevailing upwind direction from any pollution source. Place batteries in different rooms. A professional service should monitor the corrosive gas conditions regularly. 2.3 Requirements for ESD Prevention The absolute value of electrostatic voltage must be less than 1000V. Take the following measures to meet this requirement: Train operators about ESD prevention. Keep the correct humidity level in the equipment room to reduce the impact of static electricity. Lay out an ESD floor in equipment rooms. Wear ESD shoes and clothing before entering the equipment room. Use ESD tools, such as wrist straps, tweezers, and pullers. Ground all conductive materials in the room, including computer terminals. Use ESD worktables. Keep non-ESD materials (such as common bags, foam, and rubber) at least 30cm (11.81in.) away from boards and ESD-sensitive components. 2.4 Electromagnetism Requirements for the Equipment Room All interference sources, inside or outside the equipment room, can cause equipment problems with capacitive coupling, inductive coupling, electromagnetic wave radiation, and common impedance (including grounding system) coupling. Prevent interference using these approaches: Take effective measures against electrical interference from the power supply system. Do not use the working ground of the equipment as the same ground for surge protection. Separate them as far as possible. Keep the equipment far away from high-power radio transmitters, radar units, and high-frequency and high-current equipment. Use electromagnetic shielding if necessary. 2.5 Requirements for Lightning Proof Grounding Table 4 lists the requirements for lightning proof grounding.

2025-07-07 - Product Overview The NC8200-4TD switch supports a maximum of 128x 10G/25G, 64x 40G, or 32x 100G high-density full line rate ports through flexible line cards combinations of NC8200-8C, NC8200-16Q and NC8200-24BC. The NC8200-4TD can operate as leaf and spine switch for medium-sized data centers and cloud-computing data centers deployments. The switch employs an advanced cache scheduling mechanism to maximize the device's cache capability. With PFC and ECN to implement the low-latency, zero packet loss, high throughput and service forwarding rate, ensuring non-blocking transmission in the increasingly demanding data center environment. The NC8200-8C Ethernet line card supports 8x 100G QSFP28, offering line-rate forwarding from all ports. The QSFP28 ports are backward compatible with QSFP+ modules. The NC8200-16Q Ethernet line card supports 16x 40G QSFP+ ports, offering line-rate forwarding from all ports. The NC8200-24BC Ethernet line card supports 24x 25G SFP28 ports and 2x 100G QSFP28, offering line-rate forwarding from all ports. The SFP28 ports are backward compatible with SFP+ modules. The QSFP28 ports are backward compatible with QSFP+ modules. Product highlights Flexible 10/25/40/100GbE Interface Speeds, Support Stacking Support MLAG, RoCEv2, BGP4/BGP4+, EVPN-VXLAN, REUP, GR, BFD 1+1 Hot-Swappable Power Supplies, 2+1 Smart Fans Enable RDMA over Converged Ethernet Lossless (with PFC and ECN) Support SPAN/RSPAN/ERSPAN and In-band Network Telemetry (INT) for Visibility Support ACL, RADIUS, TACACS+, DHCP Snooping, etc. for Security Support Ansible, OpenFlow, NETCONF, etc. Configuration and Automation Tools Support SNMP v1/v2c/v3, CLI, Telnet, SSH Achieve Reliable Links With 100G NVIDIA Mellanox NIC Only for NC8200-8C Line Card Achieve Reliable Links with Intel XL710-BM2-Based NIC Only for NC8200-16Q Line Card Achieve Reliable Links With 25G NVIDIA Mellanox NIC Only for NC8200-24BC Line Card Platform details Platform benefits Software requirements Product specifications Quality certification At FS, our Quality Commitment lies in all aspects of processes, resources, and methods that enable us to build superior networks for our customers. Through a quality policy focusing on continuous improvement of products and services, we're able to achieve the highest levels of satisfaction for our customers. To that end, every FS employee is accountable for contributing to the value of the products and services we deliver. Figures 5 shows some of the authoritative certifications obtained by FS NC8200-4TD Switch. Optics supported For details about the optical modules available, visit： NC8200-4TD: Transceivers DACs and AOCs Supported on NC8200-4TD Switch Warranty, service and support FS NC8200 Series Switches enjoy 5 years limited warranty against defects in materials or workmanship. For more information for FS Returns & Refunds policy, visit https://www.fs.com/policies/warranty.html or https://www.fs.com/policies/day_return_policy.html FS provides a personal account manager, free professional technical support, and 24/7 live customer service to each customer.support.html Professional Lab: Test each product with the latest and advanced networking equipment. Free Technical Support: Provide free & tailored solutions and services for your businesses. 80% Same-day Shipping: Immediate shipping for in-stock items. Fast Response: Direct and immediate assistance from an expert. For more information, visit https://www.fs.com/service/fs_support.html Ordering information Additional information For more information about the NC8200 Series Switches, contact your account manager or visit https://www.fs.com/search_result?keyword=NC8200 Document history