These release notes support Cumulus Linux 2.2.0 and describe currently available features and known issues.
Cumulus Linux is licensed on a per-instance basis. Each network system is fully operational, enabling any capability to be utilized on the switch with the exception of forwarding on switch panel ports. Only eth0 and console ports are activated on an un-licensed instance of Cumulus Linux. Enabling front panel ports requires a license.
You should have received a license key from Cumulus Networks or an authorized reseller. To install the license, read the Cumulus Linux quick start guide.
Installing Version 2.2.0
To install the software, choose one of the following methods. They are ordered from the most recommended method to least recommended.
Download Cumulus Linux 2.2.0 - Final Latest Version from the Downloads page of the Cumulus Networks website, then use ONIE to perform a complete install, following the instructions in the quick start guide.
Warning: This method is destructive; any configuration files on the switch will not be saved, so please copy them to a different server before upgrading via ONIE.
There is no SNMP support for Quagga in this release (see RN 88 below). Due to this circumstance, you must remove all references to
smux in each of the following configuration files. You must also remove these references before upgrading Cumulus Linux using
apt-get. If the
smux entries are present in the configuration files, the daemons in the 2.2.0 packaged version of Quagga will not start.
grep smux *
- Delete all lines in the config files containing the smux keyword.
The references to
smux that must be removed are:
bgpd.conf, remove this line:
smux peer 22.214.171.124.4.1.33126.96.36.199 quagga_bgpd
ospf6d.conf, remove this line:
smux peer 188.8.131.52.4.1.33184.108.40.206 quagga_ospf6d
ospfd.conf, remove this line:
smux peer 220.127.116.11.4.1.3318.104.22.168 quagga_ospfd
zebra.conf, remove this line:
smux peer 22.214.171.124.4.1.33126.96.36.199 quagga_zebra
What's New in Cumulus Linux 2.2.0
Cumulus Linux 2.2 supports these new hardware platforms and features:
- Host-MLAG, for creating logical switches from LACP bonds
set ip next-hop unchanged
For a presentation highlighting the major changes, see Cumulus Linux 2:2 What's New and Different.
The following experimental features are included in Cumulus Linux 2.2.o:
- Management namespaces
- Redistribute neighbor
- Back up configurations with
- Simple image snapshots across upgrades with
You can read the technical documentation here.
Issues Fixed in Cumulus Linux 2.2.0
The following is a list of issues fixed in Cumulus Linux 2.2.0 from earlier versions of Cumulus Linux.
|Release Note ID||Summary|
|RN-111||Default arp_filter is set to 0, which causes the wrong interfaces to be associated|
|RN-113||High memory utilization on switches when large number of MAC addresses have ageing time set to 20000|
|RN-122||Spanning tree interoperability issue with common spanning tree|
|RN-127||In an OSPFv2 unnumbered interface, the "ip ospf area " in the interface command section is rejected when a "network" command is present|
|RN-149||VXLAN: Creating an OVS bridge with the VXLAN option will cause switchd to crash|
|RN-151||JunOS OSPF compatibility issues|
|RN-166||VXLAN error after quickly deleting then creating a bridge with the same ports: "ERR Cannot find vxlan port <> vpn <>"|
|RN-178||QSFP transceivers (Cisco QSFP-40G-SR4 and AOI AQOMNN85ADLN0534) not working on Quanta LY6 switch|
Known Issues in Cumulus Linux 2.2.0
Issues are categorized for easy review. Some issues are fixed but will be available in a later release.
|Release Note ID||Summary||Description|
|RN-4||ifup/ifdown must be used for interfaces with IPv6 addresses defined in /etc/network/interfaces, otherwise the IPv6 interface will go down||Two scenarios are shown below; one with ifup/ifdown, the other with ifconfig down.
swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 inet addr:188.8.131.52 Bcast:184.108.40.206 Mask:255.255.255.0 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link inet6 addr: fec0:1000:1000:1000::2/10 Scope:Site UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4231 errors:0 dropped:0 overruns:0 frame:0 TX packets:4342 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:412115 (402.4 KiB) TX bytes:425688 (415.7 KiB)
With ifconfig down:
sudo ifconfig swp1 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 inet addr:220.127.116.11 Bcast:18.104.22.168 Mask:255.255.255.0 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link inet6 addr: fec0:1000:1000:1000::2/10 Scope:Site UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:98 errors:0 dropped:0 overruns:0 frame:0 TX packets:111 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:13310 (12.9 KiB) TX bytes:12786 (12.4 KiB)
|RN-10||cl-phy-update doesn't support aggregated ports||Ports can be aggregated into a larger interface in Cumulus Linux. Unfortunately support for aggregated ports is not yet supported when running cl-phy-update.
If there are any ganged ports during a SW upgrade it is recommended to ungang these ports
|RN-48||Agema 48x10GE switch eth0 driver reports eth0 as running even when PHY link is down||The Agema 48x10GE eth0 driver reports eth0 as running even when the PHY link is down.
This can be really misleading in trying to diagnose a link-down situation on eth0. ethtool eth0 shows the correct PHY link status, but ifconfig shows eth0 as running, regardless of the PHY link status.
A fix will be released in a future version of Cumulus Linux.
|RN-52||Parameters like the router ID and DR priority cannot be changed while OSPFv2/v3 is running||Router ID and DR priority can only be changed by shutting down OSPFv2/v3, changing the ID, and restarting the OSPF process.
A change to the DR priority may not properly be reflected in the LSAs that are still aging out.
|RN-56||ipv4/ipv6 forwarding disabled mode not recognized||
If either of the following is configured:
net.ipv4.ip_forward == 0
net.ipv6.conf.all.forwarding == 0
The hardware still forwards packets if there is a neighbor table entry pointing to the destination.
|RN-58||IPv6 route is installed and active in the routing table when the associated interface is down||If an IPv6 address is assigned to a "down" interface, the associated route is still installed into the route table.
Also, the type of IPv6 address doesn't matter. Link local, site local, and global all exhibit the same problem.
If the interface is bounced up and down, then the routes are no longer in the route table.
|RN-61||BGP4 notifications missing for several conditions||In certain conditions, Quagga bgpd silently closes the peering without sending a notification. For example, if BGP receives a message with an invalid message type or invalid message length.
Ideally on any one of these cases, bgpd should send out a notification message to the peer.
General functionality of BGP4 is not affected.
|RN-62||Attributes of a BGP aggregate route may not be RFC-compliant||When BGP is configured with an aggregate route and there are more specific routes of that aggregate, the BGP speaker needs to analyze the attributes of those specific routes while forming the attributes of the aggregate route. The corresponding rules are defined in RFC 4271, Sect. 22.214.171.124.
In certain cases, the user may observe non-compliant attribute formation for the aggregate route: for example, incorrect MED and ORIGIN attributes.
|RN-63||BGP4 recursive route not supported||
Quagga's bgpd does not support recursive routing when it's resolving BGP routes. It does work with respect to IGP routes.
For example, a BGP route of the form:
R1 -> N1
where the best match for N1 is also a BGP route:
(and N2 gets resolved through IGP or directly connected) -
will not get resolved.
Quagga's Zebra specifically does not allow for such types of routes. The Zebra route path resolution algorithm needs to be fixed.
Although, Cumulus Linux's version of Quagga -- 0.99.21 -- has errors in recursive routing, the community is actively trying to fix it and patches are being tested in 0.99.22.
|RN-64||Configuring route-reflector-client requires specific order||In configuring a route to be a route reflector client, the Quagga configuration must be specified in a specific order; otherwise, the router will not be a route reflector client.
The "neighbor <IPv4/IPV6> route-reflector-client" command must be done after the "neighbor <IPV4/IPV6> Activate" command; otherwise, the route-reflector-client command is ignored.
router bgp 65000 bgp router-id 0.0.0.4 bgp log-neighbor-changes no bgp default ipv4-unicast bgp cluster-id 0.0.0.4 bgp bestpath as-path multipath-relax redistribute connected neighbor 126.96.36.199 remote-as 65000 neighbor 188.8.131.52 route-reflector-client neighbor 184.108.40.206 activate neighbor 220.127.116.11 next-hop-self neighbor 18.104.22.168 remote-as 65000 neighbor 22.214.171.124 activate neighbor 126.96.36.199 next-hop-self neighbor 2001:ded:beef::1 remote-as 65000 neighbor 2001:ded:beef:2::1 remote-as 65000 maximum-paths 4 maximum-paths ibgp 4 ! address-family ipv6 redistribute connected neighbor 2001:ded:beef::1 activate neighbor 2001:ded:beef::1 next-hop-self neighbor 2001:ded:beef:2::1 route-reflector-client neighbor 2001:ded:beef:2::1 activate neighbor 2001:ded:beef:2::1 next-hop-self maximum-paths 4 maximum-paths ibgp 4 exit-address-family
cumulus@switch:$ show ip bgp neighbor 188.8.131.52 BGP neighbor is 184.108.40.206, remote AS 65000, local AS 65000, internal link BGP version 4, remote router ID 0.0.0.6 BGP state = Established, up for 00:23:49 Last read 23:31:36, hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: 4 Byte AS: advertised and received Route refresh: advertised and received(old & new) Address family IPv4 Unicast: advertised and received Message statistics: Inq depth is 0 Outq depth is 0 Sent Rcvd Opens: 2 0 Notifications: 0 0 Updates: 1 1 Keepalives: 25 24 Route Refresh: 0 0 Capability: 0 0 Total: 28 25 Minimum time between advertisement runs is 5 seconds For address family: IPv4 Unicast >>>>>>>>>>>>>>>>>>>>>> ROUTE REFLECTOR CLIENT NOT DISPLAYED NEXT_HOP is always this router Community attribute sent to this neighbor(both) 6 accepted prefixes Connections established 1; dropped 0 Last reset never Local host: 220.127.116.11, Local port: 179 Foreign host: 18.104.22.168, Foreign port: 40290 Nexthop: 22.214.171.124 Nexthop global: 2001:ded:beef::2 Nexthop local: fe80::202:ff:fe00:4 BGP connection: non shared network Read thread: on Write thread: off cumulus@switch:$
Define in following order address-family ipv4 unicast neighbor 126.96.36.199 activate neighbor 188.8.131.52 next-hop-self neighbor 184.108.40.206 route-reflector-client >>> Must be after Activate exit-address-family neighbor 2001:ded:beef:2::1 remote-as 65000 address-family ipv6 unicast redistribute connected maximum-paths 4 maximum-paths ibgp 4 neighbor 2001:ded:beef:2::1 activate neighbor 2001:ded:beef:2::1 next-hop-self neighbor 2001:ded:beef:2::1 route-reflector-client >>> Must be after activate exit-address-family Runtime status after change:
|RN-65||Virtual links in Quagga's OSPFv2 are non-operational||Cumulus Networks testing has identified too many issues with virtual link support in Quagga's OSPFv2. The feature is unsupported.|
|RN-68||Blackhole/Unreachable/Prohibit route addition in IPv6 returns corresponding error codes||IPv6 route operations indicate the destination action via returned error codes. In the example shown below where an unreachable route is being added, the return code is:
#define ENETUNREACH 101 /* Network is unreachable */
|RN-70||ACL: Bridge traffic that matches a LOG ACTION rule is not logged in syslog||For example, a bridge with switch ports swp1, swp2, swp3 as bridge members is configured. ACL rules to LOG and DROP for icmp traffic are configured.
Ping requests are sent from host1 on swp1 to host3 on swp3, and the following was observed:
* Counters for both LOG and DROP ACL rules are incrementing properly, but the packets are not showing up on /var/log/syslog.
* Packets that are copied to the CPU from hardware for the LOG rule are dropped due to the check in kernel to disable software bridging for hardware bridged packets.
|RN-77||New routes/ECMPs can evict existing/installed||Cumulus Linux syncs routes between the kernel and the switching silicon. If the required resource pools in hardware fill up, new kernel routes can cause existing routes to move from being fully allocated to being partially allocated.
In order to avoid this, routes in the hardware should be monitored and kept below the ASIC limits.
For example, on systems with Trident+ chips, the limits are as follows:
routes: 16384 <<<< if all routes are ipv4 long mask routes 256 <<<< i.e., routes with a mask longer than the route mask limit route mask limit 64 host_routes: 8192 ecmp_nhs: 4044 ecmp_nhs_per_route: 52That translates to about 77 routes with ECMP NHs, if every route has the maximum ECMP NHs.
Monitoring this in Cumulus Linux is performed via the cl-resource-query command:
cumulus@switch:~# sudo cl-resource-query hosts : 3 all routes : 29 IP4 routes : 17 IP6 routes : 12 nexthops : 3 ecmp_groups : 0 ecmp_nexthops : 0 mac entries : 0 / 131072 bpdu entries : 500 / 512The resource to monitor is the ecmp_nexthops. If this count is close to 4044, new ECMPs may evict existing routes.
|RN-88||SNMP support for Quagga is NOT provided in Cumulus Linux||Cumulus Linux 2.2 does not provide SNMP support for Quagga.|
|RN-99||cl-img-clear-overlay is disabled if kernel is upgraded using apt-get||If you have upgraded the kernel using
|RN-103||In a VRR environment, the server that is bonded to the VRR switches could lose packets destined to the VRR's IP addresses for up to 15 seconds.||
In the following configuration:
. r1 . / \ . vrr1------vrr2 . \ / . host1
The hosts have bond interfaces where one sub-interface goes to switch, vrr1, and the other goes to the other switch, vrr2.
If the link between the host and one of the VRR switches goes down, it can take up to 15 seconds of the VRR switches to send out an ARP to clear the ARP cache on the host for the IP address on the bridge interface. This is because the host might not clear the ARP cache since the bond doesn't go down. Only a sub-interface in the bond goes down.
Steps to reproduce:
1. One of the the hosts connected to the VRR switches, ping the real IP addresses of the bridge.
2. On the same host, bring the active interface down with "ip link set down" and let the backup take over.
3. Ping the real IP addresses of the VRR switch that is connected to the active interface.
|RN-112||Enabling LACP support for non-L3/L4 modes||Issue:
The current LACP implementation only supports srcdestip (0x6) mode.
In order to use srcdestmac mode, use the following commands:
First, find the bond name to hardware ID mapping:
cumulus@switch:/var/log# sudo kill -SIGRTMIN+5 `pidof switchd`
Based on the mapping, run the following command, where psc id is the HAL:x:
cumulus@switch:$ sudo /usr/lib/cumulus/bcmcmd trunk psc id=1 rtag=0x3
1. The HAL ID is a non-persistent ID.
2. If the bond interface goes down or up, you need to do this again.
Verify the commands:
srcdestmac mode 0x3== platform dni-7448-05
|RN-116||Bridge driver issues affecting IGMP snooping behavior on STP topology change||Issue:
The Cumulus Linux bridge driver does not adhere to the IETF standard for IGMP snooping during an STP topology change.
On an STP topology change, RFC 4541, section 2.1.1, point 4 (https://tools.ietf.org/html/rfc4541, copied below) suggests what an IGMP snooping switch should do to reduce network convergence; this is not present in the bridge driver.
In addition, the bridge driver does not send a general query on receiving a global leave.
4) An IGMP snooping switch should be aware of link layer topology changes
|RN-119||LLDP frames being reported as software RX drops when received on bridge interfaces||Issue:
RX drops have been reported on interfaces (using cl-netstat) that are not reflected in hardware, when they are actually received LLDP frames.
Steps to reproduce:
|RN-120||ethtool LED blinking does not work with switch ports||Linux uses ethtool -p to identify the physical port backing an interface, or to identify the switch itself. Usually this identification is by blinking the port LED until ethtool -p is stopped.
This feature does not apply to switch ports (swpX) in Cumulus Linux.
|RN-121||PTMD: When a physical interface is in a PTM FAIL state, its subinterface still exchanges information||Issue:
When PTMD is incorrectly in a failure state and the Zebra interface is enabled, PIF BGP sessions are not establishing the route, but the subinterface on top of it does establish routes.
If the subinterface is configured on the physical interface and the physical interface is incorrectly marked as being in a PTM FAIL state, routes on the physical interface are not processed in Quagga, but the subinterface is working.
Steps to reproduce:
cumulus@switch:$ sudo vtysh -c 'show int swp8'
|RN-125||Network LSA with an old router ID isn't flushed out by the originator||Issue:
When the router ID is changed, the router should remove the previous network LSA (link-state advertisement) that it generated based on the IP address on the interface in the Network LSA.
Cumulus Networks isn't removing this LSA, so it will be naturally aged out.
|RN-128||Quagga does not start by default in Cumulus Linux 2.2||To start Quagga, modify /etc/quagga/daemons to enable the corresponding daemons.
zebra=yes (* this one is mandatory to get the others up)
Then, restart Quagga.
cumulus@switch1:~# sudo /etc/init.d/quagga start
|RN-132||You must run "apt-get update" before running any apt-get commands or after changing sources.list||
Before running any apt-get commands or after changing the source.list file in /etc/apt, you need to run apt-get update.
|RN-133||Interface names in Cumulus Linux cannot exceed 15 characters||
Device names, including interface names, in Cumulus Linux cannot exceed 16 characters – including the terminator. Cumulus Linux truncates longer interface names.
To avoid this issue, do not assign long names to your interfaces.
The following example configuration reproduces this issue:
cumulus@switch:/sys/class/net$ grep 'iface br' /etc/network/interfaces iface br2-pubmgmt inet static iface br3-prvmgmt inet manual iface br400-quarantine inet manual iface br401-peering-1k5 inet manual iface br402-peering-9k inet manual iface br500-pi-exa inet manual iface br501-akamai-exa inet manual iface br502-exa-internetfactory inet manual cumulus@switch:/sys/class/net$ brctl show | grep br bridge name bridge id STP enabled interfaces br2-pubmgmt 8000.089e01cebe37 no bond0.2 br3-prvmgmt 8000.089e01cebe3a no bond0.3 br400-quarantin 8000.089e01cebe37 no bond0.400 br401-peering-1 8000.089e01cebe3a no bond0.401 <<<
|RN-134||Installing Chef under Cumulus Linux||
The Cumulus Linux 2.2 repository contains two versions of Chef, the automation tool: 11.6.2 (the current version) and 10.30.4.
To install the latest version, connect to the switch and use
cumulus@switch:~# sudo apt-get install chef
To install 10.30.4, connect to the switch and use apt-get:
cumulus@switch:~# sudo apt-get install chef=10.30.4-0.debian.7.3
|RN-141||ACL counters incorrect||
Although ACL rules work, ACL counters for any ACL rules in ebtables, iptables, i6tables, are all incorrect. This bug is being analyzed and worked on, and a workaround/fix will be provided in a future release.
|RN-143||TERR/RFCS seen when switching direction from 40G ingress to 10G egress port, when same share pool uses different egress CoS queue.||
Under some congestion conditions, packets that should be dropped may be truncated, marked as error packets, and forwarded.
cumulus@switch:~# sudo ethtool -S swp48 | grep Error
This is a known issue that should be fixed in a future release.
|RN-147||PSU status is not properly represented by LEDs on Quanta LY2 switches||
On Quanta LY2 switches, a sysled-mgmt unhandled exception error can occur, which will lead to systems LEDs remaining off and improperly reflecting the PSU status. Normally the LEDs flash yellow to indicate when either of the two units are NOT functioning properly.
This is a known issue that will be fixed in a future release.
The following error should be seen in
Apr 14 22:27:11 cumulus : /usr/sbin/smond : : Health of system unit changed to ERROR
|RN-150||Tagged packets have their 802.1p value set to 0||
All the tagged packets get their 802.1p priority value set to 0.
This is a known issue that should be fixed in a future release.
|RN-153||BGP ECMP x64 topology is missing routes||
Sometimes in an ECMP x64 topology, the nodes learn fewer paths to a route than the expected 64.
The issue arises because Cumulus Linux bring up peers very quickly and sometimes a peer comes up before Zebra has finished providing the OS with all connected routes (Zebra identifies all of the connected routes to BGP; BGP then sanity checks the next hops from the EBGP peers against that list).
This issue will be fixed in a future release of Cumulus Linux.
|RN-161||Packets on local ports get dropped on admin state change of VXLAN instance attached to bridge||
Packets between local ports of a bridge will get dropped momentarily when user changes the admin state of a VXLAN instance attached to the bridge (as in, when running "ip link set up/down"). Bridge attributes in the hardware are modified on the state change, which causes packets between member ports of the bridge to get dropped.
There is no workaround at this time; traffic should be stopped before changing the admin state of an attached VXLAN instance.
|RN-162||Priority Flow Control doesn't work on Trident II switches||
Priority Flow Control (PFC) configuration is not correct for switches on the Trident II platform. As a result, PFC doesn't work.
There is no workaround at this time.
|RN-163||VXLAN: ovsdb-server cannot select loopback interface as source IP address, causing TOR registration to the controller to fail||
In a VXLAN using VMware NSX, ovsdb-server cannot select the loopback interface as the source IP address. This causes TOR registration to the controller to fail.
To work around this issue, run:
cl-bgp redistribute add connected
|RN-164||IFLA_VXLAN_SERVICE_NODE incompatible with upstream kernel||
This issue will be fixed in a future release of Cumulus Linux.
|RN-165||Quanta LY6 switch has memory parity error _soc_mem_array_sbusdma_read: L2_ENTRY.ipipe0 failed(ERR)||
On a Quanta LY6 switch, you may see some memory parity errors in the
switchd.log.1.gz:1402208356.479833 2014-06-08 06:19:16 sync.c:2803 IPv4 Route Summary (90) : 0 Added, 1 Deleted, 0 Updated in 30943 usecs switchd.log.1.gz:1402208356.521537 2014-06-08 06:19:16 hal_acl_bcm.c:2352 ACL: installation succeeded, switched over switchd.log.1.gz:1402208357.679543 2014-06-08 06:19:17 sync.c:2803 IPv4 Route Summary (91) : 1 Added, 299 Deleted, 0 Updated in 220336 usecs switchd.log.1.gz:1402208357.719811 2014-06-08 06:19:17 hal_acl_bcm.c:2352 ACL: installation succeeded, switched over switchd.log.1.gz:1402208359.486625 2014-06-08 06:19:19 sync.c:2803 IPv4 Route Summary (92) : 299 Added, 0 Deleted, 0 Updated in 200425 usecs switchd.log.1.gz:1402208361.042769 2014-06-08 06:19:21 hal_acl_bcm.c:2352 ACL: installation succeeded, switched over switchd.log.1.gz:1402208419.059525 2014-06-08 06:20:19 hal_bcm.c:408 caught a parity error of type parity data error: 0x4000001, 0x1c0043cd switchd.log.1.gz:1402208419.059594 2014-06-08 06:20:19 switchd.c:533 No switchd restart: restart trigger has been disabled switchd.log.1.gz:1402208419.061002 2014-06-08 06:20:19 hal_bcm.c:408 caught a parity error of type corrected data error: 0x7d6, 0x43cd switchd.log.1.gz:1402208419.061032 2014-06-08 06:20:19 switchd.c:533 No switchd restart: restart trigger has been disabled switchd.log.1.gz:1402208419.061091 2014-06-08 06:20:19 hal_bcm_console.c:169 WARN STATUS: 0x00000083 switchd.log.1.gz:OPCODE: 0x1c110200 switchd.log.1.gz:START ADDR: 0x04790180 switchd.log.1.gz:CUR ADDR: 0x1c004394 switchd.log.1.gz:_soc_mem_array_sbusdma_read: L2_ENTRY.ipipe0 failed(ERR) switchd.log.1.gz:H/W received sbus nack with error bit set. switchd.log.1.gz:Unit: 0 switchd.log.1.gz: switchd.log.1.gz:Mem: Parity error.. switchd.log.1.gz:Error in: SBUS transaction. switchd.log.1.gz:Blk: 1, Pipe: 0, Address: 0x1c0043cd, base: 0x0, stage: 7, index: 17357
While troubleshooting this issue, the error occurred only once. If you encounter this error more than once, please submit a support request.
10GTek 10G SR cables exhibit high rate of errors on Penguin Arctica 4804X switch
Some PHY-less Penguin Arctica 4804X platforms using 10GTek 10G MM SR cables exhibit high rates of errors and low bandwidth one direction.
JDSU QSFP+ LR4 cable presence not detected on Edge-Core AS-6701 switch
ECMP paths not inserted for directly connected unnumbered neighbors
Cumulus Linux does not insert multiple paths for directly connected ununumbered neighbors into hardware; it inserts only one, as determined by
This may present a problem in VXLAN configurations were a VTEP's neighbors are directly adjacent (that is, the spine switch is the VTEP) and you want to use ECMP for the tunneled traffic. If only your leaf switches are the VTEPs, this issue will not occur.
|RN-182||ICMP redirects occur on host while pinging bridge IP in VRR active-active topology (VRR and Host-MLAG)||
Sometimes an ICMP request may go to a peer switch; as a result, the peer switch sends ICMP redirect messages. This occurs in a VRR active-active topology (VRR and Host-MLAG), when the host pings any of the bridge IP addresses, because both links are active.
cumulus@host2$ ping 220.127.116.11 PING 18.104.22.168 (22.214.171.124) 56(84) bytes of data. 64 bytes from 126.96.36.199: icmp_req=1 ttl=64 time=1.23 ms 64 bytes from 188.8.131.52: icmp_req=2 ttl=64 time=0.882 ms 64 bytes from 184.108.40.206: icmp_req=3 ttl=64 time=0.832 ms 64 bytes from 220.127.116.11: icmp_req=4 ttl=64 time=0.960 ms From 18.104.22.168: icmp_seq=2 Redirect Host(New nexthop: 22.214.171.124) <<<<<<<<<<<<<<<<< From 126.96.36.199: icmp_seq=3 Redirect Host(New nexthop: 188.8.131.52) From 184.108.40.206: icmp_seq=4 Redirect Host(New nexthop: 220.127.116.11) 64 bytes from 18.104.22.168: icmp_req=5 ttl=64 time=0.985 ms From 22.214.171.124: icmp_seq=6 Redirect Host(New nexthop: 126.96.36.199) 64 bytes from 188.8.131.52: icmp_req=6 ttl=64 time=1.01 ms 64 bytes from 184.108.40.206: icmp_req=7 ttl=64 time=1.09 ms ^C --- 220.127.116.11 ping statistics --- 7 packets transmitted, 7 received, 0% packet loss, time 6022ms rtt min/avg/max/mdev = 0.832/1.001/1.238/0.126 ms cumulus@host2$ cumulus@host2$ arp Address HWtype HWaddress Flags Mask Iface jump-01.cumulusnetworks ether 52:55:c0:a8:00:03 C eth0 192.168.0.2 ether 52:55:c0:a8:00:02 C eth0 18.104.22.168 ether 00:00:5e:00:01:02 C bond0 vrrp.cbbtier3.att.net ether 00:00:5e:00:01:02 C bond0 22.214.171.124 ether 00:00:5e:00:01:02 C bond0 cumulus@host2$
Link not coming up (NO-CARRIER) on Penguin Arctica 4804xp with CAB-10GSFP-P9M 10Gtek 9 meter cable
The link does not come up on a Penguin Arctica 4804xp 720G PHY-less switch with 10Gtek 9 meter cables (CAB-10GSFP-P9M). You can determine this by running:
root@switch:~# ip link show swp15 17: swp15: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 500 link/ether 44:38:39:00:70:a1 brd ff:ff:ff:ff:ff:ff
Cumulus Linux supports copper passive cables no longer than 7m.
BGP attribute-unchanged next-hop flag does not work for IPv6 peers
This has been reported on switches where BGP sessions are IPv6 and carry IPv4 prefixes. The
For IPv4 sessions, BGP behaves correctly.
On Agema 7448 switches with QSFPs that have alarm and warning threshold flags and values set,
This is because
Cannot configure 10M or 100M speeds using ethtool on 1G copper ports
You cannot use
root@switch:~# /usr/lib/cumulus/bcmcmd port ge1 speed=100
To set a 10Mbps port speed:
root@switch:~# /usr/lib/cumulus/bcmcmd port ge1 speed=10
Rolling back installation causes Debian packages to be unusable
When you run apt-get install, Cumulus Linux creates a new snapshot automatically if at least one file is added to /etc. If you roll back the installation using cl-persistify, the installer reverts the /etc directory to the specific snapshot, but does not do anything with installed packages.
As a result, these packages will be unusable because missing configuration files. The Cumulus Linux installer does not record which Debian packages changed after a rollback. As a workaround, you can try installing again.
If you are using the
|RN-189||Fan on QuantaMesh LB9 marked as absent||
Fan modules on a QuantaMesh LB9 switch are marked as ABSENT because Cumulus Linux doesn't distinguish between F2B and B2F modes.
root@switch:~# pwmd -d QuantaLB9PWM1 Fans: Fan1 PWM: 204 Measured RPM: 13333 Fan2 PWM: 204 Measured RPM: 13106 Fan3 PWM: 204 Measured RPM: 13432 Fan4 PWM: 204 Measured RPM: 13106 Fan5 PWM: 204 Measured RPM: 0 Fan6 PWM: 204 Measured RPM: 0 Fan7 PWM: 204 Measured RPM: 0 Fan8 PWM: 204 Measured RPM: 0 Temps: Temp1 46 C Temp2 45 C Temp3 43 C Temp4 41 C Temp5 45 C
You can use smonctl to get more information on temperatures and fans:
root@switch:~# smonctl Fan1 : OK >>> F2B Fan2 : OK >>> F2B Fan3 : OK >>> F2B Fan4 : OK >>> F2B Fan5 : ABSENT >>>B2F Fan6 : ABSENT >>>B2F Fan7 : ABSENT >>>B2F Fan8 : ABSENT >>>B2F PSU1 : BAD PSU2 : OK Temp1 (Fan outlet Sensor ): OK Temp2 (MPC8541(CPU) Sensor ): OK Temp3 (Front Right Side ): OK Temp4 (Front Left Side ): OK Temp5 (Temp Sensor close to Networking ASIC ): OK root@qct-lb9-09:~# sensors -A adt7470-i2c-5-2c fan1: 13075 RPM (min = 2500 RPM, max = 14025 RPM) fan2: 13267 RPM (min = 2500 RPM, max = 14025 RPM) fan3: 13170 RPM (min = 2500 RPM, max = 14025 RPM) fan4: 13466 RPM (min = 2500 RPM, max = 14025 RPM) temp1: +45.0 C (low = +5.0 C, high = +50.0 C) temp2: +45.0 C (low = +5.0 C, high = +80.0 C) temp3: +43.0 C (low = +5.0 C, high = +58.0 C) temp4: +42.0 C (low = +5.0 C, high = +58.0 C) temp5: +46.0 C (low = +5.0 C, high = +80.0 C) adt7470-i2c-6-2f root@switch:~#
clagd unhandled traceback exception: "OSError: [Errno 2] No such file or directory: '/var/run/clagd.pid'" after running service networking restart then clagd restart
In Host-MLAG configurations, the following unhandled traceback exception occurs after running
OSError: [Errno 2] No such file or directory: '/var/run/clagd.pid'
You can see the traceback in the
Tue Jul 29 01:23:14 2014 clean exit Tue Jul 29 01:23:14 2014 error: unhandled exception: Traceback (most recent calll last): File "/usr/sbin/clagd", line 3272, in main ClagRun() File "/usr/sbin/clagd", line 3256, in ClagRun while not stopEvent.wait(60*60*24): File "/usr/lib/python2.7/threading.py", line 404, in wait self.__cond.wait(timeout) File "/usr/lib/python2.7/threading.py", line 263, in wait _sleep(delay) File "/usr/sbin/clagd", line 2353, in signal.signal(signal.SIGTERM, lambda signum=0,frame=None: self.exit()) File "/usr/sbin/clagd", line 2381, in exit os.unlink(self.pid_file) OSError: [Errno 2] No such file or directory: '/var/run/clagd.pid'
On Agema 7448 switch, QSFP vendor information is corrupted
You can see this when you run
cumulus@switch:~# sudo ethtool -m swp51 7:14 Vendor name : __9____________` Vendor OUI : 23:1b:36 Vendor PN : _=______________ Vendor rev : _ Vendor SN : ________________ 7:14 root@dni-7448-32:~# ethtool -m swp50 7:14 Vendor name : __>___________$_ Vendor OUI : 39:45:40 Vendor PN : /_O_CbQ6N_ _6S+U Vendor rev : 7 Vendor SN : ________________ 7:14 All other look fine.
PTM may crash when a large topology file has a syntax error
If there is a syntax error in a large (~ 4000 entries)
On Edge-Core AS5610 switches, SFP+ (all optics) link stays down
You can correct this issue after you upgrade to the 2.2 release. Follow these steps.
Security Update for apt and bash packages: Shellshock bug fix
For information on updating Cumulus Linux to address this issue, read this article.
Cumulus Networks has created a patch for
|RN-313||High memory utilization by
When performing an MIB walk on Cumulus Linux running the standard NET-SNMP 5.4.3 package, memory is consumed and not returned. Over time, this can lead to sub-optimal performance and ultimately can cause the system to hang and require a reboot.
|Security Update for CVE-2015-7547: glibc getaddrinfo Stack-based Buffer Overflow Vulnerability||For details on this issue and how to upgrade, read this article.|