This knowledge base has moved to the documentation site. Please visit the knowledge base here for the most up to date content. This site is no longer maintained.

Cumulus Linux 3.3.2 Release Notes



These release notes support Cumulus Linux 3.3.2 and describe currently available features and known issues. 

Stay up to Date 

  • Please sign in and click Follow above so you can receive a notification when we update these release notes.
  • Subscribe to our product bulletin mailing list to receive important announcements and updates about issues that arise in our products.
  • Subscribe to our security announcement mailing list to receive alerts whenever we update our software for security issues.


What's New in Cumulus Linux 3.3.2

Cumulus Linux 3.3.2 contains bug fixes only.

Early Access Features

The following early access features are included in Cumulus Linux 3.3.2:

  • QinQ: For hybrid cloud connectivity.
  • VXLAN routing: For IP routing between VXLAN VNIs in an overlay network.

Note: The EA version of netq is not supported under Cumulus Linux 3.3.2.


Cumulus Linux is licensed on a per-instance basis. Each network system is fully operational, enabling any capability to be utilized on the switch with the exception of forwarding on switch panel ports. Only eth0 and console ports are activated on an un-licensed instance of Cumulus Linux. Enabling front panel ports requires a license.

You should have received a license key from Cumulus Networks or an authorized reseller. To install the license, read the Cumulus Linux Quick Start Guide.

Installing Version 3.3.2

If you are upgrading from version 3.0.0 or later, use apt-get to update the software.

Cumulus Networks recommends you use the -E option with sudo whenever you run any apt-get command. This option preserves your environment variables — such as HTTP proxies — before you install new packages or upgrade your distribution.

  1. Retrieve the new version packages: 
    cumulus@switch:~$ sudo apt-get update
  2. If you are using any early access features from an older release, remove them with:
    cumulus@switch:~$ sudo apt-get remove EA_PACKAGENAME
  3. Upgrade the release: 
    cumulus@switch:~$ sudo apt-get upgrade
  4. Reboot the switch:
    cumulus@switch:~$ sudo reboot

New Install or Upgrading from Versions Older than 3.0.0

If you are upgrading from a version older than 3.0.0, or installing Cumulus Linux for the first time, download the Cumulus Linux 3.3.2 installer for Broadcom or Mellanox switches from the Cumulus Networks website, then use ONIE to perform a complete install, following the instructions in the quick start guide.

Note: This method is destructive; any configuration files on the switch will not be saved, so please copy them to a different server before upgrading via ONIE.

Important! After you install, run apt-get update, then apt-get upgrade on your switch to make sure you update Cumulus Linux to include any important or other package updates.

Updating a Deployment that Has MLAG Configured

If you are using MLAG to dual connect two switches in your environment, and those switches are still running Cumulus Linux 2.5 ESR or any other release earlier than 3.0.0, the switches will not be dual-connected after you upgrade the first switch. To ensure a smooth upgrade, follow these steps:

  1. Run cl-img-select -fr to boot the switch in the secondary role into ONIE, then reboot the switch.
  2. Install Cumulus Linux 3.3.2 onto the secondary switch using ONIE. At this time, all traffic is going to the switch in the primary role.
  3. After the install, copy the license file and all the configuration files you backed up, then restart the switchd, networking and Quagga services. All traffic is still going to the primary switch.
    cumulus@switch:~$ sudo systemctl restart switchd.service
    cumulus@switch:~$ sudo systemctl restart networking.service
    cumulus@switch:~$ sudo systemctl restart quagga.service
  4. Run cl-img-select -fr to boot the switch in the primary role into ONIE, then reboot the switch. Now, all traffic is going to the switch in the secondary role that you just upgraded to version 3.3.2.
  5. Install Cumulus Linux 3.3.2 onto the primary switch using ONIE. 
  6. After the install, copy the license file and all the configuration files you backed up.
  7. Disable clagd in the /etc/network/interfaces file (set clagd-enable to no), then restart the switchd, networking and Quagga services.
    cumulus@switch:~$ sudo systemctl restart switchd.service
    cumulus@switch:~$ sudo systemctl restart networking.service
    cumulus@switch:~$ sudo systemctl restart quagga.service
  8. Enable clagd again in the /etc/network/interfaces file (set clagd-enable to yes), then run ifreload -a.
    cumulus@switch:~$ sudo ifreload -a
  9. Now the two switches are dual-connected again and traffic flows to both switches.

 Perl, Python and BDB Modules

Any Perl scripts that use the DB_File module or Python scripts that use the bsddb module won't run under Cumulus Linux 3.3.2.


You can read the technical documentation here.

Issues Fixed in Cumulus Linux 3.3.2 Update 2017-07-14

Cumulus Networks has made important package updates available for Cumulus Linux 3.3.2 that resolve the issues listed below. These fixes were applied to the Cumulus Networks repository on July 14, 2017.

Cumulus Networks strongly recommends you upgrade your Cumulus Linux distribution to avoid these issues (do not do a binary install). Follow these steps:

  1. Run apt-get update.
  2. Run apt-get upgrade
Release Note ID Summary Description
RN-637 (CM-16222)
Cannot configure VLAN filtering/pruning using NCLU

Running net add interface swp# bridge vids <vid> does not add a bridge-vids configuration under an interface.

This issue has been fixed in the July 14, 2017 update release to Cumulus Linux 3.3.2.

RN-638 (CM-16966)
NCLU netd service crashes if sysname is missing in LLDP output

If the sysname information is not included in the LLDP peer information, netd crashes when running net show interface.

This issue has been fixed in the July 14, 2017 update release to Cumulus Linux 3.3.2.

RN-639 (CM-12534)
`netshow int all -j` returns traceback AttributeError: 'super' object has no attribute 'stp'

If a bridge is configured with STP disabled, the netd daemon returns a traceback error.

This issue has been fixed in the July 14, 2017 update release to Cumulus Linux 3.3.2.

Issues Fixed in Cumulus Linux 3.3.2 Update 2017-07-06

Cumulus Networks has made important package updates available for Cumulus Linux 3.3.2 that resolve the issues listed below. These fixes were applied to the Cumulus Networks repository on July 6, 2017.

Cumulus Networks strongly recommends you upgrade your Cumulus Linux distribution to avoid these issues (do not do a binary install). Follow these steps:

  1. Run apt-get update.
  2. Run apt-get upgrade
Release Note ID Summary Description
RN-636 (CM-16191)
On Mellanox switches, the MTU for a bond slave is not set if that interface was UP previously

This issue has been fixed in the July 6, 2017 update release to Cumulus Linux 3.3.2.

Issues Fixed in Cumulus Linux 3.3.2

The following is a list of issues fixed in Cumulus Linux 3.3.2 from earlier versions of Cumulus Linux.

Release Note ID Summary Description

RN-536 (CM-12628)
Port LEDs on ARM 1G-T platforms do not show link status (green), but blink on activity

On the DNI-3048UP, Edge-Core 4610-54P and HP 4610-54T, there is no link LED for 1G-T ports. The SFP+ ports correctly show the link LEDs.

The Dell S3048-ON, which has a Helix4 ASIC, correctly shows the link LED.

This issue is currently being investigated.

RN-630 (CM-16563)
switchd does not program routes with same link-local via different interfaces on Mellanox switches

switchd failed to program routes with the same link-local nexthop if they came from different interfaces. This was because switchd hal sync needed to add neigh_ifi as part of the key to create multiple hosts routes for these neighbor entries.

This has been corrected on Mellanox switches only. A fix for Broadcom switches is pending additional fixes.

RN-631 (CM-16441)
VXLAN/UDP 4789 datagrams dropped on ingress when using traditional bridge

A regression was found when using traditional bridging in Cumulus Linux 3.3, where the switch dropped any packets that arrived with both a dot1q tag and the UDP dst-port 4789.

This issue is fixed in Cumulus Linux 3.3.2.

RN-632 (CM-16783)
rsyslog stops logging after 4M filesize after upgrade to 3.3.1 

A change in the upgrade process from earlier releases caused a necessary file to be removed, preventing logging from occurring correctly. Customers upgrading to Cumulus Linux 3.3 or later from either Cumulus Linux 3.0.x or 3.1.x should check that the /etc/logrotate.d/ryslog file exists. If it does not, reinstall rsyslog with the following command:

apt-get -o DPkg::options::=--force-confmiss --reinstall install rsyslog

This issue is fixed in Cumulus Linux 3.3.2.

RN-633 (CM-16782)
Extraneous unicode characters in EEPROM break ZTP on Mellanox switches 

When running decode-syseeprom --json on Mellanox switches running Cumulus Linux 3.3.1, extraneous unicode characters were printed in the JSON output, breaking zero touch provisioning (ZTP). This was caused by a fixed string length configuration in the EEPROM programming. The extraneous characters have been removed from the output.

This issue is fixed in Cumulus Linux 3.3.2.

RN-634 (CM-16292)
net show interface slow to return results 

The NCLU command net show interface takes approximately 9 seconds to return results.

This issue is fixed in Cumulus Linux 3.3.2.

Known Issues in Cumulus Linux 3.3.2

The following issues are open and affect the current release.

Release Note ID Summary Description

RN-52 (CM-997,
Parameters like the router ID and DR priority cannot be changed while OSPFv2/v3 is running Router ID and DR priority can only be changed by shutting down OSPFv2/v3, changing the ID, and restarting the OSPF process.

A change to the DR priority may not properly be reflected in the LSAs that are still aging out.

RN-56 (CM-343)
IPv4/IPv6 forwarding disabled mode not recognized

If either of the following is configured:

net.ipv4.ip_forward == 0 


net.ipv6.conf.all.forwarding == 0 

The hardware still forwards packets if there is a neighbor table entry pointing to the destination.

RN-77 (CM-265)
New routes/ECMPs can evict existing/installed Cumulus Linux syncs routes between the kernel and the switching silicon. If the required resource pools in hardware fill up, new kernel routes can cause existing routes to move from being fully allocated to being partially allocated.

In order to avoid this, routes in the hardware should be monitored and kept below the ASIC limits.

For example, on systems with Trident+ chips, the limits are as follows:
routes: 16384 <<<< if all routes are ipv4 
 long mask routes 256 <<<< i.e., routes with a mask longer 
       than the route mask limit 
 route mask limit 64
 host_routes: 8192 
 ecmp_nhs: 4044 
 ecmp_nhs_per_route: 52 
That translates to about 77 routes with ECMP NHs, if every route has the maximum ECMP NHs.

Monitoring this in Cumulus Linux is performed via the cl-resource-query command:
cumulus@switch:~$ sudo cl-resource-query
 hosts : 3 
 all routes : 29 
 IP4 routes : 17 
 IP6 routes : 12 
 nexthops : 3 
 ecmp_groups : 0
 ecmp_nexthops : 0
 mac entries : 0 / 131072 
 bpdu entries : 500 / 512
The resource to monitor is the ecmp_nexthops. If this count is close to 4044, new ECMPs may evict existing routes.

RN-120 (CM-477)
ethtool LED blinking does not work with switch ports Linux uses ethtool -p to identify the physical port backing an interface, or to identify the switch itself. Usually this identification is by blinking the port LED until ethtool -p is stopped.

This feature does not apply to switch ports (swpX) in Cumulus Linux.

RN-121 (CM-2123)
PTMD: When a physical interface is in a PTM FAIL state, its subinterface still exchanges information Issue:
When PTMD is incorrectly in a failure state and the Zebra interface is enabled, PIF BGP sessions are not establishing the route, but the subinterface on top of it does establish routes.

If the subinterface is configured on the physical interface and the physical interface is incorrectly marked as being in a PTM FAIL state, routes on the physical interface are not processed in Quagga, but the subinterface is working.

Steps to reproduce:
cumulus@switch:$ sudo vtysh -c 'show int swp8' 
Interface swp8 is up, line protocol is up 
PTM status: fail
index 10 metric 1 mtu 1500 
 HWaddr: 44:38:39:00:03:88 
 inet broadcast 
 inet6 2001:cafe:0:38::1/64 
 inet6 fe80::4638:39ff:fe00:388/64 
cumulus@switch:$ ip addr show | grep swp8 
  mtu 1500 qdisc pfifo_fast state UP qlen 500 
  inet brd scope global swp8 
 104: swp8.2049@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet brd scope global swp8.2049 
 105: swp8.2050@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet brd scope global swp8.2050 
 106: swp8.2051@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet brd scope global swp8.2051 
 107: swp8.2052@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet brd scope global swp8.2052 
 108: swp8.2053@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP>
  mtu 1500 qdisc noqueue state UP 
  inet brd scope global swp8.2053 
 109: swp8.2054@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet brd scope global swp8.2054
 110: swp8.2055@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP>
  mtu 1500 qdisc noqueue state UP 
  inet brd scope global swp8.2055
cumulus@switch:$ bgp sessions: ,4 ,64057 , 958 , 1036 , 0 , 0 , 0 ,15:55:42, 0, 10472 ,4 ,64058 , 958 , 1016 , 0 , 0 , 0 ,15:55:46, 187, 10285 ,4 ,64059 , 958 , 1049 , 0 , 0 , 0 ,15:55:40, 187, 10285 ,4 ,64060 , 958 , 1039 , 0 , 0 , 0 ,15:55:45, 187, 10285 ,4 ,64061 , 958 , 1014 , 0 , 0 , 0 ,15:55:46, 187, 10285 ,4 ,64062 , 958 , 1016 , 0 , 0 , 0 ,15:55:46, 187, 10285 ,4 ,64063 , 958 , 1029 , 0 , 0 , 0 ,15:55:43, 187, 10285 ,4 ,64064 , 958 , 1036 , 0 , 0 , 0 ,15:55:44, 187, 10285 

RN-125 (CM-1576)
Network LSA with an old router ID isn't flushed out by the originator
When the router ID is changed, the router should remove the previous network LSA (link-state advertisement) that it generated based on the IP address on the interface in the Network LSA.

Cumulus Networks doesn't remove this LSA, so it will be naturally aged out.

RN-198 (CM-3290)
Port LEDs behave differently on different switch models

It's been observed that port LEDs behave differently depending upon the make and model of the switch. For example:

  • Agema AG-7448CU: the LED is off when the link is up. It blinks on briefly when there is traffic.
  • Edge-Core AS4600-54T: the LED is off when the link is up. It blinks on briefly when there is traffic.
  • QuantaMesh T3048-LY2R: the LED is on when the link is up. It blinks off briefly when there is traffic.

Cumulus Networks is currently working to fix this issue.

RN-199 (CM-2624)
When a Quagga route-map is modified, the switch could use the partial map before edits are completed

Cumulus Linux triggers a route-map update before the user finishes editing the route map, resulting in an incorrect route map being used. The route-map update trigger should only occur when user finishes editing the map.

Cumulus Networks is working to fix this issue.

RN-221 (CM-3926, CM-4501)
BGP graceful restart, including helper mode, not fully supported If you encounter issues with this, please submit a support request and include the output from cl-support with your ticket.

RN-227 (CM-3388)
BGP dynamic capability is not supported BGP peer sessions with dynamic capability are not supported under any version of Cumulus Linux at this time.

RN-322 (CM-7387)
Interfaces disabled using iproute2 become enabled after restarting Quagga By default, all interfaces have a "no shutdown" associated with them in Quagga. Thus, when you restart Quagga, it enables the interfaces. This is expected behavior in Quagga. There is no workaround at this time.

RN-327 (CM-4290)
Changing the route-map parameter of the redistribute command in OSPF and BGP doesn't affect the state of the resulting redistribution in those protocols

To work around this issue, remove any old redistribute command configurations before adding a new one with or without route-map as a parameter.

For example, if OSPF has a redistribute configuration such as redistribute bgp route-map redist-map-name, you would enable redistribution without a route-map by following these steps in OSPF configuration mode:

  1. no redistribute bgp
  2. redistribute bgp

You would perform a similar sequence of commands for redistribution changes in BGP as well.

RN-355 (CM-7994)
OSPFv2 Area ID being implicitly translated from Integer format to dotted decimal format

While OSPF area ID configuration in Quagga allows for the value to be specified in either dotted decimal format, or as an integer, values specified as an integer will be converted into dotted decimal format when displayed, causing potential confusion for the operator.

This issue does not impact OSPF functionality; only the display output. However, it is recommended that the OSPF area ID is specified in dotted decimal format for consistency.


RN-382 (CM-6692)
Quagga: Removing bridge via ifupdown2 does not remove it from Quagga Removing a bridge using ifupdown2 does not remove it from the Quagga configuration files. This issue is being investigated; however, restarting Quagga will successfully remove the bridge.

RN-384 (CM-7684)
Keeping VXLAN single-connected devices up on MLAG secondary node In the current MLAG secondary design, if the VXLAN device is not dual-connected, it is kept in a protodown state. You can keep them up with individual IP addresses rather than anycast IPs when the peerlink is down, so that all single-connected hosts will have connectivity. Further investigation regarding this issue is underway.

RN-387 (CM-8163)
Quagga appears to not honor passive interfaces if VRR is active

In a VRR configuration, any interface-specific routing configuration (e.g., OSPF mode of operation) specified on the subinterface having a virtual IP address does not take effect. This is because when an operator has specified a virtual IP on a bridge, the system creates another internal interface bridge with the virtual IP and MAC. These two interfaces are treated distinctly by Quagga, so any interface-specific routing configuration on the bridge does not get carried over to the second bridge.

In a VRR deployment needing any interface-specific routing configuration on the interface with a virtual IP address, the routing configuration has to be specified against the internally-created virtual interface also.

RN-389 (CM-8410)
switchd supports only port 4789 as the UDP port for VXLAN packets

switchd currently allows only the standard port 4789 as the UDP port for VXLAN packets. There are cases where a hypervisor could be using non-standard UDP port, which would cause VXLAN exchanges with the hardware VTEP to not work. In such a case, packets would not be terminated and encapsulated packets would be sent out on UDP port 4789.

RN-391 (CM-9631)
Dell S4048 unresponsive after TX Unit Hang detected

After booting a Dell S4048 switch, the switch becomes unresponsive and errors like the following appear in the console log:

[ 1206.440277] igb 0000:00:14.0: Detected Tx Unit Hang
[ 1206.440277]   Tx Queue             <0>
[ 1206.440277]   TDH                  <2d>
[ 1206.440277]   TDT                  <2e>
[ 1206.440277]   next_to_use          <2e>
[ 1206.440277]   next_to_clean        <2d>
[ 1206.440277] buffer_info[next_to_clean]
[ 1206.440277]   time_stamp           <1000dcd20>
[ 1206.440277]   next_to_watch        <ffff88007d81b2d0>
[ 1206.440277]   jiffies              <1000dd5d4>
[ 1206.440277]   desc.status          <300000>
[ 1208.439856] igb 0000:00:14.0: Detected Tx Unit Hang
[ 1208.439856]   Tx Queue             <0>
[ 1208.439856]   TDH                  <2d>
[ 1208.439856]   TDT                  <2e>
[ 1208.439856]   next_to_use          <2e>
[ 1208.439856]   next_to_clean        <2d>
[ 1208.439856] buffer_info[next_to_clean]
[ 1208.439856]   time_stamp           <1000dcd20>
[ 1208.439856]   next_to_watch        <ffff88007d81b2d0>
[ 1208.439856]   jiffies              <1000ddda4>
[ 1208.439856]   desc.status          <300000>
[ 1210.439414] igb 0000:00:14.0: Detected Tx Unit Hang
[ 1210.439414]   Tx Queue             <0>
[ 1210.439414]   TDH                  <2d>
[ 1210.439414]   TDT                  <2e>
[ 1210.439414]   next_to_use          <2e>
[ 1210.439414]   next_to_clean        <2d>
[ 1210.439414] buffer_info[next_to_clean]
[ 1210.439414]   time_stamp           <1000dcd20>
[ 1210.439414]   next_to_watch        <ffff88007d81b2d0>
[ 1210.439414]   jiffies              <1000de574>
[ 1210.439414]   desc.status          <300000>
[ 1212.438966] igb 0000:00:14.0: Detected Tx Unit Hang
[ 1212.438966]   Tx Queue             <0>
[ 1212.438966]   TDH                  <2d>
[ 1212.438966]   TDT                  <2e>
[ 1212.438966]   next_to_use          <2e>
[ 1212.438966]   next_to_clean        <2d>
[ 1212.438966] buffer_info[next_to_clean]
[ 1212.438966]   time_stamp           <1000dcd20>
[ 1212.438966]   next_to_watch        <ffff88007d81b2d0>
[ 1212.438966]   jiffies              <1000ded44>
[ 1212.438966]   desc.status          <300000>
[ 1212.490329] igb 0000:00:14.0 eth0: Reset adapter

Rebooting the switch again stops the behavior.

RN-404 (CM-4407)
Aggregating routes in BGP with as-set can result in high CPU usage

When BGP is configured with aggregate addresses with as-set configuration and there are many routes to be aggregated, the BGP process gets into high CPU usage.

To work around this issue, do not specify the as-set parameter for the aggregate-address configuration.

RN-406 (CM-9895)
Mellanox SN2700 power off issues

On the Mellanox SN2700 and SN2700B switches, if any of the following occur:

  • A shutdown or poweroff command is executed
  • A temperature sensor hits a critical value and shuts down the box

Once a PDU power cycle is issued, the box appears to be dead for at least 3 minutes.

RN-409 (CM-10054)
BGP may show an inaccessible path as the best path

Existing BGP issues caused peering between a VRF device and a loopback BGP session to stay up if the loopback session doesn’t advertise its local address.

This issue will be fixed in a future release.

RN-446 (CM-10513)
Redistribute neighbor does not work with more than 1024 interfaces

The rdnbrd service crashes because it cannot work with more than 1024 interfaces.

This issue should be fixed in a future release of Cumulus Linux.

RN-448 (CM-11302)
Using the json option in the "show ip bgp" command causes peer session flaps

This issue causes peer session flaps on Penguin Arctica 4806XP and Supermicro SSE-X3648S switches. It occurs with 16K IPv4 prefixes and only when you run show ip bgp json.

However, on switches with Tomahawk ASICs, with 61K IPv4 prefixes and default timers, the same show ip bgp json command causes all peer sessions to go down.

This is a known issue that should be fixed in a future release of Cumulus Linux.

RN-450 (CM-12252)
802.1p remark in traffic.conf behaves differently on Mellanox vs. Broadcom switches

The 802.1p remark defined in traffic.conf acts differently on a Mellanox switch when compared to a Broadcom switch.

On the Mellanox platform, the remark defined in the traffic.conf file takes precedence even if there is an ACL rule that is matched.

On the Broadcom platform, the ACL rule takes precedence over the remark defined in the traffic.conf file.

RN-451 (CM-12344)
Mellanox switch rejects SPAN ACL rule for an output interface that is a subinterface

This is a known issue at this time.

RN-455 (CM-12578)
The cumulus-poe package is not installed on ARM switches after upgrading to version 3.1

After upgrading Cumulus Linux from a binary image installation of version 3.1 or earlier, the cumulus-poe package does not get installed on ARM-based switches. THe following message appears after you reboot the switch:

[ OK ] Stopped Cumulus Linux POE Daemon.
Starting Cumulus Linux POE Daemon...
[FAILED] Failed to start Cumulus Linux POE Daemon.

In order to use Power over Ethernet (PoE) on an ARM switch, you need to install the cumulus-poe package:

cumulus@switch:~$ sudo apt-get update
cumulus@switch:~$ sudo apt-get install cumulus-poe
cumulus@switch:~$ sudo apt-get upgrade

RN-525 (CM-12715)
On a Quanta IX1 switch, 100G OSI LR4 module (QSFP28-LR4-OSI) doesn't advertise 40G in transceiver codes, so cannot be used at 40G speed

Cumulus Linux can only support speeds on modules that are advertised in each module's transceiver codes. You can determine this by running ethtool -m:

cumulus@switch:~$ sudo ethtool -m

    Transceiver codes                       : 0x80 0x00 0x00 0x00 0x00 0x00 0x00 0x00
    Transceiver type                        : 100G Ethernet: 100G Base-LR4


The example above shows that this module supports only 100G; it cannot support 40G speeds. Similarly, if a module advertises 40G support only, it cannot support 100G speeds.

RN-526 (CM-13037)
Upgrading the clag package fails when logrotate contains an invalid date

While upgrading from Cumulus Linux 3.0.1 to 3.1.z, the clag package does not update if the logrotate file contains an invalid date. This can occur due to bad batteries in the switch or RTC clock chip issues. The error may look like the following:

error: bad year 1929 for file /var/log/boot.log in state file /var/lib/logrotate/status
dpkg: error processing package clag (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
E: Sub-process /usr/bin/dpkg returned an error code (1)

You can work around this issue by removing the /var/lib/logrotate/status file, and forcing a logrotate. After you do this, the upgrade should be successful.

RN-537 (CM-12967)
Pause frames sent by a Tomahawk switch are not honored by the upstream switch

An issue exists when link pause or priority flow control (PFC) is enabled on a Broadcom Tomahawk-based switch, and there is over-subscription on a link, where the ASIC sends pause frames aggressively, causing the upstream switch not to throttle enough.

If you need link pause or PFC functionality, then to work around this issue, you must use a switch that does not use the Tomahawk ASIC.

RN-540 (CM-13428)
Quagga reload fills FIB after renaming the table map

Trying to rename a table map — essentially deleting the current table map and adding a new one — causes Quagga to try to install all the routes again, which fills the FIB. The table map begins functioning again on its own after some time and without any intervention, and the FIB usage returns to a normal level.

This issue is currently being investigated.

RN-542 (CM-13461)
Polling the BGP RIB with "show ip bgp" causes the peer to flap if the RIB has more than 600K entries

This is a known issue that's currently being investigated. The Quagga log shows these commands taking a very long to execute.

To work around this issue, Cumulus Networks recommends you use larger keepalive/hold timers — 60 and 180 seconds, respectively.

RN-545 (CM-13800)
OSPFv3 redistribute connected with route-map broken at reboot (or ospf6d start) This issue only affects OSPFv3 (IPv6) and is being investigated at this time.

RN-548 (CM-14061)
ethtool --show-fec does not reflect correct state The ethtool --show-fec command does not accurately report the current link state. The cause of this issue is being investigated.

RN-549 (CM-14106)
For a 100G capable port and cable, enabling auto-negotiation fails when the port speed is set to 40G in ports.conf An issue is present on 100G capable ports and cables, where auto-negotiation fails when set to 40G in the /etc/cumulus/ports.conf file. This issue is currently being investigated.

RN-552 (CM-14549)
After setting interface speed to 40G in ports.conf on a Mellanox switch, ethtool still shows interface as 100G

This is a known issue whereby ethtool does not update after restarting switchd, so it continues to display the outdated port speed.

To correctly set the port speed, use NCLU or ethtool to set the speed instead of hand editing the ports.conf file.

For example, to set the speed to 40G using NCLU:

cumulus@switch:~$ net add interface swp1 link speed 40000 

Or using ethtool:

cumulus@switch:~$ sudo ethtool -s swp1 speed 40000

RN-595 (CM-15934)
Egress ACL statistics not accurate on Broadcom-based switches

The counters for egress ACLs may not accurately represent the amount of packets matching the rule.

This issue is being investigated.

RN-597 (CM-15705)
sFlow doesn't generate flow samples to sflowd on Tomahawk-based switches At this time, sFlow is not supported on switches with Tomahawk ASICs. This is a known issue. 

RN-598 (CM-15575)
CLAGD process restarts when updating backup-ip

An error was found when an accidental change was made to the backup IP, and then corrected. ifreload -a would restart the clagd process to invoke the daemon with the new backup IP, rather than updating the backup IP with the change.

This issue is being investigated.

RN-599 (CM-15949)
DHCRELAY automatically binds to eth0 when not specified in the configuration dhcrelay listens for all interfaces that have an IP, even if not configured to listen for that interface. This causes dhcrelay to bind to unspecified ports.

This behavior is expected, due to upstream configuration. The packet is dropped later in the process, as it is not coming from a configured port.

RN-603 (CM-15739)
Broadcom Tomahawk ASIC doesn't support a mix of 10G and 25G speeds within the same warp core

On a switch with a Broadcom Tomahawk ASIC, you cannot mix 10G and 25G speeds within the same warp core (a warp core is a cluster of 4 sequential switch ports, such as swp1 through swp4 or swp29 through swp32).

On a 100G switch, each of the 4 physical ports in the core can be broken out into 4x10G or each of the 4 ports can be broken out into 4x25G speeds; you cannot have a mix of speeds.

On a 25G switch, each port in the warp core must be configured for the same speed. You can configure all ports in the warp core for 10G speed, but you cannot have a mix of 25G and 10G speeds.

These restrictions apply to SFP modules; for QSPF modules, the restriction applies if you want to break out one of the ports to 4x10G or 4x25G.

If you want to modify the speed of the ports in a warp core, you must hand edit the /etc/cumulus/ports.conf file. You cannot use NCLU to configure the port speeds on a Tomahawk switch.

This is a known issue and is being investigated.

RN-604 (CM-15959)
ARP suppression does not work well with VxLAN A-A

In some instances, ARP requests do not get suppressed (when they ought to be) in a VxLAN A-A scenario, but instead get flooded over VxLAN tunnels. This issue is caused because there is no "control plane" syncing the snooped local neighbor entries between the CLAG pair; CLAG does not perform this sync, and neither does EVPN.

This issue is being investigated.

RN-605 (CM-15515)
Unable to change the bond-modes using ifup or ifreload When the bond mode is changed from 802.3ad to balance-xor or vice versa using ifup bondx or ifreload -a, the bond-mode does not change, and the following error is produced:
2017-03-23 21:39:37,495:  DEBUG:      /
autolib.netobjects: [cumulus@] sudo: ('ifup bond1',) 2017-03-23 21:39:37,926: DEBUG: /
autolib.netobjects: warning: error writing to file /sys/class/net/bond1/bonding/mode([Errno 39] Directory not empty)

This issue is being addressed in a later release.

RN-606 (CM-6366)
BGP: MD5 password is not enforced for dynamic neighbors

It was determined that the MD5 password configured against a BGP listen-range peer-group (used to accept and create dynamic BGP neighbors) is not enforced. This means that connections are accepted from peers that don't specify a password; and only if they don't.

This issue is being investigated.

RN-608 (CM-16145)
Buffer monitoring default port group discards_pg only accepts packet collection type

The default port group discards_pg does not accept packet_extended or packet_all collection types.

This issue is currently being investigated.

RN-641 (CM-16905)
sudo dumps core in ARM platforms for TACACS+ user

This isn't an issue on x86 platforms. It occurs because the ARM system opens a library that is already opened.

This issue will be fixed in the next release of Cumulus Linux.

RN-642 (CM-17107)
No reply to SNMP request (silently dropped) if the request is received on multiple interfaces

When an SNMP server has multiple network paths to reach the switch, it is expected behavior that the SNMP request would arrive at multiple interfaces. However, it has been seen that Cumulus Linux silently drops any SNMP requests that arrive at an interface that does not match the return RIB lookup.

This issue occurs only if asymmetric routing is configured, where an SNMP request is received on multiple interfaces of the polled object.

To work around this issue, avoid asymmetric routing to the interfaces in question and configure a static route to pin the traffic to one link/bundle.


RN-643 (CM-17473)
The dhcpd process crashes when an interface request performs a DHCP request

In certain conditions, if an interface attempts to send a DHCP request packet before the kernel port structures are fully initialized, the dhcpd process crashes.

To work around this issue, remove the -nl option from the dhcpd.service.

This issue is being investigated.

RN-653 (CM-17856) 
Enabling PFC on Mellanox switches may cause switchd to crash

On Cumulus Linux versions 3.3.0 and later, enabling priority flow control (PFC) on Mellanox Spectrum-based switches may cause the switchd process to crash.

To work around this issue, populate the unlimited_egress_buffer_port_set parameter in the /etc/cumulus/datapath/traffic.conf file. The default range should be "swp<a>-swp<z>", where "swp<a>" is the first front panel port in /var/lib/cumulus/porttab and "swp<z>" is the last front panel port in the porttab file. For example:

# priority flow control
pfc.port_group_list = [pfc_port_group]
pfc.pfc_port_group.cos_list = [0]
pfc.pfc_port_group.port_set = swp1-swp5
pfc.pfc_port_group.port_buffer_bytes = 25000
pfc.pfc_port_group.xoff_size = 10000
pfc.pfc_port_group.xon_delta = 2000
pfc.pfc_port_group.tx_enable = true
pfc.pfc_port_group.rx_enable = true
pfc.pfc_port_group.unlimited_egress_buffer_port_set = swp1-swp16

RN-654 (CM-17379, CM-17837)
Stale state in switchd results in crash after config change or prevents MACs from being learned

Issues with switchd arose when a VXLAN-enabled bridge interface or VLAN was modified but some remnant of the original state remained. Such changes included:

  • Removing a VXLAN from a bridge
  • Destroying a VXLAN
  • Disabling a bridge with VXLANs associated with it

Subsequent configuration changes to the bridge, VXLAN or bridge member ports could cause errors due to the stale state.

This issue manifest itself in two ways. First, it could cause switchd to dump core after ifreload -a was run. Second, MAC addresses might not be learned because the bond forward state was disabled in switchd, even though it was enabled in the kernel.

This issue is currently being investigated.

RN-658 (CM-17338) 
Power cycling a connected host may result in control plane traffic failure on a 10G BASE-T Trident II+ switch

Switches with the Trident II+ chipset running Cumulus Linux 3.3.0 or later may experience a failure to transmit frames from the control plane following a power-cycle of a device connected via 10GBASE-T. This can result in complete loss of connectivity from the switch control plane to connected devices.

To work around this issue, restart switchd with sudo systemctl restart switchd. A fix is currently under investigation.

RN-674 (CM-17577)
Cannot set the MTU for switch ports that is different than the MTU for eth0

You cannot set both a global MTU and an individual MTU in a policy file. For example, this configuration does not work:

root@leaf01:/home/cumulus# cat /etc/network/ifupdown2/policy.d/mtu.json
 "address": {"defaults": { "mtu": "9216" }},
 "ethtool": {"iface_defaults": {"eth0": {"mtu": "1500"}}}

There is no workaround at this time. This issue will be fixed in an upcoming release of Cumulus Linux.

RN-676 (CM-17302) 
MLAG with LACP bypass breaks when used with iPXE on servers

Given the following conditions:

  • LACP bypass and MLAG are enabled on a pair of leaf switches
  • Servers are dual-attached to the MLAG switch pair
  • iPXE is installed on the servers and configures all available interfaces as LACP-passive members

When PXE attempts to chain-load iPXE on a dual-attached server, iPXE runs LACP on one of the links, bringing it out of LACP bypass mode and making it a fully operational bond. But since the other link was still in LACP bypass mode, the clagd service doesn't allow the bond to transition out of a DORMANT state. iPXE, meanwhile, only uses the bond, which didn't work because it was still in a DORMANT state.

This issue is fixed in Cumulus Linux 3.4.1 — the bond now transitions from DORMANT to UP even when the corresponding bond on the peer switch is still in LACP bypass.


This support portal has moved

Cumulus Networks is now part of the NVIDIA Networking Business Unit! The NVIDIA Cumulus Global Support Services (GSS) team has merged its operations with the NVIDIA Mellanox support services team.

You can access NVIDIA Cumulus support content from the Mellanox support portal.

You open and update new cases on the Mellanox support portal. Any previous cases that have been closed have been migrated to the Mellanox support portal.

Cases that are still open on the Cumulus portal will continue to be managed on the Cumulus portal. Once these cases close, they will be moved to the Mellanox support portal.

Powered by Zendesk