Cumulus VX 2.5.5 Release Notes

Follow

Overview

Cumulus VX is a community-supported virtual appliance that enables cloud administrators and network engineers to preview and test Cumulus Networks technology at zero cost. You can build sandbox environments to learn open networking concepts, prototype network operations and script & develop applications risk-free. With Cumulus VX, you can get started with open networking at your pace, on your time, and in your environment.

These release notes support Cumulus VX 2.5.5 and describe its features and known issues.

Stay up to date: Click Follow above so you can receive a notification when we update these release notes.

{{table_of_contents}}

Cumulus VX Features

  • Integrates with KVM, VirtualBox and VMware hypervisors
  • Runs within GNS3 and Vagrant environments
  • VM is a 64-bit operating system based on Cumulus Linux 2.5.5

Downloading Cumulus VX

You can download any of the of the four Cumulus VX images: 

  • An OVA disk image for use with VirtualBox.
  • A VMware-specific OVA disk image.
  • A qcow2 disk image for use with KVM.
  • A Box image for use with Vagrant.

Configuration Notes

Keep in mind the following issues when you are running your Cumulus VX virtual machine.

 SNMP Not Supported in Quagga

There is no SNMP support for Quagga in Cumulus VX (see RN 88 below). Due to this circumstance, you must remove all references to smux in each of the following configuration files. If the smux entries are present in the configuration files, the daemons in the 2.5 packaged version of Quagga will not start.

  1. cd /etc/quagga
  2. grep smux *
  3. Delete all lines in the config files containing the smux keyword.

The references to smux that must be removed are:

  • In bgpd.conf, remove this line:
    smux peer 1.3.6.1.4.1.3317.1.2.2 quagga_bgpd
  • In ospf6d.conf, remove this line:
    smux peer 1.3.6.1.4.1.3317.1.2.6 quagga_ospf6d
  • In ospfd.conf, remove this line:
    smux peer 1.3.6.1.4.1.3317.1.2.5 quagga_ospfd
  • In zebra.conf, remove this line:
    smux peer 1.3.6.1.4.1.3317.1.2.1 quagga_zebra

 Perl, Python and BDB Modules

Any Perl scripts that use the DB_File module or Python scripts that use the bsddb module won't run under Cumulus VX.

What's New in Cumulus VX 2.5.5

Cumulus VX 2.5.5 supports these new features:

  • netshow has been moved to the main repository in Cumulus Linux
  • Stability enhancements to BGP, MLAG and VXLAN

Documentation

You can read the technical documentation here.

Community Support

If you have any questions or feedback about Cumulus VX, visit the Cumulus VX community for further support. 

Issues Fixed in Cumulus VX 2.5.5

The following is a list of issues fixed in Cumulus VX 2.5.5 from earlier versions of Cumulus VX.

Release Note ID Summary Description

RN-319 (CM-7756)
MSTP configuration not removed from interfaces file after issuing ifreload -a

If the MSTP configuration for VLAN-aware bridge ports or for a traditional mode bridge is set to yes (MSTP configuration options include mstpctl-portbpdufilter and mstpctl-bpduguard), then this configuration is removed from /etc/network/interfaces and an ifreload -a command is issued, the port configuration is not set to the default no even though the setting was removed from the configuration file.

The root cause of this problem is that ifreload does not check the default settings for the MSTP configuration for the bridge ports and use a default when that setting is removed from the configuration. Future versions of ifreload2 may check the MSTP default settings and take action based on these defaults.

To work around this issue, instead of removing the configuration for MSTP for the bridge port or bridge, if the configuration is explicitly
set to no, and an ifreload -a is performed, the MSTP setting is correctly set.

Another workaround is to avoid using ifreload -a and explicitly running if down and then if up on the 
bridge port or bridge after the MSTP configuration for the ports is removed.


RN-321 (CM-7600)
BFD session registration fails when transitioning from one type of BFD session to another type; transitioning from single hop to multihop or multihop to single hop by deregistering and registering fails

The problem occurs only when BFD session updates (from changing BFD parameters) were done before deregistering the sessions. This is because PTM was creating new parameter configurations in the parameter hash list instead of just updating old parameters every time the client sends BFD parameter updates. So, even though BFD session is deleted on client request, multiple parameter configurations can exist for the same peer in the hash list. When a new client tries to add the same BFD peer with a different type, the type is compared against the existing parameter configuration and fails on type mismatch.

To work around this issue, when parameter updates are received, delete the old parameter configuration from the hash list and re-add the new one.


RN-323 (CM-7578)
Under rare circumstances, BGP may have redistributed routes present in its RIB even though its policy configuration disallows the route

If BGP is redistributing routes from multiple sources — static and OSPF in this case — and it has a policy configuration to allow the route to a destination only from one source — where, for example, prefix X, static is allowed but OSPF is not — it may continue to retain the route to the destination even if the best route changed from static to OSPF. 

The reason for this is because when BGP realizes that the best route to X has changed to OSPF and policy doesn't allow it, it does not purge the redistributed route to X that it already has.

This issue only occurs in the specified scenario, which is not common. You an avoid this issue by ensuring that the policy for redistributed routes is consistent for a particular destination, irrespective of the source.


RN-330 (CM-7176)
IPv6 BGP peering using global IPv6 address may not come up before the BGP connect retry times out when Quagga starts

When Quagga comes up and attempts to establish an IPv6 BGP peering (specifically eBGP) using a global IPv6 address, the initial connection attempt may happen before IPv6 Duplicate Address Detection (DAD) has completed on the IPv6 address assigned to the interface. In such a case, the TCP connection (SYN) is initiated with some other source IPv6 address of the local system, which the peer is likely to drop. The connection attempt fails after TCP SYN timeout and a new attempt would be initiated after BGP connect retry timeout.

You can significantly reduce this delay in establishing the initial BGP peering by using a low BGP connect retry timer, such as 1 second. Or you can completely avoid the issue by disabling DAD on the interface or configuring for optimistic DAD by, for example, using a pre-up directive against the corresponding interface in /etc/network/interfaces. For example, to configure for optimistic DAD, set sysctl optimistic_dad to 1.

iface swp1s0
    address 10.1.1.1/30
    address 2001:10:1::1/64
    down ip addr flush dev swp1s0
    pre-up echo 0 > /proc/sys/net/ipv6/conf/swp1s0/accept_dad

RN-339 (CE-1190)
Puppet and Chef continuously reapply their configurations

Due to a change in the interface object returned by ifquery, the interfaces configuration modules for Puppet and Chef are non-idempotent. This may result in the interfaces configuration being repeatedly reloaded. Updates to the orchestration modules that correct this behavior have been pushed to the corresponding upstream repositories (Puppet Forge, Chef Supermarket).

The following versions are affected:

  • Cumulus Linux/VX 2.5.4 python-ifupdown2 version python-ifupdown2:all=0.1-cl2.5+6
  • Puppet cumulus_interfaces module version 1.2.0
  • Chef cumulus_interface definition version 1.2.0

Fixes are available via updates to the following orchestration modules:

Known Issues in Cumulus VX 2.5.5

Issues are categorized for easy review. Some issues are fixed but will be available in a later release.

Release Note ID Summary Description

RN-4 (FR-53)
ifup/ifdown must be used for interfaces with IPv6 addresses defined in /etc/network/interfaces, otherwise the global IPv6 address will not be restored Two scenarios are shown below; one with ifup/ifdown, the other with ifconfig down.

With ifup/ifdown:
swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81
 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0
 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link
 inet6 addr: fec0:1000:1000:1000::2/10 Scope:Site
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:4231 errors:0 dropped:0 overruns:0 frame:0
 TX packets:4342 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500
 RX bytes:412115 (402.4 KiB) TX bytes:425688 (415.7 KiB)

cumulus@switch$ sudo ifdown swp1
cumulus@switch$ sudo ifconfig swp1
 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81
 BROADCAST MULTICAST MTU:1500 Metric:1
 RX packets:4248 errors:0 dropped:0 overruns:0 frame:0
 TX packets:4356 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500
 RX bytes:413990 (404.2 KiB) TX bytes:427074 (417.0 KiB)

cumulus@switch$ sudo ifconfig swp1
 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81
 BROADCAST MULTICAST MTU:1500 Metric:1
 RX packets:4248 errors:0 dropped:0 overruns:0 frame:0
 TX packets:4356 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500
 RX bytes:413990 (404.2 KiB) TX bytes:427074 (417.0 KiB)
 cumulus@switch$ sudo ifup swp1
 ADDRCONF(NETDEV_UP): swp1: link is not ready
 cumulus@switch$ sudo ifconfig swp1ADDRCONF(NETDEV_CHANGE): swp1: /
     link becomes ready 
 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 
 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0
 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link
 inet6 addr: fec0:1000:1000:1000::2/10 Scope:Site
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:4250 errors:0 dropped:0 overruns:0 frame:0
 TX packets:4362 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500
 RX bytes:414178 (404.4 KiB) TX bytes:427610 (417.5 KiB)

cumulus@switch$ 

With ifconfig down:
cumulus@switch:~$ sudo ifconfig swp1
 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81
 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0
 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link 
 inet6 addr: fec0:1000:1000:1000::2/10 Scope:Site
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:98 errors:0 dropped:0 overruns:0 frame:0
 TX packets:111 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500 
 RX bytes:13310 (12.9 KiB) TX bytes:12786 (12.4 KiB)

cumulus@switch$ sudo ifconfig swp1 down
cumulus@switch$ sudo ifconfig swp1
 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81
 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0
 BROADCAST MULTICAST MTU:1500 Metric:1
 RX packets:126 errors:0 dropped:0 overruns:0 frame:0
 TX packets:138 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500
 RX bytes:16998 (16.5 KiB) TX bytes:15998 (15.6 KiB)

cumulus@switch$ sudo ifconfig swp1 up
 ADDRCONF(NETDEV_UP): swp1: link is not ready

cumulus@switch$ sudo ifconfig swp1ADDRCONF(NETDEV_CHANGE): swp1: 
    link becomes ready 
 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 
 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0
 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 
 RX packets:130 errors:0 dropped:0 overruns:0 frame:0
 TX packets:149 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500
 RX bytes:17474 (17.0 KiB) TX bytes:17154 (16.7 KiB) 

RN-52 (CM-997,
CM-1013)
Parameters like the router ID and DR priority cannot be changed while OSPFv2/v3 is running Router ID and DR priority can only be changed by shutting down OSPFv2/v3, changing the ID, and restarting the OSPF process.

A change to the DR priority may not properly be reflected in the LSAs that are still aging out.

RN-56 (CM-343)
IPv4/IPv6 forwarding disabled mode not recognized

If either of the following is configured:

net.ipv4.ip_forward == 0 

or:

net.ipv6.conf.all.forwarding == 0 

The hardware still forwards packets if there is a neighbor table entry pointing to the destination.


RN-58 (CM-747)
IPv6 route is installed and active in the routing table when the associated interface is down If an IPv6 address is assigned to a "down" interface, the associated route is still installed into the route table.

Also, the type of IPv6 address doesn't matter. Link local, site local, and global all exhibit the same problem.

If the interface is bounced up and down, then the routes are no longer in the route table.

RN-61 (CM-1004)
BGP4 notifications missing for several conditions In certain conditions, Quagga's bgpd service silently closes the peering without sending a notification. For example, if BGP receives a message with an invalid message type or invalid message length.

Ideally on any one of these cases, bgpd should send out a notification message to the peer.

General functionality of BGP4 is not affected.

RN-64 (CM-1153)
Configuring route-reflector-client requires specific order In configuring a route to be a route reflector client, the Quagga configuration must be specified in a specific order; otherwise, the router will not be a route reflector client.

The "neighbor <IPv4/IPV6> route-reflector-client" command must be done after the "neighbor <IPV4/IPV6> Activate" command; otherwise, the route-reflector-client command is ignored.

Sample configuration:
router bgp 65000
 bgp router-id 0.0.0.4 
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 bgp cluster-id 0.0.0.4 
 bgp bestpath as-path multipath-relax 
 redistribute connected 
 neighbor 14.0.0.1 remote-as 65000 
 neighbor 14.0.0.1 route-reflector-client 
 neighbor 14.0.0.1 activate 
 neighbor 14.0.0.1 next-hop-self 
 neighbor 14.0.0.9 remote-as 65000 
 neighbor 14.0.0.9 activate 
 neighbor 14.0.0.9 next-hop-self 
 neighbor 2001:ded:beef::1 remote-as 65000 
 neighbor 2001:ded:beef:2::1 remote-as 65000 
 maximum-paths 4 
 maximum-paths ibgp 4 
 ! 
 address-family ipv6 
 redistribute connected 
 neighbor 2001:ded:beef::1 activate 
 neighbor 2001:ded:beef::1 next-hop-self 
 neighbor 2001:ded:beef:2::1 route-reflector-client 
 neighbor 2001:ded:beef:2::1 activate 
 neighbor 2001:ded:beef:2::1 next-hop-self 
 maximum-paths 4 
 maximum-paths ibgp 4 
 exit-address-family 

At runtime:
cumulus@switch:$ show ip bgp neighbor 14.0.0.1 
 BGP neighbor is 14.0.0.1, remote AS 65000, local AS 65000, internal link
 BGP version 4, remote router ID 0.0.0.6 
 BGP state = Established, up for 00:23:49
 Last read 23:31:36, hold time is 180, keepalive interval is 60 seconds 
 Neighbor capabilities: 
 4 Byte AS: advertised and received 
 Route refresh: advertised and received(old & new)
 Address family IPv4 Unicast: advertised and received 
 Message statistics: 
 Inq depth is 0 
 Outq depth is 0
 Sent Rcvd 
 Opens: 2 0
 Notifications: 0 0
 Updates: 1 1
 Keepalives: 25 24 
 Route Refresh: 0 0
 Capability: 0 0 
 Total: 28 25 
 Minimum time between advertisement runs is 5 seconds
 For address family: IPv4 Unicast 
 >>>>>>>>>>>>>>>>>>>>>> ROUTE REFLECTOR CLIENT NOT DISPLAYED 
 NEXT_HOP is always this router 
 Community attribute sent to this neighbor(both) 
 6 accepted prefixes 
 Connections established 1; dropped 0 
 Last reset never 
 Local host: 14.0.0.2, Local port: 179 
 Foreign host: 14.0.0.1, Foreign port: 40290 
 Nexthop: 14.0.0.2 
 Nexthop global: 2001:ded:beef::2 
 Nexthop local: fe80::202:ff:fe00:4
 BGP connection: non shared network
 Read thread: on Write thread: off 
 cumulus@switch:$ 

Workaround:
Define in following order 
 address-family ipv4 unicast
 neighbor 14.0.0.9 activate 
 neighbor 14.0.0.9 next-hop-self
 neighbor 14.0.0.9 route-reflector-client >>> Must be after Activate 
 exit-address-family 
 neighbor 2001:ded:beef:2::1 remote-as 65000
 address-family ipv6 unicast 
 redistribute connected
 maximum-paths 4 
 maximum-paths ibgp 4 
 neighbor 2001:ded:beef:2::1 activate 
 neighbor 2001:ded:beef:2::1 next-hop-self 
 neighbor 2001:ded:beef:2::1 route-reflector-client >>> Must be after activate 
 exit-address-family 
 Runtime status after change: 

cumulus@switch:$ show ip bgp neighbors 14.0.0.9 
 BGP neighbor is 14.0.0.9, remote AS 65000, local AS 65000, internal link 
 BGP version 4, remote router ID 0.0.0.7 
 BGP state = Established, up for 00:13:59
 Last read 22:35:13, hold time is 180, keepalive interval is 60 seconds 
 Neighbor capabilities: 
 4 Byte AS: advertised and received 
 Route refresh: advertised and received(old & new) 
 Address family IPv4 Unicast: advertised and received 
 Message statistics: 
 Inq depth is 0 
 Outq depth is 0
 Sent Rcvd 
 Opens: 1 1
 Notifications: 0 0 
 Updates: 2 1 
 Keepalives: 15 14
 Route Refresh: 0 0
 Capability: 0 0 
 Total: 18 16 
 Minimum time between advertisement runs is 5 seconds
 For address family: IPv4 Unicast 
 Route-Reflector Client >>>>>>>>>> PLEASE NOTE ME 
 NEXT_HOP is always this router 
 Community attribute sent to this neighbor(both) 
 6 accepted prefixes 
 Connections established 1; dropped 0 
 Last reset never 
 Local host: 14.0.0.10, Local port: 38813 
 Foreign host: 14.0.0.9, Foreign port: 179
 Nexthop: 14.0.0.10 
 Nexthop global: 2001:ded:beef:2::2 
 Nexthop local: fe80::202:ff:fe00:6 
 BGP connection: non shared network 
 Read thread: on Write thread: off 
 cumulus@switch:$

RN-70 (CM-1166)
ACL: Bridge traffic that matches a LOG ACTION rule is not logged in syslog For example, a bridge with switch ports swp1, swp2, swp3 as bridge members is configured. ACL rules to LOG and DROP for icmp traffic are configured.

Ping requests are sent from host1 on swp1 to host3 on swp3, and the following was observed:
* Counters for both LOG and DROP ACL rules are incrementing properly, but the packets are not showing up on /var/log/syslog.
* Packets that are copied to the CPU from hardware for the LOG rule are dropped due to the check in kernel to disable software bridging for hardware bridged packets.

RN-88 (CM-1200)
SNMP support for Quagga is NOT provided in Cumulus VX

Cumulus VX does not provide SNMP support for Quagga.

You can get this information via Nagios

However, it's possible to get it via SNMP by:

  • Writing a pass persist script in Perl or Python by filling in the OSPF or BGP (rfc) MIBs manually
  • Creating your own private MIB for the information you need.

RN-121 (CM-2123)
PTMD: When a physical interface is in a PTM FAIL state, its subinterface still exchanges information Issue:
When PTMD is incorrectly in a failure state and the Zebra interface is enabled, PIF BGP sessions are not establishing the route, but the subinterface on top of it does establish routes.

If the subinterface is configured on the physical interface and the physical interface is incorrectly marked as being in a PTM FAIL state, routes on the physical interface are not processed in Quagga, but the subinterface is working.

Steps to reproduce:
cumulus@switch:$ sudo vtysh -c 'show int swp8' 
Interface swp8 is up, line protocol is up 
PTM status: fail
index 10 metric 1 mtu 1500 
 flags: <UP,BROADCAST,RUNNING,MULTICAST>
 HWaddr: 44:38:39:00:03:88 
 inet 12.0.0.225/30 broadcast 12.0.0.227 
 inet6 2001:cafe:0:38::1/64 
 inet6 fe80::4638:39ff:fe00:388/64 
cumulus@switch:$ ip addr show | grep swp8 
 10: swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc pfifo_fast state UP qlen 500 
  inet 12.0.0.225/30 brd 12.0.0.227 scope global swp8 
 104: swp8.2049@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet 12.0.0.229/30 brd 12.0.0.231 scope global swp8.2049 
 105: swp8.2050@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet 12.0.0.233/30 brd 12.0.0.235 scope global swp8.2050 
 106: swp8.2051@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet 12.0.0.237/30 brd 12.0.0.239 scope global swp8.2051 
 107: swp8.2052@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet 12.0.0.241/30 brd 12.0.0.243 scope global swp8.2052 
 108: swp8.2053@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP>
  mtu 1500 qdisc noqueue state UP 
  inet 12.0.0.245/30 brd 12.0.0.247 scope global swp8.2053 
 109: swp8.2054@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP> 
  mtu 1500 qdisc noqueue state UP 
  inet 12.0.0.249/30 brd 12.0.0.251 scope global swp8.2054
 110: swp8.2055@swp8: <BROADCAST,MULTICAST,UP,LOWER_UP>
  mtu 1500 qdisc noqueue state UP 
  inet 12.0.0.253/30 brd 12.0.0.255 scope global swp8.2055
cumulus@switch:$ bgp sessions: 
 12.0.0.226 ,4 ,64057 , 958 , 1036 , 0 , 0 , 0 ,15:55:42, 0, 10472 
 12.0.0.230 ,4 ,64058 , 958 , 1016 , 0 , 0 , 0 ,15:55:46, 187, 10285
 12.0.0.234 ,4 ,64059 , 958 , 1049 , 0 , 0 , 0 ,15:55:40, 187, 10285 
 12.0.0.238 ,4 ,64060 , 958 , 1039 , 0 , 0 , 0 ,15:55:45, 187, 10285 
 12.0.0.242 ,4 ,64061 , 958 , 1014 , 0 , 0 , 0 ,15:55:46, 187, 10285 
 12.0.0.246 ,4 ,64062 , 958 , 1016 , 0 , 0 , 0 ,15:55:46, 187, 10285 
 12.0.0.250 ,4 ,64063 , 958 , 1029 , 0 , 0 , 0 ,15:55:43, 187, 10285 
 12.0.0.254 ,4 ,64064 , 958 , 1036 , 0 , 0 , 0 ,15:55:44, 187, 10285 

RN-125 (CM-1576)
Network LSA with an old router ID isn't flushed out by the originator
When the router ID is changed, the router should remove the previous network LSA (link-state advertisement) that it generated based on the IP address on the interface in the Network LSA.

Cumulus Networks doesn't remove this LSA, so it will be naturally aged out.

RN-132 (CM-2272)
You must run "apt-get update" before running any apt-get commands or after changing sources.list

Before running any apt-get commands or after changing the source.list file in /etc/apt, you need to run apt-get update.


RN-133 (CM-2273)
Interface names in Cumulus VX cannot exceed 15 characters

Device names, including interface names, in Cumulus VX cannot exceed 16 characters – including the terminator. Cumulus VX truncates longer interface names.

To avoid this issue, do not assign long names to your interfaces.

The following example configuration reproduces this issue:

cumulus@switch:/sys/class/net$ grep 'iface br' /etc/network/interfaces 
iface br2-pubmgmt inet static
iface br3-prvmgmt inet manual
iface br400-quarantine inet manual
iface br401-peering-1k5 inet manual
iface br402-peering-9k inet manual
iface br500-pi-exa inet manual
iface br501-akamai-exa inet manual
iface br502-exa-internetfactory inet manual
cumulus@switch:/sys/class/net$ brctl show | grep br
bridge name	bridge id	 STP enabled	interfaces
br2-pubmgmt	 8000.089e01cebe37	no	 bond0.2
br3-prvmgmt	 8000.089e01cebe3a	no	 bond0.3
br400-quarantin	 8000.089e01cebe37	no	 bond0.400
br401-peering-1	 8000.089e01cebe3a	no	 bond0.401 <<<

RN-134 (CM-2303)
Installing Chef under Cumulus VX

The Cumulus VX repository contains two versions of Chef, the automation tool: 11.6.2 (the current version) and 10.30.4.

To install the latest version, connect to the switch and use apt-get:

cumulus@switch:~$ sudo  apt-get install chef

To install 10.30.4, connect to the switch and use apt-get:

cumulus@switch:~$ sudo apt-get install chef=10.30.4-0.debian.7.3 

RN-163 (CM-2499)
VXLAN: ovsdb-server cannot select loopback interface as source IP address, causing TOR registration to the controller to fail

In a VXLAN using VMware NSX, ovsdb-server cannot select the loopback interface as the source IP address. This causes TOR registration to the controller to fail.

To work around this issue, run:

cl-bgp redistribute add connected

RN-192 (CM-3340)

PTM may crash when a large topology file has a syntax error

If there is a syntax error in a large (~ 4000 entries) topology.dot file, PTM may crash while reading/parsing this file. This can occur because the libcgraph API that PTM uses for parsing the file cannot handle the error. You should check topology.dot for syntax errors using a Graphviz tool like dot or dotty that can identify the syntax error.


RN-196 (CM-2499)
For a VXLAN in NSX, ovsdb-server cannot select a loopback interface as the SRC IP

As a result, the TOR registration to the controller fails.

To work around this issue, run:

cl-bgp redistribute add connected

RN-199 (CM-2624)
When a Quagga route-map is modified, the switch could use the partial map before edits are completed

Cumulus VX triggers a route-map update before the user finishes editing the route map, resulting in an incorrect route map being used. The route-map update trigger should only occur when user finishes editing the map.

Cumulus Networks is working to fix this issue.


RN-217 (CM-4207)
LNV: Network restart removes vxsnd anycast IP address from loopback interface

Given the following conditions:

  • You have not configured a loopback anycast IP address in /etc/network/interfaces
  • You enabled the vxsnd (service node daemon) log to automatically add anycast IP addresses

When you restart networking (with service networking restart), the anycast IP address gets removed from the loopback interface.

To prevent this issue from occurring, you should specify an anycast IP address for the loopback interface in both /etc/network/interfaces and vxsnd.conf. This way, in case the vxsnd fails, you can withdraw the IP address.


RN-221 (CM-4501)
BGP graceful restart, including helper mode, not fully supported If you encounter issues with this, please submit a support request and include the output from cl-support with your ticket.

RN-227 (CM-3388)
BGP dynamic capability is not supported BGP peer sessions with dynamic capability are not supported under any version of Cumulus VX at this time.

RN-228 (CM-4384)
ifupdown2 doesn't bring up loopback interface (lo) by default

ifupdown2 doesn't bring up loopback interface (lo) by default.

One effect is that since monit uses loopback interfaces, if you delete the lo interface in /etc/network/interfaces, monit commands will not work.


RN-229 (CM-4433)
When a bond subinterface that is part of a traditional bridge is brought down, it flaps that bridge This issue has been encountered in environments where both VLAN-aware and traditional bridges are in use, where a traditional bridge has a subinterface of a bond that is present as a normal interface in a VLAN-aware bridge.

RN-230 (CM-4327)
bond-min-links defaults to 0, which causes the bond carrier to be incorrectly up when no slaves are active Even though bond-min-links defaults to 0, Cumulus Networks recommends a minimum link setting of 1 or higher for bonds. Cumulus VX issues a warning when the setting is 0 or not specified.

RN-268 (CM-5296)
bridge fdb add command IP address type options inconsistent with Debian

The bridge fdb add command handles the IP address type in a non-standard way:

  • bridge fdb add [MAC address] dev [INTERFACE] master defaults to a static address
  • bridge fdb add [MAC address] dev [INTERFACE] master temp maps to dynamic address
  • bridge fdb add [MAC address] dev [INTERFACE] master local maps to permanent address

This issue should be fixed in a future version of Cumulus VX.


RN-270 (CM-973)
inotify support

inotify is not supported by the overlayfs root filesystem on PowerPC platforms.


RN-273 (CM-6002)
VXLAN BUM: When a next-hop port is elected as the forwarding port, multiple cables with the same next-hop cannot work with full ECMP capacity

As a best practice for when multiple next-hops are going to the same switch, you should configure a bond with an uplink port.


RN-275 (CM-5794)
BGP import-check fails for IPv6 route if static routes to null0 are used

The path that Cumulus VX originates should not be invalid since there is a matching route in the RIB. The import check works fine for IPv4 routes.


RN-279 (CM-6012)
STP not supported on bonds in LACP bypass all-active mode

Cumulus VX does not support STP on bonds in LACP bypass all active mode. To avoid this issue, enable STP BPDU guard for all bonds in LACP bypass all-active mode. For information, read this document.


RN-281 (CM-5118)
Default route not removed on ifup after removing gateway statement from eth0 configuration

If you try to remove the default route from eth0 (either by commenting out or removing the gateway statement in the eth0 configuration), the route remains after running ifup.

To work around this issue, first run ifdown, then ifup on the interface via the console. After this, the route disappears.


RN-317 (CM-5913)
When management VRF is enabled, the MLAG backup IP address does not work over front panel ports

When you enable management VRF, the clagd-backup-ip setting does not work over the switch ports.

To work around this issue, use a backup IP address that is reachable over eth0.


RN-318 (CM-7574)
In VXLAN tunnel processing, for certain traffic patterns when cut-though is enabled and link pause is asserted, you experience line errors like overflow and underflow

With cut-though mode enabled and link pause is asserted, Cumulus VX generates an TOVR and TUFL ERROR; certain error counters increment on given physical port.

cumulus@switch:~$ sudo ethtool -S swp49 | grep Error
HwIfInDot3LengthErrors: 0
HwIfInErrors: 0
HwIfInDot3FrameErrors: 0
SoftInErrors: 0
SoftInFrameErrors: 0
HwIfOutErrors: 35495749
SoftOutErrors: 0

cumulus@switch:~$ sudo ethtool -S swp50 | grep Error
HwIfInDot3LengthErrors: 3038098
HwIfInErrors: 297595762
HwIfInDot3FrameErrors: 293710518

To work around this issue, disable link pause or disable cut-through in /etc/cumulus/datapath/traffic.conf.

To disable link pause, comment out the link_pause* section in /etc/cumulus/datapath/traffic.conf:

cumulus@switch:~$ sudo nano /etc/cumulus/datapath/traffic.conf 
#link_pause.port_group_list = [port_group_0]
#link_pause.port_group_0.port_set = swp45-swp54
#link_pause.port_group_0.rx_enable = true
#link_pause.port_group_0.tx_enable = true

To enable store and forward switching, set cut_through_enable to false in /etc/cumulus/datapath/traffic.conf:

cumulus@switch:~$ sudo nano /etc/cumulus/datapath/traffic.conf 
cut_through_enable = false

RN-322 (CM-7387)
Interfaces disabled using iproute2 become enabled after restarting Quagga By default, all interfaces have a "no shutdown" associated with them in Quagga. Thus, when you restart Quagga, it enables the interfaces. This is expected behavior in Quagga. There is no workaround at this time.

RN-327 (CM-4290)
Changing the route-map parameter of the redistribute command in OSPF and BGP doesn't affect the state of the resulting redistribution in those protocols

To work around this issue, remove any old redistribute command configurations before adding a new one with or without route-map as a parameter.

For example, if OSPF has a redistribute configuration such as redistribute bgp route-map redist-map-name, you would enable redistribution without a route-map by following these steps in OSPF configuration mode:

  1. no redistribute bgp
  2. redistribute bgp

You would perform a similar sequence of commands for redistribution changes in BGP as well.


RN-331 (CM-7711)
Do not use the reboot -f command except in extreme circumstances

When performing a fast reboot (using the reboot -f command), the switch ports remain up and only the control plane goes down.

You should only use this command under abnormal circumstances, because it bypasses normal shutdown.


RN-337 (CM-7623)
Adding IPv6 default route with src address on eth0 fails without adding delay

Attempting to install an IPv6 default route on eth0 with a source address fails at reboot or when running ifup on eth0. 

The first execution of  ifup -dv returns this warning and does not install the route:

cumulus@switch:~$ sudo ifup -dv eth0
warning: eth0: post-up cmd '/sbin/ip route add default via 2001:620:5ca1:160::1 /
src 2001:620:5ca1:160::45 dev eth0' failed (RTNETLINK answers: Invalid argument)<<<<<<<<<<

Running ifup a second time on eth0 successfully installs the route. 

There are two ways you can work around this issue. 

  1. Add a sleep 2 to the eth0 stanza in /etc/network/interfaces:
    iface eth0 inet6 static
        address 2001:620:5ca1:160::45/64
        post-up /bin/sleep 2s
        post-up /sbin/ip route add default via 2001:620:5ca1:160::1 src 2001:620:5ca11
    :160::45 dev eth0
    
  2. Exclude the src parameter to the ip route add that causes the need for the delay. If the src parameter is removed, the route is added correctly.
    iface eth0 inet6 static
        address 2001:620:5ca1:160::45/64
       post-up /sbin/ip route add default via 2001:620:5ca1:160::1 dev eth0
    
    cumulus@switch:~$ sudo ifdown eth0
    Stopping NTP server: ntpd.
    Starting NTP server: ntpd.
    cumulus@switch:~$ sudo ip -6 r s
    cumulus@switch:~$ ifup eth0
    Stopping NTP server: ntpd.
    Starting NTP server: ntpd.
    cumulus@switch:~$ ip -6 r s
    2001:620:5ca1:160::/64 dev eth0  proto kernel  metric 256 
    fe80::/64 dev eth0  proto kernel  metric 256 
    default via 2001:620:5ca1:160::1 dev eth0  metric 1024 
    cumulus@switch:~$ 

RN-338 (CM-7917)
Purging existing addresses for an interface with multiple iface stanzas is not supported

Cumulus VX does not support the purging existing addresses on interfaces with multiple iface stanzas. Doing so can result in the configuration of multiple addresses for an interface after you change an interface address and reload the configuration with ifreload -a. If this happens, you must shut down and restart the interface with ifup and if down, or manually delete superfluous addresses with ip address delete specify.ip.address.here/mask dev DEVICE.

This behavior is documented in the Cumulus Linux user guide.


RN-353 (CM-7937)
VXSND and VXRD incompatibility between Cumulus VX 2.5.4 and 2.5.5

In Cumulus VX 2.5.5, a change in the vxrd registration message resulted in creating an incompatibility such that messages from vxrd in Cumulus VX 2.5.5 are not understood by vxsnd on Cumulus VX 2.5.4. This incompatibility has the following impacts:

  • In Head End Replication mode, vxsnd cannot build and distribute the vxrd database.
  • In Service Node Replication mode, vxsnd must rely on learning about any vxrd by analyzing BUM packets.
  • The reverse combination works fine (the vxsnd is running on Cumulus VX 2.5.5 and the vxrd is running on Cumulus VX 2.5.4).

To work around this issue, ensure you upgrade the vxsnd from version 2.5.4 to 2.5.5 before you upgrade any vxrd.


RN-354 (CM-8397)
dhcprelay does not work with cl-mgmtvrf

DHCP relay does not work with cl-mgmtvrf when the host and server facing ports are in the data plane. The workaround for this issue is to add an IP rule for the DHCP server address, and is documented in this knowledge base article.


RN-355 (CM-7994)
OSPFv2 Area ID being implicitly translated from Integer format to dotted decimal format

While OSPF area ID configuration in Quagga allows for the value to be specified in either dotted decimal format, or as an integer, values specified as an integer will be converted into dotted decimal format when displayed, causing potential confusion for the operator.

This issue does not impact OSPF functionality; only the display output. However, it is recommended that the OSPF area ID is specified in dotted decimal format for consistency.

 

RN-356 (CM-5745)
Quagga reload broken with BGP unnumbered and external ASN keyword

Deleting a line where a neighbor or peer is associated with a peer group, but the neighbor doesn't have a remote-as specified as part of the peer group specification, causes quagga-reload to fail, as the remote-as is unspecified.

To work around this issue, delete the line that says neighbor x.x.x.x remote-as 0, and replace it with either the correct AS, or associate it with a peer group that has the remote-as specified, then restart Quagga.

 

RN-357 (CM-7638)
MGMT-VRF doesn't support DNS lookup in static case

DNS is not handled correctly for static IP addresses by the current implementation of management VRF, as it is built around DHCP use cases. While static IP addresses will generally work, rules need to be set to point the entries in resolv.conf so that DNS lookups occur properly. For this reason, DNS lookup is not supported in 2.5.5.

 

RN-358 (CM-7106)
cl-mgmtvrf: "ip route show" output is inconsistent with "ip route get"

An incorrect result can be returned when using ip route get with management MRF, as it consults the mgmt routing table by default, rather than the main routing table. It is not possible to specify a routing table to look up, or to specify just an iif, or a mark, with ip route get. Instead, to get the desired result, specify either the loopback IP address, or both the loopback IP address and the device, as shown below:

ip route get 10.1.2.2 from 10.0.0.1
 

RN-359 (CM-6219)
isc-dhcp-server service should be off by default

The isc-dhcp-server daemon attempts to start by default after upgrading to Cumulus VX 2.5.5, and rebooting, causing the following error message to display before the login prompt:

[FAIL] startpar: service(s) returned failure: isc-dhcp-server ... failed!

This issue is harmless and has no functionality impact.

Have more questions? Submit a request

Comments

Powered by Zendesk