Cumulus VX 2.5.3 Release Notes

Follow

Overview

Cumulus VX is a community-supported virtual appliance that enables cloud admins and network engineers to preview and test Cumulus Networks technology at zero cost. You can build sandbox environments to learn open networking concepts, prototype network operations and script & develop applications risk-free. With Cumulus VX, you can get started with open networking at your pace, on your time, and in your environment!

These release notes support Cumulus VX 2.5.3 and describe its features and known issues.

Stay up to date: Click Follow above so you can receive a notification when we update these release notes.

{{table_of_contents}}

Cumulus VX Features

  • Integrates with KVM, VirtualBox and VMware hypervisors
  • Runs within GNS3 and Vagrant environments
  • VM is a 64-bit operating system based on Cumulus Linux 2.5.3

Downloading Cumulus VX

You can download any of the of the four Cumulus VX images: 

  • An OVA disk image for use with VirtualBox.
  • A VMware-specific OVA disk image.
  • A qcow2 disk image for use with KVM.
  • A Box image for use with Vagrant.

Configuration Notes

Keep in mind the following issues when you are running your Cumulus VX virtual machine.

Enabling SNMP in Quagga

There is no SNMP support for Quagga in Cumulus VX (see RN 88 below). Due to this circumstance, you must remove all references to smux in each of the following configuration files. If the smux entries are present in the configuration files, the daemons in the 2.5 packaged version of Quagga will not start.

  1. cd /etc/quagga
  2. grep smux *
  3. Delete all lines in the config files containing the smux keyword.

The references to smux that must be removed are:

  • In bgpd.conf, remove this line:
    smux peer 1.3.6.1.4.1.3317.1.2.2 quagga_bgpd
  • In ospf6d.conf, remove this line:
    smux peer 1.3.6.1.4.1.3317.1.2.6 quagga_ospf6d
  • In ospfd.conf, remove this line:
    smux peer 1.3.6.1.4.1.3317.1.2.5 quagga_ospfd
  • In zebra.conf, remove this line:
    smux peer 1.3.6.1.4.1.3317.1.2.1 quagga_zebra

 Perl, Python and BDB Modules

Any Perl scripts that use the DB_File module or Python scripts that use the bsddb module won't run under Cumulus VX.

Documentation

You can read the technical documentation here.

Community Support

If you have any questions or feedback about Cumulus VX, visit the Cumulus VX community for further support. 

Known Issues in Cumulus VX

Issues are categorized for easy review. Some issues are fixed but will be available in a later release.

Release Note ID Summary Description
RN-4 (FR-53) ifup/ifdown must be used for interfaces with IPv6 addresses defined in /etc/network/interfaces, otherwise the global IPv6 address will not be restored Two scenarios are shown below; one with ifup/ifdown, the other with ifconfig down.

With ifup/ifdown:
 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81
 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0
 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link
 inet6 addr: fec0:1000:1000:1000::2/10 Scope:Site
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:4231 errors:0 dropped:0 overruns:0 frame:0
 TX packets:4342 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500
 RX bytes:412115 (402.4 KiB) TX bytes:425688 (415.7 KiB)

cumulus@switch$ sudo ifdown swp1
cumulus@switch$ sudo ifconfig swp1 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:4248 errors:0 dropped:0 overruns:0 frame:0 TX packets:4356 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:413990 (404.2 KiB) TX bytes:427074 (417.0 KiB)
cumulus@switch$ sudo ifconfig swp1 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:4248 errors:0 dropped:0 overruns:0 frame:0 TX packets:4356 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:413990 (404.2 KiB) TX bytes:427074 (417.0 KiB) cumulus@dni-7448-13$ sudo ifup swp1 ADDRCONF(NETDEV_UP): swp1: link is not ready cumulus@switch$ sudo ifconfig swp1ADDRCONF(NETDEV_CHANGE): swp1: /
link becomes ready swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link inet6 addr: fec0:1000:1000:1000::2/10 Scope:Site UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4250 errors:0 dropped:0 overruns:0 frame:0 TX packets:4362 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:414178 (404.4 KiB) TX bytes:427610 (417.5 KiB)
cumulus@switch$

With ifconfig down:
 sudo ifconfig swp1
 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81
 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0
 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link 
 inet6 addr: fec0:1000:1000:1000::2/10 Scope:Site
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:98 errors:0 dropped:0 overruns:0 frame:0
 TX packets:111 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:500 
 RX bytes:13310 (12.9 KiB) TX bytes:12786 (12.4 KiB)

cumulus@switch$ sudo ifconfig swp1 down
cumulus@switch$ sudo ifconfig swp1 swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:126 errors:0 dropped:0 overruns:0 frame:0 TX packets:138 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:16998 (16.5 KiB) TX bytes:15998 (15.6 KiB)
cumulus@switch$ sudo ifconfig swp1 up ADDRCONF(NETDEV_UP): swp1: link is not ready
cumulus@switch$ sudo ifconfig swp1ADDRCONF(NETDEV_CHANGE): swp1: link becomes ready swp1 Link encap:Ethernet HWaddr 44:38:39:00:01:81 inet addr:11.0.0.2 Bcast:11.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::4638:39ff:fe00:181/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:130 errors:0 dropped:0 overruns:0 frame:0 TX packets:149 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:17474 (17.0 KiB) TX bytes:17154 (16.7 KiB)
RN-52 (CM-997, CM-1013) Parameters like the router ID and DR priority cannot be changed while OSPFv2/v3 is running Router ID and DR priority can only be changed by shutting down OSPFv2/v3, changing the ID, and restarting the OSPF process.

A change to the DR priority may not properly be reflected in the LSAs that are still aging out.
RN-56 (CM-343) IPv4/IPv6 forwarding disabled mode not recognized

If either of the following is configured:

net.ipv4.ip_forward == 0 

or:

net.ipv6.conf.all.forwarding == 0 

The hardware still forwards packets if there is a neighbor table entry pointing to the destination.

RN-58 (CM-747) IPv6 route is installed and active in the routing table when the associated interface is down If an IPv6 address is assigned to a "down" interface, the associated route is still installed into the route table.

Also, the type of IPv6 address doesn't matter. Link local, site local, and global all exhibit the same problem.

If the interface is bounced up and down, then the routes are no longer in the route table.
RN-61 (CM-1004) BGP4 notifications missing for several conditions In certain conditions, Quagga bgpd silently closes the peering without sending a notification. For example, if BGP receives a message with an invalid message type or invalid message length.

Ideally on any one of these cases, bgpd should send out a notification message to the peer.

General functionality of BGP4 is not affected.
RN-64 (CM-1153) Configuring route-reflector-client requires specific order In configuring a route to be a route reflector client, the Quagga configuration must be specified in a specific order; otherwise, the router will not be a route reflector client.

The "neighbor <IPv4/IPV6> route-reflector-client" command must be done after the "neighbor <IPV4/IPV6> Activate" command; otherwise, the route-reflector-client command is ignored.

Sample configuration:
 router bgp 65000
 bgp router-id 0.0.0.4 
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 bgp cluster-id 0.0.0.4 
 bgp bestpath as-path multipath-relax 
 redistribute connected 
 neighbor 14.0.0.1 remote-as 65000 
 neighbor 14.0.0.1 route-reflector-client 
 neighbor 14.0.0.1 activate 
 neighbor 14.0.0.1 next-hop-self 
 neighbor 14.0.0.9 remote-as 65000 
 neighbor 14.0.0.9 activate 
 neighbor 14.0.0.9 next-hop-self 
 neighbor 2001:ded:beef::1 remote-as 65000 
 neighbor 2001:ded:beef:2::1 remote-as 65000 
 maximum-paths 4 
 maximum-paths ibgp 4 
 ! 
 address-family ipv6 
 redistribute connected 
 neighbor 2001:ded:beef::1 activate 
 neighbor 2001:ded:beef::1 next-hop-self 
 neighbor 2001:ded:beef:2::1 route-reflector-client 
 neighbor 2001:ded:beef:2::1 activate 
 neighbor 2001:ded:beef:2::1 next-hop-self 
 maximum-paths 4 
 maximum-paths ibgp 4 
 exit-address-family 

At runtime:
 cumulus@switch:$ show ip bgp neighbor 14.0.0.1 
 BGP neighbor is 14.0.0.1, remote AS 65000, local AS 65000, internal link
 BGP version 4, remote router ID 0.0.0.6 
 BGP state = Established, up for 00:23:49
 Last read 23:31:36, hold time is 180, keepalive interval is 60 seconds 
 Neighbor capabilities: 
 4 Byte AS: advertised and received 
 Route refresh: advertised and received(old & new)
 Address family IPv4 Unicast: advertised and received 
 Message statistics: 
 Inq depth is 0 
 Outq depth is 0
 Sent Rcvd 
 Opens: 2 0
 Notifications: 0 0
 Updates: 1 1
 Keepalives: 25 24 
 Route Refresh: 0 0
 Capability: 0 0 
 Total: 28 25 
 Minimum time between advertisement runs is 5 seconds
 For address family: IPv4 Unicast 
 >>>>>>>>>>>>>>>>>>>>>> ROUTE REFLECTOR CLIENT NOT DISPLAYED 
 NEXT_HOP is always this router 
 Community attribute sent to this neighbor(both) 
 6 accepted prefixes 
 Connections established 1; dropped 0 
 Last reset never 
 Local host: 14.0.0.2, Local port: 179 
 Foreign host: 14.0.0.1, Foreign port: 40290 
 Nexthop: 14.0.0.2 
 Nexthop global: 2001:ded:beef::2 
 Nexthop local: fe80::202:ff:fe00:4
 BGP connection: non shared network
 Read thread: on Write thread: off 
 cumulus@switch:$ 

Workaround:
 Define in following order 
 address-family ipv4 unicast
 neighbor 14.0.0.9 activate 
 neighbor 14.0.0.9 next-hop-self
 neighbor 14.0.0.9 route-reflector-client >>> Must be after Activate 
 exit-address-family 
 neighbor 2001:ded:beef:2::1 remote-as 65000
 address-family ipv6 unicast 
 redistribute connected
 maximum-paths 4 
 maximum-paths ibgp 4 
 neighbor 2001:ded:beef:2::1 activate 
 neighbor 2001:ded:beef:2::1 next-hop-self 
 neighbor 2001:ded:beef:2::1 route-reflector-client >>> Must be after activate 
 exit-address-family 
 Runtime status after change: 

cumulus@switch:$ show ip bgp neighbors 14.0.0.9 BGP neighbor is 14.0.0.9, remote AS 65000, local AS 65000, internal link BGP version 4, remote router ID 0.0.0.7 BGP state = Established, up for 00:13:59 Last read 22:35:13, hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: 4 Byte AS: advertised and received Route refresh: advertised and received(old & new) Address family IPv4 Unicast: advertised and received Message statistics: Inq depth is 0 Outq depth is 0 Sent Rcvd Opens: 1 1 Notifications: 0 0 Updates: 2 1 Keepalives: 15 14 Route Refresh: 0 0 Capability: 0 0 Total: 18 16 Minimum time between advertisement runs is 5 seconds For address family: IPv4 Unicast Route-Reflector Client >>>>>>>>>> PLEASE NOTE ME NEXT_HOP is always this router Community attribute sent to this neighbor(both) 6 accepted prefixes Connections established 1; dropped 0 Last reset never Local host: 14.0.0.10, Local port: 38813 Foreign host: 14.0.0.9, Foreign port: 179 Nexthop: 14.0.0.10 Nexthop global: 2001:ded:beef:2::2 Nexthop local: fe80::202:ff:fe00:6 BGP connection: non shared network Read thread: on Write thread: off cumulus@switch:$
RN-68 (CM-923) Blackhole/Unreachable/Prohibit route addition in IPv6 returns corresponding error codes IPv6 route operations indicate the destination action via returned error codes. In the example shown below where an unreachable route is being added, the return code is:
 #define ENETUNREACH 101 /* Network is unreachable */ 
cumulus@switch:$ sudo ip addr add 9000:1000:1000:1000::1/80 dev lo cumulus@switch:$ sudo ip -6 route unreachable 9000:1000:1000:1000::/80 dev lo proto kernel metric 256 error -101
RN-70 (CM-1166) ACL: Bridge traffic that matches a LOG ACTION rule is not logged in syslog For example, a bridge with switch ports swp1, swp2, swp3 as bridge members is configured. ACL rules to LOG and DROP for icmp traffic are configured.

Ping requests are sent from host1 on swp1 to host3 on swp3, and the following was observed:
* Counters for both LOG and DROP ACL rules are incrementing properly, but the packets are not showing up on /var/log/syslog.
* Packets that are copied to the CPU from hardware for the LOG rule are dropped due to the check in kernel to disable software bridging for hardware bridged packets.
RN-88 (CM-1200) SNMP support for Quagga is NOT provided in Cumulus VX Cumulus VX does not provide SNMP support for Quagga.
RN-125 (CM-1576) Network LSA with an old router ID isn't flushed out by the originator When the router ID is changed, the router should remove the previous network LSA (link-state advertisement) that it generated based on the IP address on the interface in the Network LSA.

Cumulus Networks doesn't remove this LSA, so it will be naturally aged out.
RN-132 (CM-2272) You must run "apt-get update" before running any apt-get commands or after changing sources.list

Before running any apt-get commands or after changing the source.list file in /etc/apt, you need to run apt-get update.

RN-133 (CM-2273) Interface names in Cumulus VX cannot exceed 15 characters

Device names, including interface names, in Cumulus VX cannot exceed 16 characters – including the terminator. Cumulus VX truncates longer interface names.

To avoid this issue, do not assign long names to your interfaces.

The following example configuration reproduces this issue:

cumulus@switch:/sys/class/net$ grep 'iface br' /etc/network/interfaces 
iface br2-pubmgmt inet static
iface br3-prvmgmt inet manual
iface br400-quarantine inet manual
iface br401-peering-1k5 inet manual
iface br402-peering-9k inet manual
iface br500-pi-exa inet manual
iface br501-akamai-exa inet manual
iface br502-exa-internetfactory inet manual
cumulus@switch:/sys/class/net$ brctl show | grep br
bridge name	bridge id	 STP enabled	interfaces
br2-pubmgmt	 8000.089e01cebe37	no	 bond0.2
br3-prvmgmt	 8000.089e01cebe3a	no	 bond0.3
br400-quarantin	 8000.089e01cebe37	no	 bond0.400
br401-peering-1	 8000.089e01cebe3a	no	 bond0.401 <<<
RN-163 (CM-2499) VXLAN: ovsdb-server cannot select loopback interface as source IP address, causing TOR registration to the controller to fail

In a VXLAN using VMware NSX, ovsdb-server cannot select the loopback interface as the source IP address. This causes TOR registration to the controller to fail.

To work around this issue, run:

cl-bgp redistribute add connected
RN-192 (CM-3340)

PTM may crash when a large topology file has a syntax error

If there is a syntax error in a large (~ 4000 entries) topology.dot file, PTM may crash while reading/parsing this file. This can occur because the libcgraph API that PTM uses for parsing the file cannot handle the error. You should check topology.dot for syntax errors using a Graphviz tool like dot or dotty that can identify the syntax error.

RN-196 (CM-2499) For a VXLAN in NSX, ovsdb-server cannot select a loopback interface as the SRC IP

As a result, the TOR registration to the controller fails.

To work around this issue, run:

cl-bgp redistribute add connected
RN-199 (CM-2624) When a Quagga route-map is modified, the switch could use the partial map before edits are completed

Cumulus VX triggers a route-map update before the user finishes editing the route map, resulting in an incorrect route map being used. The route-map update trigger should only occur when user finishes editing the map.

Cumulus Networks is working to fix this issue.

RN-210 (CM-3716) Restarting or clearing BGP can lead to SYN flooding warning, triggering an unnecessary alarm

When you run bgp clear or restart BGP, this message can appear on console (in vtysh):

179 dynamic neighbor(s), limit 90
switch07# clear bgp *
switch07#TCP: Possible SYN flooding on port 179. /  Sending cookies. Check SNMP counters. 

The warning also appears in syslog:

Sep 18 17:27:40 switch07 kernel: TCP: Possible SYN flooding on port 179. /  Sending cookies. Check SNMP counters.

This issue is caused by the maximum number of socket connections that are accepted on a listening socket is reached.

You can safely ignore this warning. Or to silence, it, you can increase the number of socket connections by changing the value for net.conf.somaxconn, which defaults to 128. If you increase net.conf.somaxconn to a larger number, the issue is no longer occurs.

RN-217 (CM-4207) LNV: Network restart removes vxsnd anycast IP address from loopback interface

Given the following conditions:

  • You have not configured a loopback anycast IP address in /etc/network/interfaces
  • You enabled the vxsnd (service node daemon) log to automatically add anycast IP addresses

When you restart networking (with service networking restart), the anycast IP address gets removed from the loopback interface.

To prevent this issue from occurring, you should specify an anycast IP address for the loopback interface in both /etc/network/interfaces and vxsnd.conf. This way, in case the vxsnd fails, you can withdraw the IP address.

RN-221 (CM-4501) BGP graceful restart, including helper mode, not fully supported If you encounter issues with this, please submit a support request and include the output from cl-support with your ticket.
RN-227 (CM-3388) BGP dynamic capability is not supported BGP peer sessions with dynamic capability are not supported under by Cumulus VX at this time.
RN-228 (CM-4384) ifupdown2 doesn't bring up loopback interface (lo) by default

ifupdown2 doesn't bring up loopback interface (lo) by default.

One effect is that since monit uses loopback interfaces, if you delete the lo interface in /etc/network/interfaces, monit commands will not work.

RN-229 (CM-4433) When a bond subinterface that is part of a traditional bridge is brought down, it flaps that bridge This issue has been encountered in environments where both VLAN-aware and traditional bridges are in use, where a traditional bridge has a subinterface of a bond that is present as a normal interface in a VLAN-aware bridge.
RN-230 (CM-4327) bond-min-links defaults to 0, which causes the bond carrier to be incorrectly up when no slaves are active Even though bond-min-links defaults to 0, Cumulus Networks recommends a minimum link setting of 1 or higher for bonds. Cumulus VX issues a warning when the setting is 0.
RN-268 (CM-5296) bridge fdb add command IP address type options inconsistent with Debian

The bridge fdb add command handles the IP address type in a non-standard way:

  • bridge fdb add [MAC address] dev [INTERFACE] master defaults to a static address
  • bridge fdb add [MAC address] dev [INTERFACE] master temp maps to dynamic address
  • bridge fdb add [MAC address] dev [INTERFACE] master local maps to permanent address

This issue should be fixed in a future version of Cumulus VX.

RN-275 (CM-5794) BGP import-check fails for IPv6 route if static routes to null0 are used

The path that Cumulus VX originates should not be invalid since there is a matching route in the RIB. The import check works fine for IPv4 routes.

RN-276 (CM-5754) MLAG peer is alive on one switch but is not alive on its peer switch

One MLAG node can ping its peer switch, but the peer cannot do so. You can see this when you run clagctl:

cumulus@switch3:~$ sudo clagctl 
The peer is not alive
     Our Priority, ID, and Role: 32768 00:e0:ec:25:2f:83 primary
          Peer Interface and IP: peerlink.4000 11.1.2.2
                      Backup IP: 12.34.9.36 (active)
                     System MAC: 00:00:00:03:12:01
cumulus@switch4:~$ sudo clagctl 
The peer is alive
    Peer Priority, ID, and Role: 32768 00:e0:ec:25:2f:83 primary
     Our Priority, ID, and Role: 32768 44:38:39:00:1e:82 secondary
          Peer Interface and IP: peerlink.4000 11.1.2.1
                      Backup IP: 12.34.9.38 (active)
                     System MAC: 00:00:00:03:12:01

To work around this issue, you can do one of two things:

  • Restart clagd on both nodes using service clagd restart
  • Bring the peer link down then up using ifdown <peer-link> then ifup <peer-link>
RN-278 (CM-6048) Setting new metric value with BGP redistribute command does not take effect

If you configure a new metric value using the BGP redistribute command, the running configuration does not change. To work around this issue, restart the routing process (bgpd — the preferred method since it is less disruptive) or the quagga service:

cumulus@switch:~$ sudo service quagga restart bgpd

OR

cumulus@switch:~$ sudo service quagga restart
RN-279 (CM-6012) STP not supported on bonds in LACP bypass all-active mode

Cumulus VX does not support STP on bonds in LACP bypass all active mode. To avoid this issue, enable STP BPDU guard for all bonds in LACP bypass all-active mode. For information, read this document.

RN-281 (CM-5118) Default route not removed on ifup after removing gateway statement from eth0 configuration

If you try to remove the default route from eth0 (either by commenting out or removing the gateway statement in the eth0 configuration), the route remains after running ifup.

To work around this issue, first run ifdown, then ifup on the interface via the console. After this, the route disappears.

RN-297 (CM-6801) "Unable to determine platform" error seen in Cumulus VX

Cumulus VX does not report a platform (as reported by the platform-detect command), so some commands will not execute correctly. This renders dependent packages, such as update-grub, unusable.

For example:

cumulus@switch:~$ sudo platform-detect 
ERROR:  Unable to determine platform for ARCH: x86_64
UNKNOWN
cumulus@switch:~$ sudo update-grub
Generating grub.cfg ...
/etc/grub.d/10_cumulus: 27: /etc/cumulus/init/platform.conf: log_failure_msg: not found
RN-298 (CM-6169) Cannot start clagd service unless you specify the --vm option

If you are configuring MLAG, in order to start the clagd service, you must specify the clagd-args --vm option in /etc/network/interfaces:

auto peer16.4000
iface peer16.4000 inet static
    address 11.0.5.2
    netmask 255.255.255.0
    clagd-enable yes
    clagd-peer-ip 11.0.5.1
    clagd-sys-mac 44:38:39:ff:ff:ff
    clagd-args --vm
Have more questions? Submit a request

Comments

Powered by Zendesk