Configuring a Management Namespace

Follow

Deprecated Feature

This feature has been deprecated, but the content is being preserved for users on older versions of Cumulus Linux. You should use management VRF instead.

Issue

I would like to separate my management network and my in-band network so routes don't overlap (for example, with dual default routes).

Background

Network namespaces create multiple discrete copies of the networking stack in Linux. Cumulus Linux uses network namespaces to separate the routing table for eth0, much like you would do with a management VRF on other boxes.

Initially, all interfaces belong to the default namespace. Cumulus Linux supports placing eth0 into namespaces other than the default namespace. If a front panel port is placed into a non-default namespace, it will not be able to send or receive traffic.

Placing the eth0 interface in a management namespace allows you to separate the routing table for your management network from the routing table for the front panel switch ports. This prevents traffic from coming in a front panel port and exiting via eth0 and vice versa.

Environment

Configuration

To create a management namespace for eth0, use the cl-ns-mgmt utility. A typical set of options to pass the tool are:

cumulus@switch:~/usr/cumulus/bin$ sudo cl-ns-mgmt --mgmt-services ntp,ssh --data-services ssh --setup

Note: The cl-ns-mgmt utility is an experimental feature.

Running cl-ns-mgmt with the options above does the following:

  • Creates a management namespace called mgmt and moves the eth0 interface into this namespace. You can name this namespace something else by using the --ns option. For example: cl-ns-setup --ns Management
  • Creates a lo interface with 127.0.0.1 in the mgmt namespace. Your normal lo IP will remain in the default namespace.
  • Creates an alias for the default namespace called default.
  • Extracts the eth0 config from /etc/network/interfaces and places it in /etc/netns/mgmt/network/interfaces.
  • Creates a NAMESPACE environment variable which is used as part of the bash prompt.
  • Creates aliases that can be used to move between namespaces and to execute commands in a different namespace.
  • Edits /etc/rc.local so that cl-ns-setup runs after a reboot to put eth0 back into the mgmt namespace.
  • Starts the SSH service in the mgmt namespace.
  • Starts the NTP service in the mgmt namespace. The majority of the time your NTP servers are reachable via eth0 so you need to run the NTP service in this namespace.
  • Starts the SSH service in the default namespace.

Additionally, if snmpd is running, cl-ns-mgmt will also:

  • Create a pair of veth interfaces, one in mgmt and one in default.
  • Use iptables to create a NAT translation from eth0 UDP port 161 to the veth interface in default.
  • Restart snmpd in the default namespace.
  • These steps allow an snmpwalk targeted at eth0 to "see" the front panel ports in the default namespace.

To reverse the changes made by cl-ns-mgmt, use the --undo option. If you do use --undo, it is recommended that you reboot the device afterwards.

Example

When you run the script you will lose your SSH session to the switch and will need to log in again:

cumulus@switch:~$ sudo cl-ns-mgmt --mgmt-services ntp,ssh --data-services ssh --setup

See /var/log/cl-ns-mgmt.log for logs.

[ ok ] Stopping NTP server: ntpd.
[ ok ] Stopping OpenBSD Secure Shell server: sshd.
Unmount /etc/ssh failed: Invalid argument
[ ok ] Stopping OpenBSD Secure Shell server: sshd.
Unmount /etc/ssh failed: Invalid argument
Write failed: Broken pipe
cumulus@switch[bin]#

When you log in you will automatically be placed in the default namespace. This is because the majority of the changes that you make to the switch (like Quagga, switch ports, and so forth) will occur in the default namespace.

-set aliases have been created that allow you to move from one namespace to another. Note that the bash prompt changes as you -ns-set from one namespace to another.

cumulus@switch:default:~:$
cumulus@switch:default:~:$ sudo mgmt-ns-set
cumulus@switch:mgmt:~:$
cumulus@switch:mgmt:~:$ default-ns-set
cumulus@switch:default:~:$
cumulus@switch:default:~:$

While in a namespace, you can only see the interfaces, routes, and so on for that namespace. For example, you are in the mgmt namespace, so you only see eth0 and lo but not any of the swp ports.

cumulus@switch:mgmt:~:$
cumlus@switch:mgmt:~:$ ip link show
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 00:e0:ec:25:2f:3c brd ff:ff:ff:ff:ff:ff
55: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
cumlus@switch:mgmt:~:$

The -ns aliases can be used to execute a command in a different namespace. For example, you are in the default namespace, but you run ip link show in the mgmt namespace.

cumulus@switch:default:~:$
cumulus@switch:default:~:$ mgmt-ns ip link show
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 00:e0:ec:25:2f:3c brd ff:ff:ff:ff:ff:ff
55: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
cumulus@switch:default:~:$

Caveats/Gotchas

  • When you SSH into a certain namespace, you will see only the routes and network information from that network namespace.
  • Quagga can only deal with interfaces in the default namespace.
  • Restarting switchd and quagga must be done from the default namespace.

Namespaces Created in Cumulus Linux 2.2.x Break when You Upgrade to Cumulus Linux 2.5

Namespaces created in Cumulus Linux 2.2.x break when you upgrade your switches to Cumulus Linux 2.5. To preserve the namespaces, do the following:

  1. Download the Cumulus Linux 2.5 package.
    For PowerPC, run:
    wget http://repo.cumulusnetworks.com/pool/CumulusLinux-2.5/main/cl-utilities_1.1-cl2.5_powerpc.deb
    For x86, run:
    wget http://repo.cumulusnetworks.com/pool/CumulusLinux-2.5/main/cl-utilities_1.1-cl2.5_amd64.deb
  2. Install the software.
    For PowerPC, run:
    sudo dpkg -i cl-utilities_1.1-cl2.5_powerpc.deb
    For x86, run:
    sudo dpkg -i cl-utilities_1.1-cl2.5_amd64.deb
  3. Launch the root shell:
    sudo /bin/bash
  4. Create a NAMESPACE environment variable:
    export NAMESPACE=default
  5. Execute --mnt-persist:
    cl-ns-mgmt --mnt-persist
  6. Exit the root shell:
    exit
  7. Install the Cumulus Linux image onto the switch:
    sudo cl-img-install <url>
  8. Select the new image slot:
    sudo cl-img-select <slot_number>
  9. Reboot the switch:
    sudo reboot

 

Have more questions? Submit a request

Comments

Powered by Zendesk