Using dtach for In-band apt-get Upgrades

Follow

ud540.md:

Issue

My ssh session was terminated while performing an in-band upgrade.

Environment

  • Cumulus Linux 1.5, 2.y
  • In-band ssh connection: upgrading via ssh connection over a logical (bridge or bond) or switch (swp) port, rather than the management eth0 port.

Resolution

When upgrading in-band, it is recommended to run the upgrade with the dtach command. This makes it possible to re-attach to the upgrade process in the event the connection is interrupted.

cumulus@switch$ sudo apt-get update
cumulus@switch$ dtach -c /tmp/apt-get -z sudo apt-get dist-upgrade

Where:

  • /tmp/apt-get is a socket that will be used to re-attach later
  • -z disables the suspend key from suspending dtach

To re-attach to the apt-get process after reconnecting to the switch:

cumulus@switch$ dtach -a /tmp/apt-get

If the dtach socket file does not exist as in the following error:

cumulus@switch$ dtach -a /tmp/apt-get
dtach: /tmp/apt-get: No such file or directory

... then upgrade has finished. Prior to rebooting (if needed) results of the upgrade can be reviewed in /var/log/apt/term.log.

cumulus@switch$ less /var/log/apt/term.log

Root Cause

Upgrades involving the switchd package may result in a temporary loss of in-band ssh connections (generally a minute or two) to the switch.

Diagnostic Steps

The following commands to check if an upgrade will include switchd:

cumulus@switch$ sudo apt-get update
cumulus@switch$ sudo apt-get dist-upgrade -uV --assume-no | grep switchd
Have more questions? Submit a request

Comments

Powered by Zendesk