While trying to install Cumulus Linux for the first time, ONIE does not receive the DHCP address and uses the default 192.168.3.10 address. As a result, it cannot find the install server.
Even if the correct DHCP address is received, ONIE does not find the install server because the new DHCP address is not installed.
- ONIE versions prior to Feb 2014
There were a couple of issues in the 2013 versions of ONIE where DHCP would fail in 15 seconds or less, and once it had failed, the default IP address was installed permanently, even if DHCP later correctly received a correct address. As a result, in networks where the upstream switch is running "classic" Spanning-Tree Protocol (802.1D), DHCP often times out before the upstream port allows the DHCP request to travel to the DHCP server.
The DHCP issues were resolved in ONIE version 2014.02 (available on GitHub). However, ONIE versions are platform-specific and must be obtained from the switch manufacturer. On Edge-Core switches, the ONIE version with these fixes is version 18.104.22.168.
Once Cumulus Linux is installed the first time, the ONIE DHCP function is generally not needed again during the life of the switch, so the better solution is to work around the issue.
The best workaround is to enable Rapid Spanning-Tree Protocol (RSTP, IEEE 802.1w) on the upstream switch. There are two features in RSTP that makes this the most optimal solution:
- RSTP defaults to Edge configuration when a point-to-point port comes up (detected by the port operating in full-duplex mode), meaning that it assumes a host exists on each point-to-point link until it hears a bridge BPDU and therefore puts the port in forwarding state immediately. This allows the DHCP request/response to work nearly instantaneously. Note that ONIE acts as a host to the upstream switch — no bridging functions are active in ONIE.
- RSTP will "down-rev" to classic STP (802.1D) if it detects a classic STP bridge on a port, so there is no risk of bridge loops due to incompatible bridge protocols.
A more dangerous workaround would to be to disable STP temporarily or configure Portfast on the upstream switch port. This allows the DHCP request/response to be forwarded when the port comes up and allows ONIE to complete the install. However, if this STP-disabled/Portfast configuration on the upstream switch is made persistent or is not removed immediately, larger problems may occur: once the new downstream switch is operational, there will be no STP mechanism in place to prevent a spanning tree loop.