40G Active Optical Cable (AOC) Doesn't Come Up or Pass Traffic on Boot

Follow

Issue

The 40G active optical cable (AOC) cable doesn't come up unless it's physically removed from the slot and reinserted after power cycle.

Environment

Software:

  • Cumulus Linux 1.5.x
  • Cumulus Linux 2.0.x

Hardware (Switch):

  • Penguin Arctica 4804X
  • Penguin Arctica 3200XL

Hardware (Cable):

Cause

The AOC QSFP module requires a toggle of the QSFP reset pin on subsequent bootups after the initial insertion. The lpmod setting of the QSFP is also defaulting to low-power-mode; however, changing the value does not restore the link. This may possibly be unrelated, but is included for completeness.

The lpmodreset, and present signals are not exposed via sysfs.

Solution

lpmodreset and present lines were exposed via sysfs in Cumulus Linux 2.0.2.

Cumulus Networks has created a port-watch daemon, portwd, to poll QSFP ports and reset if they are in a hung state. The portwd daemon will be rolled into a future release.

The intention is for this daemon to expand its use to other front-panel operations.

Workaround

In the meantime, the daemon is available as a .deb package in the testing repository.

To install the package on affected systems, please use the following instructions:

  1. Add the testing repository:
    sudo vi /etc/apt/sources.list
  2. Uncomment the testing repo line (Remove the leading #)
    # deb http://repo.cumulusnetworks.com CumulusLinux-2.0 testing
  3. Save and quit vi:
    :wq
  4. Add the portwd package:
    sudo apt-get install portwd 
Have more questions? Submit a request

Comments

This support portal has moved

Cumulus Networks is now part of the NVIDIA Networking Business Unit! The NVIDIA Cumulus Global Support Services (GSS) team has merged its operations with the NVIDIA Mellanox support services team.

You can access NVIDIA Cumulus support content from the Mellanox support portal.

You open and update new cases on the Mellanox support portal. Any previous cases that have been closed have been migrated to the Mellanox support portal.

Cases that are still open on the Cumulus portal will continue to be managed on the Cumulus portal. Once these cases close, they will be moved to the Mellanox support portal.

Powered by Zendesk