Share

How to bring up Oracle Exadata infiniband switch ports when they are down due to AutomaticHighErrorRate?

When the number of errors on Oracle Exadata infiniband switch ports reaches the specified threshold values, it is automatically brought down to AutomaticHighErrorRate. After the problem is resolved, it can be brought up as follows.

First of all, you should connect to the problematic Infiniband switch via SSH.

Detection of port in down state:

[root@exasw-iba01 ~]# listlinkup
Connector  0A Not present
Connector  1A Not present
Connector  2A Present <-> Switch Port 24 is up (Enabled)
Connector  3A Not present
Connector  4A Not present
Connector  5A Present <-> Switch Port 30 is up (Enabled)
Connector  6A Not present
Connector  7A Not present
Connector  8A Present <-> Switch Port 31 is up (Enabled)
Connector  9A Present <-> Switch Port 14 is up (Enabled)
Connector 10A Present <-> Switch Port 16 is up (Enabled)
Connector 11A Present <-> Switch Port 18 is up (Enabled)
Connector 12A Not present
Connector 13A Not present
Connector 14A Not present
Connector 15A Not present
Connector 16A Not present
Connector 17A Not present
Connector  0B Not present
Connector  1B Present <-> Switch Port 21 is up (Enabled)
Connector  2B Not present
Connector  3B Not present
Connector  4B Present <-> Switch Port 27 is up (Enabled)
Connector  5B Present <-> Switch Port 29 is up (Enabled)
Connector  6B Present <-> Switch Port 36 is down (AutomaticHighErrorRate)
Connector  7B Not present
Connector  8B Not present
Connector  9B Present <-> Switch Port 13 is up (Enabled)
Connector 10B Present <-> Switch Port 15 is up (Enabled)
Connector 11B Present <-> Switch Port 17 is up (Enabled)
Connector 12B Not present
Connector 13B Not present
Connector 14B Not present
Connector 15B Not present
Connector 16B Not present
Connector 17B Not present

As can be seen from the output above, port 36 is in down state. You can bring this port up as follows.

[root@exasw-iba01 ~]# enableswitchport --automatic Switch 36
Enable connector 6B Switch port 36
Adminstate:......................Enabled
LinkWidthEnabled:................1X or 4X
LinkWidthSupported:..............1X or 4X
LinkWidthActive:.................4X
LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps
LinkState:.......................Down
PhysLinkState:...................Polling
LinkSpeedActive:.................2.5 Gbps
LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps
NeighborMTU:.....................2048
OperVLs:.........................VL0-7

After waiting for a while, when you check with the listlinkup command, the relevant port will be in Up and Enabled status.

[root@exasw-iba01 ~]# listlinkup
Connector  0A Not present
Connector  1A Not present
Connector  2A Present <-> Switch Port 24 is up (Enabled)
Connector  3A Not present
Connector  4A Not present
Connector  5A Present <-> Switch Port 30 is up (Enabled)
Connector  6A Not present
Connector  7A Not present
Connector  8A Present <-> Switch Port 31 is up (Enabled)
Connector  9A Present <-> Switch Port 14 is up (Enabled)
Connector 10A Present <-> Switch Port 16 is up (Enabled)
Connector 11A Present <-> Switch Port 18 is up (Enabled)
Connector 12A Not present
Connector 13A Not present
Connector 14A Not present
Connector 15A Not present
Connector 16A Not present
Connector 17A Not present
Connector  0B Not present
Connector  1B Present <-> Switch Port 21 is up (Enabled)
Connector  2B Not present
Connector  3B Not present
Connector  4B Present <-> Switch Port 27 is up (Enabled)
Connector  5B Present <-> Switch Port 29 is up (Enabled)
Connector  6B Present <-> Switch Port 36 is up (Enabled)
Connector  7B Not present
Connector  8B Not present
Connector  9B Present <-> Switch Port 13 is up (Enabled)
Connector 10B Present <-> Switch Port 15 is up (Enabled)
Connector 11B Present <-> Switch Port 17 is up (Enabled)

 

Loading

You may also like