How to bring up Oracle Exadata infiniband switch ports when they are down due to AutomaticHighErrorRate?
When the number of errors on Oracle Exadata infiniband switch ports reaches the specified threshold values, it is automatically brought down to AutomaticHighErrorRate. After the problem is resolved, it can be brought up as follows.
First of all, you should connect to the problematic Infiniband switch via SSH.
Detection of port in down state:
[root@exasw-iba01 ~]# listlinkup
Connector 0A Not present
Connector 1A Not present
Connector 2A Present <-> Switch Port 24 is up (Enabled)
Connector 3A Not present
Connector 4A Not present
Connector 5A Present <-> Switch Port 30 is up (Enabled)
Connector 6A Not present
Connector 7A Not present
Connector 8A Present <-> Switch Port 31 is up (Enabled)
Connector 9A Present <-> Switch Port 14 is up (Enabled)
Connector 10A Present <-> Switch Port 16 is up (Enabled)
Connector 11A Present <-> Switch Port 18 is up (Enabled)
Connector 12A Not present
Connector 13A Not present
Connector 14A Not present
Connector 15A Not present
Connector 16A Not present
Connector 17A Not present
Connector 0B Not present
Connector 1B Present <-> Switch Port 21 is up (Enabled)
Connector 2B Not present
Connector 3B Not present
Connector 4B Present <-> Switch Port 27 is up (Enabled)
Connector 5B Present <-> Switch Port 29 is up (Enabled)
Connector 6B Present <-> Switch Port 36 is down (AutomaticHighErrorRate)
Connector 7B Not present
Connector 8B Not present
Connector 9B Present <-> Switch Port 13 is up (Enabled)
Connector 10B Present <-> Switch Port 15 is up (Enabled)
Connector 11B Present <-> Switch Port 17 is up (Enabled)
Connector 12B Not present
Connector 13B Not present
Connector 14B Not present
Connector 15B Not present
Connector 16B Not present
Connector 17B Not present
As can be seen from the output above, port 36 is in down state. You can bring this port up as follows.
[root@exasw-iba01 ~]# enableswitchport --automatic Switch 36
Enable connector 6B Switch port 36
Adminstate:......................Enabled
LinkWidthEnabled:................1X or 4X
LinkWidthSupported:..............1X or 4X
LinkWidthActive:.................4X
LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps
LinkState:.......................Down
PhysLinkState:...................Polling
LinkSpeedActive:.................2.5 Gbps
LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps
NeighborMTU:.....................2048
OperVLs:.........................VL0-7
After waiting for a while, when you check with the listlinkup command, the relevant port will be in Up and Enabled status.
[root@exasw-iba01 ~]# listlinkup
Connector 0A Not present
Connector 1A Not present
Connector 2A Present <-> Switch Port 24 is up (Enabled)
Connector 3A Not present
Connector 4A Not present
Connector 5A Present <-> Switch Port 30 is up (Enabled)
Connector 6A Not present
Connector 7A Not present
Connector 8A Present <-> Switch Port 31 is up (Enabled)
Connector 9A Present <-> Switch Port 14 is up (Enabled)
Connector 10A Present <-> Switch Port 16 is up (Enabled)
Connector 11A Present <-> Switch Port 18 is up (Enabled)
Connector 12A Not present
Connector 13A Not present
Connector 14A Not present
Connector 15A Not present
Connector 16A Not present
Connector 17A Not present
Connector 0B Not present
Connector 1B Present <-> Switch Port 21 is up (Enabled)
Connector 2B Not present
Connector 3B Not present
Connector 4B Present <-> Switch Port 27 is up (Enabled)
Connector 5B Present <-> Switch Port 29 is up (Enabled)
Connector 6B Present <-> Switch Port 36 is up (Enabled)
Connector 7B Not present
Connector 8B Not present
Connector 9B Present <-> Switch Port 13 is up (Enabled)
Connector 10B Present <-> Switch Port 15 is up (Enabled)
Connector 11B Present <-> Switch Port 17 is up (Enabled)