During routine maintenance, I’ve had occasions where HPE DL380 Gen9 Servers lose the plot when rebooted. Last week was a prime example and the second time this has happened for a server. The server was patched with Windows Updates and restarted before the installation of the HPE Service Support Pack.
At which point it became clear the HPE FlexibleLOM NIC was missing from the server configuration. When checking the system’s BIOS there were no references to the PCI card, in the iLO system information it had completely disappeared. The only NIC’s visible were Adapter 1 – iLO and Adapter 2 – HPE Ethernet 1GB 4-Port 331i Adapter.
Thankfully I’ve been through this before and knew how to work around the issue. Unfortunately, the usual cold boot or iLO resets didn’t work, I had to utilise one of the most trusty troubleshooting tips of all time. The classic ‘fix’ from HPE was to pull the power cords from the server, this then magically ‘resolved’ the issue and we were able to carry on with the scheduled maintenance. After patching the server, I was hoping it would resolve the issue, but alas it didn’t. When the same fault occurred again, I thought there must be an easier way than driving to the location of the server or initiating remote hands support.
This time I decided to utilise a different method to get the server operational. Since iLO 4 version 2.55 a feature was added to perform an auxiliary power-cycle on a server remotely. This can be done by making use of the RESTful Interface Tool. The RESTful Interface Tool connects to the Integrated Lights-Out system and allows the use of a comprehensive iLO RESTful API’s to perform tasks on an iLO system. I followed the HPE reference guide and performed the following commands to initiate the AuxCycle on the server.
Sequence AuxCycle:
- Install HPE RESTful Interface Tool
- Create a text file called auxcycle.json and enter the information below, save it in the location in which you are running the tool. In my example it was located in the user’s documents folder
- Run RESTful Interface Tool as administrator
- Type the following commands
- ilorest login
- ilorest login 10.0.1.1 -u adminusername -p password123
- ilorest rawpost auxcycle.json
- ilorest logout
{
“path”: “/redfish/v1/Systems/1/Actions/Oem/Hp/ComputerSystemExt.SystemReset/”,
“body”: { “Action”: “SystemReset”, “ResetType”: “AuxCycle”}
}
Contents of auxcycle.json
Please remember to change the IP Address, Username, and Password values to ones that apply to your environment.
After performing the commands listed, the iLO reset itself, the system was turned on and after completing a success POST the FlexibleLOM NIC appeared.
Server Configuration:
- HPE DL380 Gen9 BIOS Version P89 V2.76 (21st October 2019)
- ILO Version 2.73 (11th Feb 2020)
- TPM 2.0
- HPE FlexFabric 10Gb 2-Port 556FLR-SFP+ Adapter Firmware 12.0.1110.11
HPE Reference: https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-a00047494en_us
HPE Restful Interface Tool: https://buy.hpe.com/us/en/software/infrastructure-management-software/system-server-management-software/hpe-system-server-software-management-software/restful-interface-tool/p/7630408