HBoot Analysis

For the complete list of my articles on Android devices and software, including analysis of devices and system firmware, lists of external resources and tools, and How-To instructions, check the front page of this wiki under the Android heading

As part of my investigation into methods for replacing the HTC Linux kernel and Android system via a firmware update I am noting interesting information here.

The aim of this analysis is to eventually be able create a GPL-ed boot-loader that will remove reliance on the HTC hboot and can be extended freely by others.

Current Status

9th November: Moved the instructions for establishing a remote serial terminal connection to the radio to the Radio Analysis page.

8th November: Added summary explanation of rtask D in the rtask list. Added information on FastBoot mode commands for a host PC to query the device when in "FASTBOOT USB" mode.

7th November: Finally pinned down the exact command to unlock security (S-ON/S-OFF) via the @secu_flag. I can query it successfully. I'm holding off attempting to use it until I fully understand some code and data structures that surround it. I don't want to cause a silicon logic fuse to blow. For the interested, the radio modem command to query the flag is "AT@SIMLOCK?AA". The command to set the flag is "AT@SIMLOCK=7,X" where X is 0 for S-OFF and 1 for S-ON. See the section further down rtask C - Talking to the Radio.

6th November: Listing the hboot "task" values and, where possible, what they do. rtask C - Talking to the Radio.

5th November: Taking a break from directly analysing hboot to work on my Linux tools that do the semi-automatic decomposition of the binary - think of them as an automated IDA Pro (the Windows dissasembler many users favour). I was spurred into this by the move to analysing the hboot RAM image since I needed to repeat days of manual analysis. I'm adding a database to the tools (sqlite) that stores symbol names and recognition signatures so once the information is gathered the tools can be re-run against any hboot image and in about 5 minutes produce a fully decoded pseudo source-code - instead of several days.

3rd November: Switched to analysing the in-memory (RAM) hboot image captured by Adam (Teferi) last week. This is already offering some great insights since it contains real values for variables as they were last before hboot started Linux. When analysing the hboot update ROM image these variable locations contain no useful values, which makes the code more opaque. Unfortunately I've got to repeat a lot of groundwork to get the RAM image analysis to the same level as the ROM image. One reason I held back from tackling this earlier was that the ROM image is a later version and therefore they don't match byte-for-byte, meaning my tools and my brain have to do extra work to figure out where variables, structures and functions are when mapping the ROM analysis to the RAM image.

3rd November: Completed analysis of embedded  zlib compression library, version 1.2.3 is used for extracting files from ZIP archives. Something I've looked at before but not documented yet is the way the CID can be changed via a micro_SD key-card. The card has a magic number embedded and contains a FAT file-system with the file DMCID.dat containing the new CID data. I've not dug into the routines in-depth but the value read from the file is superimposed into a radio modem command string "AT@CID=".

3rd November: Completed analysis of the libc (standard library) functions, and added them to the bootloader-ap project project. Spent several hours analysing some very large data structures that appear to be related to mapping regions and associated flags. This will need more work to discover the full extent of the structures, as well as what it is used for. Found a couple of code paths leading up from the S-OFF functions but got lost whilst tracking back and forward in the call stacks - will return to that with a clear head another time.

2nd November: Found the data structures that define the starting block, sizes and flags of all the eMMC partitions. Discovered that the 'radio' partition is treated as a conglomerate of the first 16 partitions of which the actual REXX/AMSS partition appears to be partition 5 (mmcblk0p5).

30th October: I need help identifying the purpose of two identical tables. Solved: They are page tables in easy-to-write-by-humans form that are then translated by additional code into ARM page table entries. I've attached them to this article as hboot-tables.tar.gz Download. They occur in the initialisation section of hboot. They are binary tables consisting of 364 elements each. Each element consists of three 32-bit values. They look like some kind of memory management unit (MMU) tables by their contents but their layout doesn't match either first- or second-level ARM page table entries (PTEs). See the new PageTables section below.

30th October: Realised that the hboot initialisation code (the first ~1700 bytes) is op-code identical to that contained in the Vision Linux kernel source-file arch/arm/mach-msm/arch-init-7x30.S. Not only has this revealed at a stroke what this code is doing (from the comments in the source-code file) but it also shines light on the legal status of the hboot binary, since the source-code file is licensed using the BSD 3-clause open source license. This helps clarify the status of the bootloader-ap project and means the source-file can be incorporated there directly.

28th October: Created the new GPLv3 bootloader-ap project. Using what I've learned from analysing hboot images I'm beginning the process of creating a binary-compatible source-code implementation that, once it is complete, can be used to provide users with a totally open-source applications-processor software stack. The initial commit to the git repository contains a working build system and architecture-specific assembler source-code that creates a ROM image that is binary-compatible with the hboot images.

24th Oct 2010: Investigations continuing, identifying function names and data structures.

  • hboot image fully disassembled
  • load address determined
  • some libc functions identified
  • source code for original hboot found
  • mapping original bootloader source to image. Successfully maps all the functions so far studied to machine code.
  • investigated Broadcom radio configuration settings
  • investigating msm_mpu_emmc_protect()
  • added latest function list as attachment. As at 2010-10-22 there are 1,292 functions recognised with 184 identified and named.
  • found routines that indicate an SD-Card "key-card" can set any CID
  • completed function list from embedded hboot clues. As at 2010-10-23 there are 1,308 functions recognised with 358 identified and named.

Time taken to date: 49 hours.


task commands

Hboot console has a "task ??" command where ?? is a hexadecimal number that represents some, possibly destructive, command to execute immediately. These are the values and what they do:

28  Format userdata
29  Format BIN-FS (system, cache, userdata)

rtask commands

This hboot console command is the radio-task command. Like 'task' it takes a single hexadecimal parameter that represents some function to perform. These are the values and what they do:

3 ?
A REXX shell
B Does rtask3 then rtask A (AMSS AT command interpreter (radio modem))
C Does rtask3 then lets AMSS take over the HS-USB port
D Sends the modem the "LTErfpath" command then starts the UART router on port #1

rtask C - Talking to the Radio

This is proving interesting since it allows us to communicate directly with the radio modem over USB serial interfaces. There's a bit of jiggling to be done first. First, issue the "rtask C" command and after a moment the serial link will disappear and, if you're using screen, it'll quit.

Check the list of USB devices and you'll now see there's a Qualcomm device attached:

lsusb | egrep '0bb4|05c6
Bus 001 Device 043: ID 05c6:9002 Qualcomm, Inc. 

Check the kernel log and you'll find:

tail /var/log/kern.log
 usb 1-1: USB disconnect, address 42
 generic ttyUSB0: generic converter now disconnected from ttyUSB0
 usbserial_generic 1-1:1.0: device disconnected
 usb 1-1: new high speed USB device using ehci_hcd and address 43
 usb 1-1: configuration #1 chosen from 1 choice
 USB Serial support registered for Qualcomm USB modem
 usb 1-1: unknown number of interfaces: 3
 usb 1-1: unknown number of interfaces: 3
 usb 1-1: unknown number of interfaces: 3
 usbcore: registered new interface driver qcserial

Because this is now a 'radio' analysis issue I've refined and moved the instructions. See Talking to the Radio.


There are a large number of functions (1,308), more than 384 identified. To save filling this page with the list I'll keep the latest version of the function names list in an attached file Download.

Function msm_mpu_emmc_protect()

This function and an associated sub-function give some indication of how HBoot deals with the eMMC write protection. The function name can be expanded to:

msm_ Mobile Station Modem (the Qualcomm system-on-chip)
mpu_ Memory Protection Unit (the ARM9 configures what addresses and types of access the ARM11 and other co-processors have access to)
emmc_ Embedded MultiMediaCard (the NAND flash storage device that presents as an external block device)

The function does a couple of interesting operations, each time calling a sub-function msm_mpu_emmc_protect_set(char *first, char *last, int bFlag) that takes a named range of partitions. E.g.

msm_mpu_emmc_protect_set("hboot", "mfg", 1);
msm_mpu_emmc_protect_set("hboot", "system", 1);

Page Tables

Generated using my pagetables tool:

./pagetables -f ../Vision/roms/RUU_1.22.405.1-WWE/hboot-table-0260.bin 
Base:	hexadecimal
Pagetable:		        1092	../Vision/roms/RUU_1.22.405.1-WWE/hboot-table-0260.bin
 Virtual     Physical    Count
{0x40000000, 0x05500000, 0x00000047},
{0x10000000, 0x09C00000, 0x00000064},
{0x49C00000, 0x19C00000, 0x00000064},
{0x16400000, 0x10000000, 0x0000009C},
{0x50000000, 0x20000000, 0x00000100},
{0x80000000, 0x04000000, 0x00000015},
{0x8D000000, 0x00000000, 0x00000004},
{0x00100000, 0x00400000, 0x00000001},
{0x8D400000, 0x00400000, 0x00000001},
{0x8C000000, 0x00700000, 0x00000003},
{0x90000000, 0xA7000000, 0x0000000D},
{0x90D00000, 0xAB000000, 0x00000001},
{0x90E00000, 0xA8900000, 0x00000001},
{0x90F00000, 0xA0000000, 0x00000001},
{0x91000000, 0xA8400000, 0x00000001},
{0x91100000, 0xA0200000, 0x00000002},
{0x91300000, 0xA3600000, 0x00000001},
{0x91400000, 0xA3100000, 0x00000001},
{0x91500000, 0xA3000000, 0x00000001},
{0x91600000, 0xA0500000, 0x00000001},
{0x91700000, 0xA0400000, 0x00000001},
{0x91800000, 0xA3300000, 0x00000001},
{0x91900000, 0xA3200000, 0x00000001},
{0x91A00000, 0xA3400000, 0x00000001},
{0x91B00000, 0xA3900000, 0x00000001},
{0x91C00000, 0xAD700000, 0x00000001},
{0x91D00000, 0xAD600000, 0x00000001},
{0x91E00000, 0xAD500000, 0x00000001},
{0x91F00000, 0xAD400000, 0x00000001},
{0x92000000, 0xAD300000, 0x00000001},
{0x92100000, 0xAC800000, 0x00000001},
{0x92200000, 0xACE00000, 0x00000001},
{0x92300000, 0xACD00000, 0x00000001},
{0x92400000, 0xAC900000, 0x00000001},
{0x92500000, 0xA3F00000, 0x00000001},
{0x92600000, 0xACC00000, 0x00000001},
{0x92700000, 0xACB00000, 0x00000001},
{0x92800000, 0xACA00000, 0x00000001},
{0x92900000, 0xA9800000, 0x00000001},
{0x92A00000, 0xAC500000, 0x00000001},
{0x92B00000, 0xAC400000, 0x00000001},
{0x92C00000, 0xAC300000, 0x00000001},
{0x92D00000, 0xAC200000, 0x00000001},
{0x92E00000, 0xAC100000, 0x00000001},
{0x92F00000, 0xAC000000, 0x00000001},
{0x93000000, 0xABF00000, 0x00000001},
{0x93100000, 0xABE00000, 0x00000001},
{0x93200000, 0xAB800000, 0x00000001},
{0x93300000, 0xAB500000, 0x00000001},
{0x93400000, 0xA8400000, 0x00000001},
{0x93500000, 0xA8700000, 0x00000001},
{0x93600000, 0xAB250000, 0x00000001},
{0x93700000, 0xAB240000, 0x00000001},
{0x93800000, 0xAB230000, 0x00000001},
{0x93900000, 0xAB220000, 0x00000001},
{0x93A00000, 0xAB210000, 0x00000001},
{0x93B00000, 0xAB200000, 0x00000001},
{0x93D00000, 0xB0000000, 0x00000001},
{0x93E00000, 0xB0200000, 0x00000001},
{0x93F00000, 0xB0300000, 0x00000001},
{0x94000000, 0xB0400000, 0x00000001},
{0x94100000, 0xB0500000, 0x00000001},
{0x94200000, 0xB0600000, 0x00000001},
{0x94300000, 0xB1000000, 0x0000000D},
{0x95000000, 0xB8000000, 0x00000001},
{0x95100000, 0xB8100000, 0x00000001},
{0x95200000, 0xB8200000, 0x00000001},
{0x95300000, 0xC0000000, 0x00000001},
{0x95400000, 0xC0100000, 0x00000001},
{0x95500000, 0xC0200000, 0x00000001},
{0x95600000, 0x88000000, 0x00000001},
{0x95700000, 0x89000000, 0x00000001},
{0x95800000, 0x8A000000, 0x00000001},
{0x95900000, 0x8B000000, 0x00000001},
{0x95A00000, 0x8C000000, 0x00000001},
{0x95B00000, 0x88000000, 0x00000001},
{0x95C00000, 0xA1000000, 0x00000001},
{0x95D00000, 0xA1100000, 0x00000001},
{0x95E00000, 0xA8000000, 0x00000001},
{0x95F00000, 0xA0F00000, 0x00000001},
{0x96000000, 0xB8300000, 0x00000001},
{0x96100000, 0xABD00000, 0x00000001},
{0x96200000, 0xABC00000, 0x00000001},
{0x96300000, 0xA8A00000, 0x00000001},
{0x96400000, 0xA8800000, 0x00000001},
{0x96500000, 0xAB600000, 0x00000001},
{0x96600000, 0xA8900000, 0x00000001},
{0x96700000, 0xAC100000, 0x00000001},
{0x96800000, 0xAD900000, 0x00000001},
{0x96900000, 0xA8300000, 0x00000001},
{0x00000000, 0x00000000, 0x00000000},
{0x00000000, 0x00000000, 0x00000000},

FastBoot Mode Commands

When the device is started in FastBoot mode (hold down the Optical Joystick (OJ) button, press and release the power button, then release the OJ when the menu is displayed) or FASTBOOT is chosen from the HBoot menu, it will expect to receive commands over the serial USB link using the fast-boot protocol. This is defined in the Android Software Development Kit (SDK).

A fastboot program is built when an Android system build is done for the host PC. The program has a set of command line options and parameters that cause it to send commands to the device using the 'fast-boot' protocol and report the responses. It is mainly used by developers to send updated firmware image files for hboot, boot, recovery and system to the device or to issue other commands.

It is important to know that the fastboot tool on the host PC may not know all the commands that the device will respond to. Where analysis of the hboot image reveals commands that the fastboot tool doesn't know, the source-code of fastboot would need to be modified and a custom build done.

The well-known fastboot commands are:


  update <filename>                        reflash device from
  flashall                                 flash boot + recovery + system
  flash <partition> [ <filename> ]         write a file to a flash partition
  erase <partition>                        erase a flash partition
  getvar <variable>                        display a bootloader variable
  boot <kernel> [ <ramdisk> ]              download and boot kernel
  flash:raw boot <kernel> [ <ramdisk> ]    create bootimage and flash it
  devices                                  list all connected devices
  reboot                                   reboot device normally
  reboot-bootloader                        reboot device into bootloader

To provide for device-specific commands the protocol provides the "oem" command which takes a sub-command that is device specific. To discover what "oem" commands the device supports do:

$ fastboot oem ?

... INFOcommand list

Most of these are the same commands as are accessible from an HBoot remote terminal. Further commands are available if the device has S-OFF.

In addition, the Vision supports the following non-oem commands that the fastboot tool may not be able to handle without modification:

signature <filename>                       send a 256 byte SHA signature to be used to verify files
download <length> <filename>               length is 8 hex characters which will be converted to a 32-bit long value

Enquiries can be sent; for example:

$ fastboot getvar all

INFOversion: 0.5
INFOversion-bootloader: 0.85.0005
INFOversion-cpld: None
INFOversion-microp: 0425
INFOversion-main: 1.34.405.5
INFOserialno: HT0AFRT01426
INFOimei: 352212040039153
INFOproduct: vision
INFOplatform: HBOOT-7230
INFOmodelid: PC1011000
INFOcidnum: HTC__001
INFObattery-status: good
INFObattery-voltage: 3825mV
INFOpartition-layout: Generic
INFOsecurity: on
INFObuild-mode: SHIP
INFOcommitno-bootloader: dirty-9ddda97c
INFOhbootpreupdate: 11
INFOgencheckpt: 0
all: Done!

HBoot Shell Commands

(recovered from browser cache thanks to Hymie after being deleted accidentally from the wiki)

battcheck <param1>                                              // battery check
bdaddress <param1>:<param2>:<param3>:<param4>:<param5>:<param6> // set bluetooth address (the bluetooth MAC)
bkflash2emmc                                                    // ?
btrouter                                                        // USB blue-tooth router?
emapiCountryID                                                  // display country signature and ID
emapiCrsuprs <channel>                                          // set carrier suppression mode (channel is 1-14 or 0 to stop test)
emapiRateset "default" | "all" | <arbitrary rateset>            // determine the WiFi rates to use
emapiReadCal                                                    // read the WiFi calibration configuration
emapiSetDefCal                                                  // set default WiFi calibration
emapiWlanMac                                                    // display the WLAN MAC address and where it is stored in non-volatile storage
emptypagecheck                                                  // check empty pages (of what - memory, cache?)
erase <partition_name>                                          // erase data in <partition_name>
erasebcid                                                       // erase back-up CID
eraseconfig <param1>                                            // erase the config fields
erasesd ?                                                       // erase micro SD-card
eraseWifiFlash                                                  // erase WiFi flash memory
fmrouter                                                        // modem AT command shell to switch audio path (for FM radio) and start bt_router
fmtx                                                            // FM transmitter ?
gotofastboot                                                    // switch to fastboot mode
gotohboot                                                       // switch to hboot mode
heap                                                            // report heap memory usage
heaptable                                                       // display the Free and Allocated heap tables
imgcrc                                                          // calculate checksums on hboot, recovery, boot and system partitions
jump                                                            // immediately continue booting the device into the operating system
keytest                                                         // enter key-test mode: displays names of keys being pressed. Exits after 5 presses.
listpartition                                                   // list partition names
partition_test <name> [auto_mark_bad_flag]                      // test a partition
powerdown                                                       // immediately powers down the device
ram_test <start address> <length> <count> <mode>                // test RAM with or without cache
rbchk [partition | block]                                       // read bad-block (artition = <all|recovery|boot|system|cache|userdata>, block id in HEX format)
rebootRUU                                                       // immediately reboot in Remote Update Utility mode (boots with white-on-black HTC logo and waits for update)
readbcid                                                        // read back-up CID
readconfig                                                      // read the config fields
readmbserialno                                                  // read main-board serial number
readserialno                                                    // read device serial number
readsku                                                         // read the SKU fields (PCBID and others)
resetautoimage                                                  // ?
resetpreferdiag                                                 // ?
reset                                                           // immediately reset the device (reboots)
resetuP                                                         // reset microP ?
rflash <param1>                                                 // read NAND flash
rMfgTp                                                          // read manufacturers test points
savefb2sd <file name> [main|ruu]                                // save frame buffer to SD-card
savemem2sd <memory offset> <length> <file name>                 // save memory region to SD-card
saveprt2sd <partition name> <-n> <file name> <-a>               // save partition to SD-card
sdtest                                                          // test the micro SD-card
task <task number>
tflash                                                          // test NAND flash
tick                                                            // report clock tick
usbspeed ?
usbtestmode <value>
wMfgTp <param1> <...>                                           // write manufacturer test points 
writebcid <param1>                                              // write back-up CID
writeconfig ?
writemid <param1>                                               // write model ID
writeserialno <param1>                                          // write serial number
writesku <param1> <param2>                                      // write SKU field with value 

Broadcom Radio

There are six separate instances in the data sections of HBoot of configuration strings for the  Broadcom BCM4329 which is described as "Low-Power 802.11n with Bluetooth® 2.1 + EDR and FM (Tx and Rx)". That last bit of information - FM transmitter - confirms what I'd previously suspected and hoped, that the device does indeed have the capability to transmit a low-power FM signal. Let's hope that when we get that functionality working with kernel plus user-space code there is also an antenna connected to the BCM4329!

This appears to suggest the HBoot may well have some provision for operating over WiFi or Bluetooth since it is hard to imagine why the boot-loader would get involved in configuring a device otherwise.

This is the first configuration set:

RAW1=80 32 fe 21 02 0c 00 22 2a 01 01 00 00 c5 0 e6 00 00 00 00 00 40 00 00 ff ff 80 00 00 00 00 00 00 00 00 00 00 c8 00 00 00 00 00 00 00 00 00 00 00 00 00 ff 20 04 D0 2 29 43 21 02 0c 00 22 04 00 20 00 5A

And here are the differentials created against that for the following five configuration sets, beginning with set 2:

diff -NU0 hboot01.txt hboot02.txt
--- hboot01.txt	2010-10-24 05:29:54.000000000 +0100
+++ hboot02.txt	2010-10-24 05:30:05.000000000 +0100
@@ -6 +6 @@
@@ -12,3 +12,3 @@
@@ -16,7 +16,7 @@
@@ -27,0 +28 @@

Set 3:

diff -NU0 hboot01.txt hboot03.txt
--- hboot01.txt	2010-10-24 05:29:54.000000000 +0100
+++ hboot03.txt	2010-10-24 05:30:19.000000000 +0100
@@ -6 +6 @@
@@ -12,3 +12,3 @@
@@ -16,7 +16,7 @@
@@ -27,0 +28 @@

Set 4:

diff -NU0 hboot01.txt hboot04.txt
--- hboot01.txt	2010-10-24 05:29:54.000000000 +0100
+++ hboot04.txt	2010-10-24 05:30:29.000000000 +0100
@@ -6 +6 @@
@@ -12,3 +12,3 @@
@@ -16,7 +16,8 @@

Set 5:

diff -NU0 hboot01.txt hboot05.txt
--- hboot01.txt	2010-10-24 05:29:54.000000000 +0100
+++ hboot05.txt	2010-10-24 05:30:40.000000000 +0100
@@ -6 +6 @@
@@ -12,3 +12,3 @@
@@ -16,7 +16,8 @@

Set 6:

diff -NU0 hboot01.txt hboot06.txt
--- hboot01.txt	2010-10-24 05:29:54.000000000 +0100
+++ hboot06.txt	2010-10-24 05:29:18.000000000 +0100
@@ -4,3 +4,3 @@
@@ -10 +10 @@
@@ -12,3 +12,3 @@
@@ -16,4 +16,25 @@
@@ -23,0 +45,4 @@
@@ -26,2 +50,0 @@
@@ -29 +52,2 @@
@@ -33,3 +56,0 @@
-RAW1=80 32 fe 21 02 0c 00 22 2a 01 01 00 00 c5 0 e6 00 00 00 00 00 40 00 00 ff ff 80 00 00 00 00 00 00 00 00 00 00 c8 00 00 00 00 00 00 00 00 00 00 00 00 00 ff 20 04 D0 2 29 43 21 02 0c 00 22 04 00 20 00 5A


A list of partition name strings:

01 BOOTLOADER    hboot
02 RADIO_V2      radio
03 MFG_DIAG      mfg
04 SPLASH1       sp1
05 SPLASH2       sp2
06 SP_CUSTOM     spcustom
07 SP_CHG        spchg
08 RECOVERY      recovery
09 BOOT          boot
10 SYSTEM        system
11 SYSTEM        dzsystem
12 OPL           opl
13 USERDATA      userdata
14 USERDATA      dzdata
15 MERGEMFG      merge_mfg
16 CPLD          cpld            // '''complex programmable logic device'''
17 MICROP        microp
18 RADIO_NV      nv
19 RADIO_CUST    rcdata
20 TP_MELFAS     tp-melfas
21 TP_ATMEL      tp_atmel
22 TP            tp
23 TOUCHSCREEN   touchscreen
24 RAW_TOUCH1    x_touchscreen
25 RAW_TOUCH2    d_touchscreen
26 XLOADER       xloader
27 WIFI          wifi
28 CMMB          cmmb
29 MDM9K         mdm9k
30 MDMNV         mdmnv