For the complete list of my articles on Android devices and software, including analysis of devices and system firmware, lists of external resources and tools, and How-To instructions, check the front page of this wiki under the Android heading.
Other devices are code-named:
- "Dynasty" (PG3010000)
- "Glacier" (PD1510000), possibly the T-Mobile myTouch 4G
- "Spade" (PD9810000) a.k.a "Ace" a.k.a "Desire HD"
- Current Status
- Vision Virgin (or, contents and photo tear-down of a U.K. Desire Z)
- Description of the device boot process and terminal commands
- eMMC Partitioning (and the partitioning bug)
- HBoot Analysis
- Radio Analysis
- Linux Analysis
- How to cross-compile ARM Linux kernel on Intel IA32
- Understanding eMMC User Capacity (diagram and explanation of the 4GB eMMC reporting 2.10GiB size)
- Extract ROM Image from Windows RUU Exe on Linux
- Exploring ROM images on a development PC
- Hacking Notes
- Miscellaneous Tools
- To Do List
11th November: Figured out how to use the proc comm RPC interface and the commands and parameters that are available. These include the commands to disable HTC security measures on the radio side (S-OFF). Because proc comm common to all devices based on Qualcomm AMSS I've put the Proc Comm explanatory article outside the Vision-specific articles. I'll be adding more to it as I progress. From what I see it appears we can issue the commands directly from Linux. I need to analyse it more and do some experiments but it appears that a pretty simple Linux user-space application could issue the commands.
9th November: Added information on the radio AT commands discovered on 7th November to the Radio Analysis page.
8th November: Began work on an article describing the Qualcomm MSM chip-set design and function.
7th November: Tested the HTC Sense "Fast Boot" mode. Updated the Boot Process page. After receiving the Desire Z yesterday and confirming that my analysis and understanding of the radio and hboot matches the reality of how they operate, and having a little excitement with S-OFF being tantalizing close, I'm back to finishing off my analysis tools. Scott's struggles to understand REXX/AMSS (the radio side) have prodded me to rework the tools to be more generic so they are useful for more than analysing hboot images. Needs a bit more strategic planning and forethought but the benefits in the future to a wide range of researchers explorers and hackers should be worthwhile.
7th November: Made some significant progress. Identified the command to issue to achieve S-OFF (security off). Can talk to the radio modem and issue commands to query and set various SIMLOCK values. See the HBoot Analysis for all the details.
6th November: Adding the Vision Virgin page, a photo tear-down guide of a U.K. Desire Z, and documenting device contents as shipped before the user touches it.
5th November: Taking a break from directly analysing hboot to work on my Linux tools that do the semi-automatic decomposition of the binary. Reduces several days manual analysis and fix-up work to a few minutes on any hboot image.
3rd November: Several developments with HBoot and bootloader-ap. Discovered eMMC partitions data-structures. Completed libc analysis.
30th October: Several developments with HBoot and bootloader-ap. Documented the HBoot page tables.
28th October: Added information on the reboot process and OEM codes. Adam (teferi) did some tests and managed to wipe his device using one, so beware! Added a new To Do list page to keep track of pending tasks.
27th October: I found something I'd previously missed about the Desire Z today: two videos demonstrating a new feature of HTC Sense they call "fast boot". It means that instead of taking close to a minute to have the device ready to use from power-on it is ready in around five seconds. I've documented it on the Boot Process page. Basically it is a suspend-to-RAM with most device power turned off. What could be important about this, in conjunction with our new-found ability to patch memory, is that it most likely turns off power to the eMMC device. The implication of that is that on resume from suspend the eMMC write-protect will/should be disabled. If a fast-boot restart is handled by hboot then there's a possibility that we can patch hboot code in RAM before shutting the device off so that on restart hboot doesn't write-protect the EMMC. That opens the way to directly write custom updates to the eMMC partitions. Right now this is all my own speculation, but it offers one more potential path to installing customised firmware. Bad news for T-Mobile G2 owners though - this does sound like it is HTC Sense specific so there's no guarantee the functionality exists in G2s for us to take advantage of. There appears to be support for this apps-processor suspend in arch/arm/mach-msm/pm.c::msm_pm_power_collapse_standalone().
27th October: I've added a new page to the wiki: Exploring ROM images on a development PC. Last thing yesterday we discovered the Vision Linux kernel is built with the devmem device included. This means that physical memory is directly accessible from user-space via the /dev/mem device. Adam (teferi) was able to dump memory using dd and confirmed the hboot image starts at physical address 0x00000000 (it is loaded to 0x8D000000 virtual). I now have a dump of the state of hboot's variables which will assist greatly in understanding its state. Scott confirmed /dev/mem can be written to so we now have user-space access to physical memory to patch hboot or any other structures. This opens up a new avenue where we can use a simple construct to jump out of the Linux kernel and back into hboot and test whether it can correctly function. If so, we have a mechanism to both patch code in memory and then execute it, which may provide a way to have hboot accept a non-signed image. This doesn't address whether hboot can unlock the eMMC partitions at that stage; that is something else to discover.
26th October: Early hours, was looking at the contents of eMMC partitions 13 and 14 which appear encrypted. They have identical 16-byte headers as well as identical 32-byte (256 bit) structures at the end of the images which could be public keys. Wrote a binary-match utility to quickly search for a binary needle in a binary haystack. Used it to check whether the same patterns appear in any of the engineering firmware image files. More details and the source code is available from the Hacking Tools page.
25th October: bogged down figuring out a single function, but is important to understand since it is called from do_update_zip(). It does a lot of adjusting for 4-byte boundaries and copying bytes about and is confusing the heck out of me! I originally thought it was a unicode handling function but now I'm doubtful. Early hours of the day I was tracing this functionality after a user was able to cause a "Lack of heap!" error trying to update the phone from an engineering RUU and I'm still stuck in that analysis.
24th October: completed identification of hboot functions from embedded clues, and looking at source-code of external libraries and projects used by hboot. 384 functions identified out of 1,308 recognised. List is attached to the HBoot Analysis page.
24th October: investigations continuing. Aiming to complete reverse-engineering of HBoot this weekend. Likelihood: 50%. More likely to be the 27th before it is complete, the more done the harder the remaining parts are to deduce.
23rd October: "hboot" analysis continues to identify libc functions, boot-loader functions, and interesting data structures. Discovered several configuration settings for the Broadcom 4329 wireless chip, and also began digging into msm_mpu_emmc_protect(). Added latest list of functions as an attachment.
22nd October: "radio" analysis 1st pass 40% complete (setting up pointers to string constants). Moved back to "hboot", 5th pass (identifying functions that match source-code), identified cprintf, __xprintf, xputc, xputs, alloc, alloc_page_aligned, investigating whether init.S code maps to binary.
21st October: "radio" analysis 1st pass 33% complete (setting up pointers to string constants).
20th October: "radio" analysis 1st pass 10% complete (setting up pointers to string constants).
19th October: "hboot" analysis 3rd pass complete (fixing up function names from clues in associated string constants).
18th October: "hboot" analysis 1st pass complete (setting up pointers to string constants). 2nd pass complete (identifying all .text (code) sections).
17th Oct 2010: Analysis of Linux kernel eMMC capacity report ("2.10 GiB" when device is a 4GB SanDisk SEM04G part). Confirmed capacity calculation code is correct. Confirmed device name comes directly from the device's CID register.
Beginning in mid October 2010, after release of the T-Mobile G2 in the USA and just prior to the release of the same device as the HTC Desire Z in the rest of the world, I became interested in the capabilities of the Vision. After customising the boot-loader and operating system on the HTC Desire I was interested in doing the same on the HTC Desire Z.
It turned out the regular custom firmware creators such as Cyanogen hadn't yet managed a custom firmware update and there was an effort in progress by a handful of device owners to discover how to achieve it. This effort centred around the XDA forums and wiki and the Freenode IRC channels #G2ROOT, #G2-DEV and #g2-chat. I observed several determined efforts that determined that the firmware storage device (eMMC NAND flash) took advantage of the device's temporary write-protect functionality, and to find ways to unlock it or in some other way get a custom boot-loader and/or operating system image written to the eMMC.
By the 20th October the effort had largely petered out as lines of enquiry dried up.
In the meantime I had been familiarising myself with the Vision GNU/Linux kernel source-code released by HTC (vision-2.6.32-g814e0a1). In addition I had begun studying the binary images from the device's "radio" and "hboot" partitions with a view to completely understanding what they do, how, and why. The hope being that by being able to recreate the images as psuedo source-code would allow insights into new methods of replacing the "hboot" boot-loader, "boot" GNU/Linux kernel and "system" Android operating system.
Open Source Code and Tool Resources
- Android Open Source-code Project (AOSP) repository
- ARM Linux Kernel (good source of early postings of new chip-set support and understanding new registers, etc.)
- HTC-Linux MSM 2.6.32
- Qualcomm Surf git repository. Code for the secondary boot-loader and building .nbh files (Useful for insights into HTC's HBoot since HTC base their code on the Qualcomm code). See especially boot/tools/mkbootloader and scripts/. The scripts give insights into the combined ARM9/ARM11 aspect.
- Android project open-source boot-loader (active project, supports latest devices)
- Android project legacy boot-loader (inactive project, hboot base)
- PTXdist - Reproducable Embedded Linux Systems. Used by the Qualcomm AMSS build tools to create the root file system.
- L4 micro-kernel
- L4A Pistachio embedded micro-kernel
- Iguana operating system
- Re-exec Hboot from Linux with 2ndboot-ng
Non-Free Code and Tool Resources
- ARM Real View Compilation Tools. Required for AMSS (Qualcomm Advanced Mobile Subscriber Software) modem builds. Obtainable from ARM's connect silver web site. ARM's RVCT (4.0) for Symbian Foundation Community is free for application developers working in companies with less than 20 employees.
ARM CoreSight On-chip Trace & Debug Architecture (JTAG and Debugging)
Controlling the ARM9 L4/Iguana baseband boot-loader ( example session)
Installing kernel GNU Debugger (kgdb) on Android
Background Information and Resources
HTC Linux project (community project, not affiliated with HTC)
AMBA AXI on-chip interconnect
ARM9 radio (baseband) uses REX (Real-time EXecutive) operating system, which has an L4A Pistachio micro-kernel (see NICTA L4 Microkernel to be Utilised in Select QUALCOMM Chipset Solutions) and Iguana operating system.
Radio (baseband) operating-system (used on ARM9) Iguana
Microsoft BinFS Binary ROM Image File System
Microsoft RomImage.exe BinFS creator
FreeScale application note 4137 BinFS Implementors Guide
Embedded Multi-Media Card (eMMC) a.k.a Samsung moviNAND
U-boot Universal Boot Loader
Code Aurora Qualcomm Android Enablement Project
Android G1 Serial to USB Cable (example of creating a serial connection for accessing the radio's L4/Iguana menu at start-up)
Becker Hsieh HTC Linux kernel developer
Shyam Sundar Android Project Engineer for MSM7630 and MSM8655, Senior Staff Engineer / Manager at Qualcomm Innovation Center