wiki:Linux/Kernel/SimulateNandMtdDevice

Simulate NAND MTD Device

When working on embedded devices, especially Android-based smart-phones, tablets, and similar, it is sometimes useful to be able to create, modify, and examine binary images created for the device's NAND Flash storage. This is usually a Memory Technology Device (MTD).

YAFFS

Many Android devices use YAFFS/YAFFS2 (Yet Another Flash File System) for the recovery, system, and data partition images. YAFFS/YAFFS2 are friendly to the wear characteristics of Flash devices and implement - in the case of YAFFS2 - a fully sequential log-based file-system (A log-based file system means that every write to the device is to the next free location and any changes to existing files are implemented by markers that indicate patches to the original file, rather than over-writing or freeing the original location in a random-access process such as is used by ext2/3/4, vfat, and so on).

Building YAFFS

YAFFS is not part of the mainline Linux kernel yet, so it has to be built separately. Fortunately this a simple operation: fetch the code, configure it for multi-version support, and make:

git clone git://www.aleph1.co.uk/yaffs2
cd yaffs2
ln -s yportenv_multi.h yportenv.h
make

 make -C /lib/modules/2.6.38-8-generic/build M=/home/all/SourceCode/yaffs2 modules
 make[1]: Entering directory `/usr/src/linux-headers-2.6.38-8-generic'
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_mtdif.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_mtdif2_multi.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_mtdif1_multi.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_packedtags1.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_ecc.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_vfs_multi.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_guts.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_packedtags2.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_tagscompat.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_checkptrw.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_nand.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_nameval.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_allocator.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_bitmap.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_attribs.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_yaffs1.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_yaffs2.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_verify.o
  CC [M]  /home/all/SourceCode/yaffs2/yaffs_summary.o
  LD [M]  /home/all/SourceCode/yaffs2/yaffs2multi.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/all/SourceCode/yaffs2/yaffs2multi.mod.o
  LD [M]  /home/all/SourceCode/yaffs2/yaffs2multi.ko
 make[1]: Leaving directory `/usr/src/linux-headers-2.6.38-8-generic'

We now have the yaffs2multi.ko kernel module ready to install:

sudo make mi

 make -C /lib/modules/2.6.38-8-generic/build M=/home/all/SourceCode/yaffs2 modules_install
 make[1]: Entering directory `/usr/src/linux-headers-2.6.38-8-generic'
  INSTALL /home/all/SourceCode/yaffs2/yaffs2multi.ko
  DEPMOD  2.6.38-8-generic
 make[1]: Leaving directory `/usr/src/linux-headers-2.6.38-8-generic'

The module can now be loaded like any other module, and has parameters we might want to use:

modinfo yaffs2multi

 filename:       /lib/modules/2.6.38-8-generic/extra/yaffs2multi.ko
 license:        GPL
 author:         Charles Manning, Aleph One Ltd., 2002-2011
 description:    YAFFS2 - a NAND specific flash file system
 srcversion:     614E40AFECA0383D5D06FA4
 depends:        mtd
 vermagic:       2.6.38-8-generic SMP mod_unload modversions 686 
 parm:           yaffs_trace_mask:uint
 parm:           yaffs_wr_attempts:uint
 parm:           yaffs_auto_checkpoint:uint
 parm:           yaffs_gc_control:uint
 parm:           yaffs_bg_enable:uint


Mount YAFFS images

One thing that can't be done (yet) with YAFFS is something many Linux hackers are used to doing - mounting a file-system image on the loop file system:

sudo mount -t yaffs2 -o loop system.img
# this failed to work as expected

This silently fails to do what is expected. What we have to do is create a NAND/MTD device simulator in memory large enough to contain the image we wish to mount.

NAND Simulator

There is a technique, however, that can achieve the same end result - albeit not as cleanly. It depends upon the kernel having been built with the mtdblock, nandsim and other supporting modules. A typical Ubuntu kernel will have these modules available. This example creates a 256MB NAND device using 512-byte erase-blocks, copies the contents of the system.img YAFFS partition image into it, then mounts it so the file system can be accessed like any other.

For larger devices, the system may not have enough memory to adequately back the simulated device with RAM, therefore it is possible to tell nandsim to use a file as the backing store. I've done that here for a 256MB device:

sudo modprobe mtdblock
sudo modprobe nandsim first_id_byte=0x20 second_id_byte=0x71 cache_file=/tmp/nandsim.bin
sudo modprobe yaffs2multi

cat /proc/mtd

 dev:    size   erasesize  name
 mtd0: 10000000 00004000 "NAND simulator partition 0"

sudo dd if=system.img of=/dev/mtdblock0 

 463650+0 records in
 463650+0 records out
 237388800 bytes (237 MB) copied, 11.0737 s, 21.4 MB/s

mkdir -p mnt/system

sudo mount -t yaffs2 /dev/mtdblock0 mnt/system

ls -l system

 total 56
 drwxr-xr-x  2 tj tj 12288 2011-06-10 03:26 app
 drwxr-xr-x  2 tj tj  4096 2011-06-10 03:26 bin
 -rw-r--r--  1 tj tj  4598 2011-06-10 03:26 build.prop
 drwxr-xr-x  5 tj tj  4096 2011-06-10 03:26 customize
 drwxr-xr-x 11 tj tj  4096 2011-06-10 03:26 etc
 drwxr-xr-x  2 tj tj  4096 2011-06-10 03:26 fonts
 drwxr-xr-x  2 tj tj  4096 2011-06-10 03:26 framework
 drwxr-xr-x  6 tj tj  4096 2011-06-10 03:26 lib
 drwxr-xr-x  4 tj tj  4096 2011-06-10 03:26 media
 drwxr-xr-x  6 tj tj  4096 2011-06-10 03:26 usr
 drwxr-xr-x  2 tj tj  4096 2011-06-10 03:26 xbin

The size of the NAND device can be set by changing the ID bytes passed to the nandsim module on loading. I have these in a modprobe.conf file at /etc/modprobe.d/nandsim.conf:

# Simulate a 256MB 512-byte erase-block sized MTD device backed by a file on disk, not RAM
options nandsim first_id_byte=0x20 second_id_byte=0x71 cache_file=/tmp/nandsim.bin

A range of useful sizes can be set, and there is a YAFFS test script that details many of them in the YAFFS source-code in the script linux-tests/initnamdsim:

#!/bin/sh
modprobe mtd
modprobe mtdblock

case $1 in
	16MiB-512 )
		modprobe nandsim first_id_byte=0x20 second_id_byte=0x33
		;;
	32MiB-512 )
		modprobe nandsim first_id_byte=0x20 second_id_byte=0x35
		;;
	64MiB-512 )
		modprobe nandsim first_id_byte=0x20 second_id_byte=0x36
		;;
	128MiB-512 )
		modprobe nandsim first_id_byte=0x20 second_id_byte=0x78
		;;
	256MiB-512 )
		modprobe nandsim first_id_byte=0x20 second_id_byte=0x71
		;;
	64MiB-2048 )
		modprobe nandsim first_id_byte=0x20 second_id_byte=0xa2 third_id_byte=0x00 fourth_id_byte=0x15
		;;
	128MiB-2048 )
		modprobe nandsim first_id_byte=0xec second_id_byte=0xa1 third_id_byte=0x00 fourth_id_byte=0x15
		;;
	256MiB-2048 )
		modprobe nandsim first_id_byte=0x20 second_id_byte=0xaa third_id_byte=0x00 fourth_id_byte=0x15
		;;
	512MiB-2048 )
		modprobe nandsim first_id_byte=0x20 second_id_byte=0xac third_id_byte=0x00 fourth_id_byte=0x15
		;;
	1GiB-2048 )
		modprobe nandsim first_id_byte=0xec second_id_byte=0xd3 third_id_byte=0x51 fourth_id_byte=0x95
		;;
	* )
		echo "No nand config specified. Need one of:"
		echo " 16MiB-512, 32MiB-512, 64MiB-512, 128MiB-512, 256MiB-512, 64MiB-2048, 128MiB-2048, 256MiB-2048, 512MiB-2048, 1GiB-2048"
		return 1
esac

cat /proc/mtd