Replicant

h1. Replicant 4.0 Porting Guide

{{>toc}}

This guide is a step-by-step explanation of the process of porting a new device to Replicant 4.0.

h2. Overview

Porting a new device to Replicant is a long task, so make sure you're ready to go through all the steps mentioned below. While it's not technically hard (unless you have to write free software replacements yourself), the process itself takes time as many steps are involved:

* Checking that your devices meets [[Replicant40PortingGuide#Prerequisites|the required prerequisites]]

* Discovering the phone's hardware and the amount of non-free blobs involved

* Getting Replicant sources, reading all the relevant material about developing on Replicant

* Adding the device-specific repositories and files

* Setting up the build environment for the new device

* Building a recovery image

h2. Prerequisites

Before porting your device to Replicant, you must make sure it complies with the following requirements:

* The phone is already supported by CyanogenMod (CM) 9.x (or, worst-case scenario, by a non-official CyanogenMod 9.x port)

* The phone has a GSM modem: Replicant doesn't support CDMA phones (but you can add support for it if you're skilled)

* There is a way of installing another operating system, either through the bootloader or via recovery (this is likely if there is a CM port)

* The kernel is not signed: this means that the bootloader doesn't check the kernel's signature to match with the vendor's key to allow it to run

If your device fails to comply with one of these requirements, it won't be possible to port Replicant to it.

If you don't know about whether your device complies or not, you'll probably learn it along the way.

h2. Discovering the phone's hardware and associated blobs

h3. Finding the device's codenames

First of all, you'll have to find out the device's codename that was given by its manufacturer. "Wikipedia":http://www.wikipedia.org/ usually has that information on the device's article. For instance, the codename for the European version of the Nexus S given by Samsung is i9023. This codename will help in the process of getting informations about the device.

Then, a second codename (that can turn out the be the same as the previous one) is given to the device at Android-level. If your device is supported by CyanogenMod, you can find it out from the "CyanogenMod Wiki":http://wiki.cyanogenmod.org/ or on "CyanogenMod download page":http://get.cm/. For instance, the "Nexus S":http://wiki.cyanogenmod.org/w/Crespo_Info codename is: @crespo@.

h3. Investigating the hardware

It is useful to have a general idea of what kind of hardware is present in the phone. From the Wikipedia and CyanogenMod pages about the device, it's already possible to know what System on a Chip (SoC) it uses and a couple other details.

To learn more details, you can consider looking for a teardown of the device (for instance on "iFixit":http://www.ifixit.com/), that will reveal what chips are used on the device. Looking at the kernel defconfig for the device will also help a lot, you can also try to find the service manual for the device.

You can then compare that to the devices that are already supported in Replicant to get an idea of what will possibly work.

h3. Finding out if the device checks the kernel's signature

One very important step is to find out if the device is Tivoized: that means that even though the manufacturer releases the kernel source code for the device, the bootloader checks the kernel signature and will refuse to start it if it's not properly signed by the manufacturer. In other words, if you build the kernel yourself, the device will refuse to run it since it's not signed by the manufacturer. Since the Linux kernel is released under the GPLv2, there are no specific dispositions to counter Tivoization, and so porting the device to Replicant is pointless as it will require a prebuilt and signed kernel from the manufacturer.

This is not an easy information to find out, but the developers involved in the CyanogenMod port will probably know that information. It's a good idea to just ask them.

h3. Discovering the way of flashing the device

To install the future Replicant image on the device, you have to find out how the device can be flashed with a new operating system. The "CyanogenMod Wiki":http://wiki.cyanogenmod.org/ has install guides for the supported devices and you'll probably find install guides for non-official CM ports as well. It is very important to understand the flashing procedure as it will have to be documented on the Replicant wiki.

There are basically two ways of flashing a new operating system:

# Through the bootloader: a program has to send the images to the phone in bootloader mode. Make sure that program is free if your device supports flashing via bootloader.

# With recovery: a recovery image has to be installed instead of the current kernel so that at next reboot, recovery permits the installation of another operating system. Make sure this doesn't involve rooting the phone using non-free software.

h3. The non-free blobs

The key information to get before starting the port is the list of the non-free components that are required by CyanogenMod.

The easiest way to do this is to spot the device repository in "CyanogenMod repos":https://github.com/CyanogenMod/ and look for the @extract-files.sh@ or @proprietary-blobs.txt@ file on the @ics@ branch.

There is usually a link to the device repository from the "CyanogenMod Wiki":http://wiki.cyanogenmod.org/

For instance, the list of non-free components for the "Nexus S":https://github.com/CyanogenMod/android_device_samsung_crespo is "extract-files.sh":https://github.com/CyanogenMod/android_device_samsung_crespo/blob/ics/extract-files.sh

From that list, spot what is related to what hardware component (audio, camera, sensors, gps, modem, etc): that gives an idea of the amount of work required to add support for the phone.

h2. Getting started with Replicant development

In order to prepare everything for the Replicant port:

* Install CyanogenMod on the phone

* Install the [[BuildDependencies]]

* Get the sources: [[GettingReplicantSources]]

* Read the Replicant developer guide: [[DeveloperGuide]]

* Learn how to do debug: [[GettingLogs]] [[GDBDebugging]]

h2. Cloning the device files

Once your Replicant tree is ready, you can start adding the necessary repos for your device.

That means cloning the necessary repos in the right place. These repos are:

* A *device-specific repo*. On CyanogenMod, it is usually called: @android_device_vendor_device@.

* Sometimes one or more *common repo(s)*, usually called: @android_device_vendor_devices-common@.

Some devices don't need any common repo, but some do.

* A *kernel repo*. On CyanogenMod, it is usually called: @android_kernel_samsung_devices@.

The kernel repo can be shared across a family of devices (for instance, on kernel repo for Samsung Exynos, one for Samsung OMAP, etc).

You can find the device-specific repo from the device's page on the "CyanogenMod Wiki":http://wiki.cyanogenmod.org/. 

Make sure you check out the branches that match the CM 9.0 version (the branch may be called @ics@).

Once you have cloned the device-specific repo for your device and checked out the correct branch, refer to the @cm.dependencies@ file to find what repos are left to clone.

Clone these repos in the correct locations and remove the prefix (e.g. @android_device_samsung_crespo@ must be cloned in @device/samsung/@ and renamed to @crespo@).

If your cloned the kernel source for your device, it is likely that the kernel build is already integrated, so you can skip the next sections below.

h3. In case of a missing kernel repository

If the kernel repo is nowhere to be found (make sure you've asked the CyanogenMod team), you'll need to get the kernel source directly from the vendor, especially if your device is supported by a 3rd party CyanogenMod fork.

Keep in mind that the Linux kernel is GPLv2, so vendors have the legal obligation to release the modified kernel sources as soon as they sell you the device.

That means the kernel sources will be available online. Here are some websites where such releases are done:

* "Samsung Open Source Release Center":http://opensource.samsung.com/

For Samsung kernels. Search the device codename (e.g. I9000) and download the package called "Opensource Update" (e.g. GT-I9000_Opensource_GB_Update2.zip).

This will hold a kernel archive with all the sources and instructions on how to build it and which defconfig to use.

* "CodeAurora":https://www.codeaurora.org/contribute/projects/qaep

For MSM/QSD devices, and particularly HTC ones.

* "HTCdev":http://www.htcdev.com/devcenter/downloads

* "Motorola Sourceforge":http://sourceforge.net/motorola/

* "Sony Mobile Developer":http://developer.sonymobile.com/downloads/opensource/

* "LG OpenSource":http://www.lg.com/global/support/opensource/opensource.jsp

Once you have the kernel sources, read the instructions to find out which defconfig to use.

Since manufacturers usually don't release the git history along with the files, you'll need to recreate a git repo:

* Clone the mainline kernel in the same version as the Makefile from the sources you just obtained

* Remove the cloned files *except the .git directory*

* Move the manufacturer kernel tree at the place of the files you just removed

* Add all the files in git (@git add -A@) and commit (@git commit@) with a message explaining what you just imported (e.g. "GT-I9000 GB Opensource Update 2")

Now that you have a git repo, you can move it to the Replicant code tree, under the name: @kernel/vendor/devices@ (e.g. @kernel/samsung/aries@).

Make sure to make the @devices@ name match the @devices@ in @android_device_vendor_devices-common@ if the kernel is shared across these devices or to match the @device@ in @android_device_vendor_device@.

h3. In case of a prebuilt kernel

Some devices are still using a prebuilt kernel. Even though the CyanogenMod team is trying to avoid that, it remains in many repos.

For such devices, you will need to remove the prebuilt binaries and the instructions to copy the prebuilt kernel and its modules.

In the device repository (@device/vendor/device@) and common repository for your device (if any), remove the prebuilt kernel and modules (usually called @kernel@ and @module.ko@ (replace module with the name of a module) or a @modules@ directory).

Remove the instructions to copy these prebuilts on the makefiles. Remove instructions such as:

<pre>

PRODUCT_COPY_FILES += \

    $(LOCAL_KERNEL):kernel

LOCAL_KERNEL := $(LOCAL_PATH)/kernel

</pre>

and anything regarding @TARGET_PREBUILT_KERNEL@ as well as the instructions to copy the prebuilt modules.

The @BoardConfig.mk@ (or @BoardConfigCommon.mk@ in the common directory for your device) will most likely hold a line like:

<pre>

TARGET_PREBUILT_KERNEL := device/samsung/p5/kernel

</pre>

you must remove this line.

Now that the device repository has no prebuilt instructions, you can add the instructions to build the kernel. In the @BoardConfig.mk@ file, add the following lines:

<pre>

TARGET_KERNEL_SOURCE := kernel/samsung/p3

TARGET_KERNEL_CONFIG := samsung_p5_defconfig

</pre>

and make sure to replace the location and defconfig by the correct values for your devices (being the location of the device kernel tree and the appropriate defconfig).

h3. Building the correct kernel image format

There are different types of kernel images:

* Android image: that is a zImage, without a separate initramfs

* zImage: that is a zImage, with a built-in initramfs

* uImage: that is an image for the u-boot bootloader, with a built-in initramfs

You need to find out which type of kernel image your device uses. Asking people who know about that is the best idea.

h4. Android image

This is the easiest case to handle: just make sure the @CONFIG_INITRAMFS_SOURCE@ option in the kernel defonfig is left blank or undefined:

<pre>

CONFIG_INITRAMFS_SOURCE=""

</pre>

h4. zImage with built-in initramfs

Building a zImage with a built-in initramfs requires the following steps:

In the kernel defconfig, define the @CONFIG_INITRAMFS_SOURCE@ option that way:

<pre>

CONFIG_INITRAMFS_SOURCE="../../root"

</pre>

Once this is done, duplicate the defconfig and add the @_recovery@ prefix before the @_defconfig@ ending (e.g. @herring_recovery_defconfig@), edit that file and replace @CONFIG_INITRAMFS_SOURCE@ with:

<pre>

CONFIG_INITRAMFS_SOURCE="../../recovery/root"

</pre>

Back to the device repository, edit the @BoardConfig.mk@ file and add the following line:

<pre>

TARGET_KERNEL_RECOVERY_CONFIG := samsung_p5_recovery_defconfig

</pre>

and make sure to replace the defconfig by the appropriate defconfig you just cloned (the one with the @_recovery_defconfig@ ending).

Still in the device repository, create a @bootimg.mk@ file containing the following:

<pre>

LOCAL_PATH := $(call my-dir)

$(INSTALLED_BOOTIMAGE_TARGET): $(INSTALLED_KERNEL_TARGET)

	$(ACP) $(INSTALLED_KERNEL_TARGET) $@

$(INSTALLED_RECOVERYIMAGE_TARGET): $(INSTALLED_RECOVERY_KERNEL_TARGET)

	$(ACP) $(INSTALLED_RECOVERY_KERNEL_TARGET) $@

</pre>

Edit the @BoardConfig.mk@ file and add the following line:

<pre>

BOARD_CUSTOM_BOOTIMG_MK := device/vendor/device/bootimg.mk

</pre>

and make sure to replace @device/vendor/device/@ to the correct path to your device's repository.

h4. uImage with built-in initramfs

Follow the previous instructions (*zImage with built-in initramfs*) and set the @BOARD_USES_UBOOT@ variable in the @BoardConfig.mk@ file:

<pre>

BOARD_USES_UBOOT := true

</pre>

h2. Setting up the build environment

Now that the repos are cloned, you need to modify some makefiles to cope with Replicant paths.

In the device repository (@device/vendor/device@), modify the file called @cm.mk@ and replace the @vendor/cm/@ occurrences by @vendor/replicant/@. Other makefiles may need that as well (in any case, build will fail very early if you missed one). In that same @cm.mk@ file, change the PRODUCT_NAME variable by replacing the @cm@ prefix with @replicant@ (e.g. change PRODUCT_NAME := cm_crespo  to PRODUCT_NAME := replicant_crespo).

Now that your device files are ready, you can declare a new build target: these are held in @vendor/replicant/jenkins-build-targets@.

Modify that file and add a line (at the end) with the PRODUCT_NAME you set and the @-eng@ suffix (e.g. @replicant_crespo-eng@).

From now on, everything should be ready to start a build. To check for errors or missed occurrences, start a terminal in the Replicant tree root and lunch:

<pre>

source build/envsetup.sh

lunch replicant_device-eng

</pre>

Adapt replicant_device-eng from what you added to the @jenkins-build-target@ (e.g. @replicant_crespo-eng@).

If an error occurs, it will explicitly report it and you'll need to fix it before doing anything.

If everything works correctly, you should see something like:

<pre>

============================================

PLATFORM_VERSION_CODENAME=REL

PLATFORM_VERSION=4.0.4

TARGET_PRODUCT=replicant_crespo

TARGET_BUILD_VARIANT=eng

TARGET_BUILD_TYPE=release

TARGET_BUILD_APPS=

TARGET_ARCH=arm

TARGET_ARCH_VARIANT=armv7-a

HOST_ARCH=x86

HOST_OS=linux

HOST_BUILD_TYPE=release

BUILD_ID=IMM76L

============================================

</pre>

*You must repeat these steps everytime before building anything on a freshly-opened terminal.*

Remember:

<pre>

source build/envsetup.sh

lunch replicant_device-eng

</pre>

(make sure to replace device by your device's product name).

h2. Building a recovery image

Now that everything is set-up, you can build the first image to test on your device: the recovery image.

The build target is *recoveryimage*, so all you have to do is:

<pre>

make -j9 recoveryimage

</pre>

This should trigger the kernel build and the recovery initramfs build and in the end, produce the @out/target/product/device/recovery.img@ file.

Once your image is built (it takes some time), flash it to the recovery partition of your device (if any). It's a good idea to look at the CyanogenMod installation guide to find out how to install that recovery image.

There is usually also a key combination to hold to boot directly to recovery: hopefully, your recovery image will start.

h2. Building the system

It is time to build a complete set of Replicant images. This includes at least the system and kernel images. Depending on the installation method, an userdata image might be needed too.

h3. Building the kernel

Let's start by building the boot image, that is both the kernel and the Android initramfs. The build target is *bootimage*:

<pre>

make -j9 bootimage

</pre>

In the end, the @out/target/product/device/boot.img@ file will be produced.

h3. Setting up the system image format

It is time for you to take a good look at the installation process. Mainly, about whether the images will be flashed using the bootloader or recovery.

Since CyanogenMod uses the zip installation method, that we do not want to use, you're on your own here.

h4. Finding the appropriate filesystem

It will be easy to find out the filesystem for the different partitions if the device already runs CyanogenMod:

<pre>

$ adb shell mount

rootfs / rootfs ro,relatime 0 0

tmpfs /dev tmpfs rw,nosuid,relatime,mode=755 0 0

devpts /dev/pts devpts rw,relatime,mode=600 0 0

proc /proc proc rw,relatime 0 0

sysfs /sys sysfs rw,relatime 0 0

none /acct cgroup rw,relatime,cpuacct 0 0

tmpfs /mnt/asec tmpfs rw,relatime,mode=755,gid=1000 0 0

tmpfs /mnt/obb tmpfs rw,relatime,mode=755,gid=1000 0 0

none /dev/cpuctl cgroup rw,relatime,cpu 0 0

/dev/block/mtdblock2 /system yaffs2 ro,relatime 0 0

/dev/block/mtdblock3 /cache yaffs2 rw,nosuid,nodev,relatime 0 0

/dev/block/mtdblock5 /radio yaffs2 rw,relatime 0 0

/dev/block/mmcblk0p2 /data ext4 rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered,noauto_da_alloc 0 0

/dev/block/mtdblock6 /datadata yaffs2 rw,relatime 0 0

/dev/block/mtdblock4 /efs yaffs2 rw,relatime 0 0

/sys/kernel/debug /sys/kernel/debug debugfs rw,relatime 0 0

/dev/block/vold/179:1 /mnt/sdcard vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0

/dev/block/vold/179:1 /mnt/secure/asec vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0

tmpfs /mnt/sdcard/.android_secure tmpfs ro,relatime,size=0k,mode=000 0 0

</pre>

So we can deduce that system is yaffs2 and data is ext4. Don't bother about the other partitions and mount-points, only */system* and */data* matter for now.

h4. Changing the images format for bootloader installation

You have to modify the @BoardConfig.mk@ file on the main device repository (it might be delegated to @BoardConfigCommon.mk@ on the common repos).

To build ext4 system and userdata images, make sure you have:

<pre>

TARGET_USERIMAGES_USE_EXT4 := true

</pre>

To build yaffs2 system and userdata images, make sure you have:

<pre>

TARGET_USERIMAGES_USE_EXT4 := false

</pre>

h4. Changing the images format for recovery installation

MORE TO COME

h3. Building the system image

Building the system is the longest task. The build target is *systemimage*:

<pre>

make -j9 systemimage

</pre>

You might encounter build errors due to the lack of non-free libs. You'll need to find clean workarounds for that. Removing options from @BoardConfig.mk@ can help solve the situation.

For instance, the following error:

<pre>

make: *** No rule to make target `out/target/product/i9300/obj/lib/libTVOut.so', needed by `out/target/product/i9300/obj/EXECUTABLES/mediaserver_intermediates/LINKED/mediaserver'.  Stop.

</pre>

Was solved by turning @BOARD_USE_SECTVOUT@ to false:

<pre>

BOARD_USE_SECTVOUT := false

</pre>