h1. Build a Native Developement Kit (NDK) with Replicant 4.2

h2. Principle of operation

The [[ReplicantSourceCode|Android 4.2 source code]] ships a @ndk@ directory with NDK build scripts.
These tools download various upstream imports from https://android.googlesource.com/toolchain/XXX/ with optional commit date (no branches or tags), apply patches, and build them.

Note: as of 2015-07, the upcoming Android version moved to a different build system, and will populate $topdir/toolchain/ through @repo@ instead of invoking git in a temporary directory (i.e. ditched @download-toolchain-sources.sh@).

h2. Ubuntu 12.04 build environment with LXC

Replicant 4.2 is based on CyanogenMod 10.1 which is based on AOSP 4.2.2 (2013-02).
The build environment used by NDK release managers should be Ubuntu LTS 12.04.
We recreate it through LXC.  If you know how to use Triskel 6 instead, please contribute the instructions.

<pre>
lxc-create -n replicant -t download -- -d ubuntu -r precise -a amd64
# Note: or Triskel 6, if you know how to configure LXC/debootstrap for it
lxc-start -n replicant -d
lxc-attach -n replicant

# clean-up non-free sources
sed -i -e 's/ restricted//' -e 's/ multiverse//' /etc/apt/sources.list
apt-get update

# prepare build user
apt-get install openssh-server
useradd replicant --shell /bin/bash --create-home
mkdir -p -m 700 ~replicant/.ssh
cat <<EOF >> ~replicant/.ssh/authorized_keys
your public key
EOF
chown -R replicant: ~replicant
</pre>

Note: compiling from Debian 8 will yield errors from texinfo (doc in binutils) and Perl (POD in LLVM) due to more recent, stricter versions of these.

h2. Build dependencies

<pre>
# Base Android build dependencies (from https://source.android.com/source/initializing.html, section "Installing required packages (Ubuntu 12.04)")
# Dropping mingw to skip windows builds for now.
# Installing g++-multilib first otherwise apt-get complains.
apt-get install build-essential g++-multilib
apt-get install git gnupg flex bison gperf \
  zip curl libc6-dev libncurses5-dev:i386 x11proto-core-dev \
  libx11-dev:i386 libreadline6-dev:i386 libgl1-mesa-glx:i386 \
  libgl1-mesa-dev \
  python-markdown libxml2-utils xsltproc zlib1g-dev:i386

# Additional NDK build dependencies (completed from https://android.googlesource.com/platform/ndk/+/master/README.md)
apt-get install curl texinfo bison flex libtool pbzip2 groff

# Make 'bash' the default shell otherwise 'build-gabi++.sh' will fail
dpkg-reconfigure dash
</pre>

h2. Preparing the sources

Based on [[ReplicantSourceCode]] with additional configuration.

<pre>
mkdir ~/tools/
cd ~/tools/
wget http://commondatastorage.googleapis.com/git-repo-downloads/repo
chmod a+x repo
cd ~

cat <<EOF >> ~/.bashrc
export USE_CCACHE=1
EOF

# Avoid prompts
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git config --global color.ui true

mkdir replicant-4.2/
cd replicant-4.2/
../tools/repo init -u git://git.replicant.us/manifest.git -b replicant-4.2
../tools/repo sync -j4 
# > 1h, 27GB (inc. 17GB .repo)
</pre>

h2. Determining the checkout date

Since there's no NDK release tag for the AOSP @toolchain/@ checkouts, we need to find the right date.

@cd ndk/ && git log@ says the last commit is 2012-10-05, so it's around NDK r8b (2012-07) and NDK r8c (2012-11).
Checking @build/tools/toolchain-patches/@ in the NDKs shows r8c has matching patches.

Also NDKs from google contain several @SOURCES@ files listing Git repos and commit IDs.
With r8c's release date (@ls --full-time RELEASE.TXT@), @build/tools/download-toolchain-sources.sh@ checkouts the same Git commit IDs, and the patches apply.
Only, r8c references a LLVM Git repo, while our scripts download a tarball.
So our scripts match a NDK version between r8b and r8c, and can almost exactly reproduce r8c. Let's call it r8b2.

h2. Building the SDK

Based on @ndk/docs/DEVELOPMENT.html@, with much improvement :)

<pre>
export NDK=~/replicant-4.2/ndk
cd $NDK

# Allow checking build errors
export NDK_LOGFILE=$HOME/ndk.log

# Follow Python download redirection
sed -i -e 's/curl -S/curl -L -S/' build/tools/download-toolchain-sources.sh

# Get the sources from toolchain/ repos, with r8c release date:
bash $NDK/build/tools/download-toolchain-sources.sh --git-date='2012-10-22T07:59:40+0200' ~/ndk-dl
# ndk-dl: 2.5GB

# Define NDK_TMPDIR and and use --incremental so the build can be resumed on error
export NDK_TMPDIR=~/ndk-tmp
mkdir -p $NDK_TMPDIR/release-r8b2/

# Build the release
bash $NDK/build/tools/make-release.sh --toolchain-src-dir=$HOME/ndk-dl --release=r8b2 --incremental
# 30mn with 4 cores, 28GB (inc. 17GB .repo), 2.5GB ndk-dl, 1GB /tmp
</pre>

The GNU/Linux NDK release is in @/tmp/ndk-replicant/release/android-ndk-r8b2-linux-x86.tar.bz2@ :)


h2. TODOs

h3. Windows build

Build deps:
<pre>
# windows build is triggered by the presence of mingw32
apt-get install mingw32 tofrodos
# and requires running maketab.exe
apt-get install wine
# I had troubles in LXC registering direct .exe execution:
# $ /tmp/ndk-replicant/tmp/build-3038/maketab.exe 
# run-detectors: unable to find an interpreter for /tmp/ndk-replicant/tmp/build-3038/maketab.exe
# Work-around:
update-binfmts --disable
dpkg-reconfigure wine1.4 --pri=high
</pre>

Code fixes:
<pre>
# build-ndk-stack/elff/ produces a warning with mingw32, and uses -Werror
# cf. https://android.googlesource.com/platform/ndk/+/32e74f3f1b969ff65f037e1ee89e21a5cbc0ecf0
sed -i -e 's/-Werror//' sources/host-tools/ndk-stack/GNUMakefile
# TODO: maybe 'export EXTRA_CFLAGS=-Wall' would be enough?
# TODO: does this mean google was rebuilding from Ubuntu 10.04 instead of 12.04?
</pre>

...

<pre>
bash $NDK/build/tools/make-release.sh --toolchain-src-dir=$HOME/ndk-dl --release=r8b2 --incremental
# /home/replicant/ndk-dl/build/../binutils/binutils-2.21/gold/arm.cc:2171: internal compiler error: in make_rtl_for_nonlocal_decl, at cp/decl.c:4971
# => let's disable the windows build for now.. maybe better with 10.04? or a later ndk.git revision?
</pre>

h3. Use of prebuilts

Check how much of $topdir/prebuilts/ is used to compile the NDK. We shouldn't rely on untrusted binaries.

h3. Other build method

Attempt to rebuild r8e with the same build environment but getting .git repos directly from AOSP:
less patches + local LLVM repo + precise control over the ndk scripts revision

According to @ndk/docs/DEVELOPMENT.html@ one only needs:
  git clone https://xxx/platform/ndk.git ndk
  git clone https://xxx/platform/development.git development

This means we wouldn't rely on exactly the build scripts shipped in Replicant 4.2 - but we already grab all the @toolchain/@ sources from AOSP, don't we?

h4. Yet another build method

@ndk/docs/DEVELOPMENT.html@ references using an existing (trusted) NDK to build the next one.