Compiling LineageOS on Linux - quick 'n' dirty


Updated: 2018-09-15

This is a quick summary on how to compile an existing LineageOS tree on Linux. LineageOS is one of the bigger Android distributions out there, so I'll provide instructions based on their tree, although most project handle a very similar workflow, often being based on AOSP. LineageOS offers multiple features not found in stock AOSP, like built-in profile support, and further customisation options. You can find a rather exhaustive list in its Wikipedia entry. Keep in mind you'll need about 100 GiB of free space to build a modern Android. If you'd like to use ccache, add an extra 50 GiB.

Prerequisites

You'll need a Java Development Kit (JDK) installed along with Android's fastboot and adb tools. On Debian, that looks like this:

$ sudo apt-get install android-tools-adb android-tools-fastboot openjdk-8-jdk

Next, you'll need quite a few libraries and headers to complement what might already be on your system, depending on what you use it for, so this list is not exhaustive - you might need to expand it as you go (and compilation breaks because of missing tools; the build process will tell you).

$ sudo apt-get install sudo apt-get install bc bison build-essential ccache curl flex g++-multilib gcc-multilib git gnupg gperf imagemagick lib32ncurses5-dev lib32readline-dev lib32z1-dev libesd0-dev liblz4-tool libncurses5-dev libsdl1.2-dev libssl-dev libwxgtk3.0-dev libxml2 libxml2-utils lzop pngcrush rsync schedtool squashfs-tools xsltproc zip zlib1g-dev

You'll need Google's repo script to handle the huge codebase (it is a wrapper script that interfaces with git, amongst others). Grab it from Google itself, and drop it in your folder where you keep scripts etc. Make it executable, and, to make things easier on yourself, you can also add it to your PATH.

$ cd ~/path/to/scripts/
$ wget https://storage.googleapis.com/git-repo-downloads/repo
$ chmod +x repo
$ export PATH="$PATH:/home/youruser/path/to/scripts/"

If you want to grab the necessary blobs from a LineageOS installation image instead of pulling them from the device itself (see further down), you'll need the sdat2img script as well:

$ git clone https://github.com/xpirt/sdat2img ~/path/to/scripts/sdat2img/

Preparing the codebase while keeping a minimal footprint

Next we'll grab the actual code and trim the fat.

Setting up and syncing your local checkout.

You probably don't need the whole history (backlog) that comes with Android, so use --depth 1. On LineageOS 14.1, this shaves off about 12 GiB. The command below will check out LineageOS (forked from CyanogenMod) 14.1.

$ mkdir ~/code/lineage-14.1
$ cd ~/code/lineage-14.1
$ repo init --depth 1 -u https://github.com/LineageOS/android.git -b cm-14.1

For LineageOS 15 and higher, use lineage-$version as your branch name, e.g.:

$ repo init --depth 1 -u https://github.com/LineageOS/android.git -b lineage-15.1

Repo init won't take long, the real work is done by syncing the code. Depending on your internet connection, this might take quite a bit (half an hour or longer). While repo init is a one-time operation, synchronising the code is something you should do every time you want to build off the latest codebase.

$ repo sync

repo sync will use four threads (-j4) by default; you can override this if you'd like.

Grabbing the kernel settings for your device

Once you're done syncing, you can pull in the kernel settings. These are device-dependent. The breakfast command will add the necessary repositories to .repo/local_manifests/roomservice.xml and sync them. For the HTC One (M7) - codename m7 - e.g. it would look like this:

$ source build/envsetup.sh
$ breakfast m7

If you are building for a device that LineageOS does not support officially, you need to add them manually. The Sony Xperia Z3 Compact, for example, has never seen any official releases with Lineage, but had unofficial builds available through XDA. You can add additional roomservice.xml files to service multiple devices - just make sure there are no duplicate entries. Adding unofficial repositories is straightforward; you strip the Github URL and use the path that follows; e.g. to use the repository of one of the main developers for the Z3 Compact, your roomservice.xml entry would look like this:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <project name="tomascus/android_device_sony_z3c" path="device/sony/z3c" remote="github" />
[...]
</manifest>

The Github repository pulled in would be https://github.com/tomascus/android_device_sony_z3c.

Removing projects you don't need

The Android codebase is gigantic and chances are you'll pull in stuff you do not need at all for your device. You can add projects you don't need to the .repo/local_manifests/trim.xml file (create it if it does not exist). When building on Linux, e.g., you don't need any OS X stuff:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <remove-project name="LineageOS/android_prebuilts_gcc_darwin-x86_aarch64_aarch64-linux-android-4.9" />
  <remove-project name="LineageOS/android_prebuilts_gcc_darwin-x86_arm_arm-linux-androideabi-4.9" />
  <remove-project name="LineageOS/android_prebuilts_gcc_darwin-x86_x86_x86_64-linux-android-4.9" />
  <remove-project name="LineageOS/android_prebuilts_gcc_linux-x86_aarch64_aarch64-linux-android-4.9" />
  <remove-project name="platform/prebuilts/clang/darwin-x86/host/3.6" />
  <remove-project name="platform/prebuilts/clang/host/darwin-x86" />
  <remove-project name="platform/prebuilts/gcc/darwin-x86/arm/arm-eabi-4.8" />
  <remove-project name="platform/prebuilts/gcc/darwin-x86/host/i686-apple-darwin-4.2.1" />
  <remove-project name="platform/prebuilts/gcc/linux-x86/host/x86_64-w64-mingw32-4.8" />
  <remove-project name="platform/prebuilts/gdb/darwin-x86" />
  [...]
</manifest>

Make sure to sync before stripping projects.

Cherry-picking patches and patch sets

The repopick command allows you to pick patches or patch sets sitting in the LineageOS review process. You can find more info on the LineageOS wiki page. For a single patch, you'd use the number referring to the proposed commit on https://review.lineageos.org/.

$ source build/envsetup.sh
$ repopick 288870

For a patch set (comes in especially handy with the monthly ASB patches) you'd use the topic, e.g. for August's security patches:

$ repopick -t n_asb_08-2018

The ASB updates might require you to forcibly update a few repos - e.g. because LineageOS starts tracking their own tree instead of the abandoned upstream one. The ASB patch sets will break until you do this. Look for messages like 'track our own libxml2' (like this patch being part of the September 2018 set) and adapt your defaults.xml accordingly, then force a sync for the specific repo(s):

$ reposync --force-sync external/libxml2 external/neven

Adding binary blobs

There are three ways to go about this. The first being the easiest, that's what I'll cover; the remaining options are thoroughly explained on the LineageOS wiki.

Pulling blobs from TheMuppets Github repository

All you need to do is add the repo to your roomservice.xml. You'll need to specify the manufacturer (vendor). For the HTC One, e.g. it would look like this:

<project name="TheMuppets/proprietary_vendor_htc" path="vendor/htc" remote="github" />

After adding it, run repo sync again.

Compiling Android itself

At this point, you are finally ready to put your computer to work. If you plan on doing more than one build, you should enable ccache (in fact, you already pulled it in if you copy/pasted the commands I provided earlier). Ccache might claim a lot of cache, and by default it will store this on in your $HOME folder, regardless of where you are actually storing and compiling the Android code; make sure to have enough free space there. The commands below will allow ccache to use 50 GB.

$ export USE_CCACHE=1
$ ccache -M 50G

The LineageOS 14.1 toolchain needs to be reconfigured to avoid out of memory (OOM) errors:

$ export ANDROID_JACK_VM_ARGS="-Dfile.encoding=UTF-8 -XX:+TieredCompilation -Xmx4G"

With the final bits set, you can now compile for your device:

$ croot
$ brunch m7

When compilation is done (this might take half an hour or more, even on modern, heavier systems), grab the binaries from the $OUT directory:

$ ls $OUT/*m7.zip*
/home/test/code/lineage-14.1/out/target/product/m7/lineage-14.1-20180819-UNOFFICIAL-m7.zip
/home/test/code/lineage-14.1/out/target/product/m7/lineage-14.1-20180819-UNOFFICIAL-m7.zip.md5sum

That's it; happy flashing.