Optimising LEDE (OpenWrt) for the PC Engines APU2

Published: 2016-09-03
Updated: 2019-12-06

The PC Engines APU2 is an embedded system based on AMD's Jaguar family of processors. Many people prefer to run pfSense on it, but since the rest of my hardware is already on LEDE, it's easier for me to stick with that; the APU2 will manage a network running other LEDE nodes, with customised images (see my article on UCI defaults for more information on that). To get an idea of all the tidbits the APU2 offers, the OpenWrt wiki page is a good starting point - since there are no subtargets anymore for x86/64 (alas), you have to install all extra packages manually to get full support for the hardware capabilities.

Default configuration

Because LEDE uses generic targets - a kind of catch-all - for the x86 and x86_64 platforms, by default it only configures eth0, the leftmost interface. It took me a while (coming from consumer platforms) to realise the images I built were fine, I just kept trying the two rightmost interfaces, figuring those were the LAN interfaces... So if you don't seem to be getting an IP, try all of those ports. You never know. There's also no WAN interface defined, so it won't work as a router out of the box.

Hardware support

A stock LEDE image will boot just fine from internal mSATA/S-ATA storage or USB altogether. I have not been able to verify if booting off SD works yet.

Gigabit Ethernet

The APU2 comes with Intel's i210AT or i211AT Gigabit Ethernet NICs, which need the igb driver on Linux. The stock x86_64 LEDE builds include this driver by default.

SDHC card reader

For the card reader, install the kmod-sdhci package. If you'd like to include it in your own build, enable it under Kernel modules > Other modules in the buildroot. To boot off an SD card, however, you need the following symbols to be enabled:

CONFIG_MMC_BLOCK=y
CONFIG_MMC_SDHCI=y
CONFIG_MMC_SDHCI_PCI=y

You can grep target/linux/x86/64/config-default for them, they should all be enabled by default. Add them to the same file if they aren't. I have been unable to test so far if booting off SD cards works; I've seen some statements that the APU2 will ignore SD or USB 3 boot devices if there is an mS-ATa SSD installed.

USB 3.0 support

Booting off USB 3.0 works just fine out of the box - not much to add.

Temperature sensor

For basic temperature reading support - which may come in handy as you essentially rely on passive cooling - install the kmod-hwmon-k10temp package, or enable kmod-hwmon-k10temp in the buildroot under Kernel modules > Hardware Monitoring Support. Reading out the temperature is as simple as typing sensors:

# sensors
fam15h_power-pci-00c4
Adapter: PCI adapter
power1:        1.66 W  (interval =   0.01 s, crit =   6.00 W)

k10temp-pci-00c3
Adapter: PCI adapter
temp1:        +52.1°C  (high = +70.0°C)
               (crit = +105.0°C, hyst = +104.0°C)

As you can see, though, it only reports one temperature instead of one per core.

VLAN support

For VLAN support, you need the 8021q module installed. The generic target statically includes it. You will find a package under Kernel modules > Network Support; however, this is only useful for targets that do not include the module by default. On targets like e.g. ar71xx or x86, it will just generate a package only containing a modprobe file, nothing more. You can set up tagged VLANs through LEDE's framework -in /etc/config/network, e.g.:

config interface 'guest'
    option type 'bridge'
    option proto 'static'
    option netmask '255.255.255.0'
    option ip6assign '60'
    option ifname 'eth1.20 eth2.20'
    option ipaddr '10.0.1.1'

This will set up a bridge named br-guest, containing both eth1 and eth2, tagged with VLAN ID 20. This is standard; the bridge itself apparently does not get tagged, only the member interfaces.

# for i in eth1.20 eth2.20 br-guest; do ip -d link show $i; done
9: eth1.20@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-guest state UP mode DEFAULT group default qlen 1000
    link/ether 00:0d:b9:41:45:61 brd ff:ff:ff:ff:ff:ff promiscuity 1 
    vlan protocol 802.1Q id 20 <REORDER_HDR> 
    bridge_slave state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on addrgenmode eui64 
10: eth2.20@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-guest state UP mode DEFAULT group default qlen 1000
    link/ether 00:0d:b9:41:45:62 brd ff:ff:ff:ff:ff:ff promiscuity 1 
    vlan protocol 802.1Q id 20 <REORDER_HDR> 
    bridge_slave state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on addrgenmode eui64 
8: br-guest: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:0d:b9:41:45:61 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    bridge forward_delay 200 hello_time 200 max_age 2000 ageing_time 30000 stp_state 0 priority 32767 vlan_filtering 0 addrgenmode eui64

I can confirm this works as expected - for more info on VLANs on OpenWrt/LEDE, see my older post.

LED support

By default only the leftmost LED is active, and it just stays on. As of commit fe12d51, LEDE has a driver for the APU2's PCB LEDs and reset button. I have backported this to my LEDE 17.01 builds.

GPIO header

The GPIO headers are controlled by the Nuvoton NCT5104D chip. To use them, enable the kmod-gpio-nct5104d package under Kernel modules > Other modules.

AMD's cryptographic coprocessor ('CCP')

There's a few things you need to do before you can use the CCP. It can only work if you also enable the right cryptographic engine in OpenSSL; otherwise, you'll see an error like this in dmesg:

[    9.700375] kmodloader: 1 module could not be probed
[    9.705437] kmodloader: - ccp-crypto - 0

To enable the CCP, do the following. The last two steps as necessary as of 19.07, since engine support has been split out.

  • Enable the kmod-crypto-hw-ccp package under Kernel modules > Cryptographic API modules.
  • Under Libraries > SSL > libopenssl, tick Enable engine support and enable libopenssl-devcrypto.
  • Optionally, you can tick Build chosen engines into libcrypto. Keep in mind you need to enable the engine(s) manually if you don't, and add /etc/ssl/openssl.cnf to/etc/sysupgrade.conf`, if you'd like the settings to persist between upgrades.

Hardware watchdog

Enable the kmod-sp5100-tco package under Kernel modules > Other modules.

Optimising LEDE for the APU2

GCC optimisation

GCC has its own target for the Jaguar SoCs. You can optimise by using the -march=btver2 -mtune=btver2 GCC options, with a recent GCC version. In the buildroot, enable Advanced configuration options (for developers), then tick Target Options and add the compiler flags to Target Optimizations. Do keep in mind though this will greatly limit your testing capabilites - I haven't found a way to emulate a CPU supporting the same feature set yet, if you do, let me know, that would greatly simplify testing. For now I build a separate generic x86_64 build so I can test it in a VM, before deploying it.

AES-NI

One of the cooler things with the APU2 is it supports AES-NI, which is handy for applications relying on encryption (e.g. a VPN). LEDE enabled the necessary bits and bolts for that in the meantime. AES-NI is handy for e.g. IPsec and OpenVPN. Since a leaner solution like WireGuard does not rely on AES, it won't use the hardware acceleration the Jaguar SoC offers either.
If you like to use AES-NI hardware acceleration, you'll need to the devcrypto engine in /etc/ssl/openssl.cnf and make sure you have the appropriate packages installed (see the CCP section above). You can also add hardware acceleration support to e.g. GnuTLS as explained on the OpenWrt wiki, if you use that as well.

Resizing partitions

Chances are you'll be running conventional storage, so the default 4/48 configuration for the kernel and root partitions do not really use the space available. You can change this by setting the CONFIG_TARGET_ROOTFS_PARTSIZE= (for the root file system) and CONFIG_TARGET_KERNEL_PARTSIZE= for the kernel partition size.

UCI defaults

I have preconfigured my LEDE builds for the APU2 to set up a WAN interface on eth0 and a LAN bridge on the eth1 and eth2 interfaces, as well as some other tidbits. Imho, if you want to deploy the APU2 with LEDE/OpenWrt, prepping the network is a must.