Optimising LEDE (OpenWrt) for the PC Engines APU2Published: 2016-09-03
The PC Engines APU2 is an embedded system based on AMD's Jaguar family of processors. Many people prefer to run pfSense on it, but since the rest of my hardware is already on LEDE, it's easier for me to stick with that; the APU2 will manage a network running other LEDE nodes, with customised images (see my article on UCI defaults for more information on that). To get an idea of all the tidbits the APU2 offers, the OpenWrt wiki page is a good starting point - since there are no subtargets anymore for x86/64 (alas), you have to install all extra packages manually to get full support for the hardware capabilities.
Because LEDE uses generic targets - a kind of catch-all - for the x86 and x86_64 platforms, by default it only configures
eth0, the leftmost interface. It took me a while (coming from consumer platforms) to realise the images I built were fine, I just kept trying the two rightmost interfaces, figuring those were the LAN interfaces... So if you don't seem to be getting an IP, try all of those ports. You never know. There's also no WAN interface defined, so it won't work as a router out of the box.
A stock LEDE image will boot just fine from internal mSATA/S-ATA storage or USB altogether. I have not been able to verify if booting off SD works yet.
The APU2 comes with Intel's i210AT or i211AT Gigabit Ethernet NICs, which need the
igb driver on Linux. The stock x86_64 LEDE builds include this driver by default.
SDHC card reader
For the card reader, install the
kmod-sdhci package. If you'd like to include it in your own build, enable it under
Kernel modules > Other modules in the buildroot. To boot off an SD card, however, you need the following symbols to be enabled:
CONFIG_MMC_BLOCK=y CONFIG_MMC_SDHCI=y CONFIG_MMC_SDHCI_PCI=y
You can grep
target/linux/x86/64/config-default for them, they should all be enabled by default. Add them to the same file if they aren't. I have been unable to test so far if booting off SD cards works; I've seen some statements that the APU2 will ignore SD or USB 3 boot devices if there is an mS-ATa SSD installed.
USB 3.0 support
Booting off USB 3.0 works just fine out of the box - not much to add.
For basic temperature reading support - which may come in handy as you essentially rely on passive cooling - install the
kmod-hwmon-k10temp package, or enable
kmod-hwmon-k10temp in the buildroot under
Kernel modules > Hardware Monitoring Support. Reading out the temperature is as simple as typing
# sensors fam15h_power-pci-00c4 Adapter: PCI adapter power1: 1.66 W (interval = 0.01 s, crit = 6.00 W) k10temp-pci-00c3 Adapter: PCI adapter temp1: +52.1°C (high = +70.0°C) (crit = +105.0°C, hyst = +104.0°C)
As you can see, though, it only reports one temperature instead of one per core.
For VLAN support, you need the
8021q module installed. The generic target statically includes it. You will find a package under
Kernel modules > Network Support; however, this is only useful for targets that do not include the module by default. On targets like e.g.
x86, it will just generate a package only containing a modprobe file, nothing more. You can set up tagged VLANs through LEDE's framework -in
config interface 'guest' option type 'bridge' option proto 'static' option netmask '255.255.255.0' option ip6assign '60' option ifname 'eth1.20 eth2.20' option ipaddr '10.0.1.1'
This will set up a bridge named
br-guest, containing both eth1 and eth2, tagged with VLAN ID 20. This is standard; the bridge itself apparently does not get tagged, only the member interfaces.
# for i in eth1.20 eth2.20 br-guest; do ip -d link show $i; done 9: eth1.20@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-guest state UP mode DEFAULT group default qlen 1000 link/ether 00:0d:b9:41:45:61 brd ff:ff:ff:ff:ff:ff promiscuity 1 vlan protocol 802.1Q id 20 <REORDER_HDR> bridge_slave state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on addrgenmode eui64 10: eth2.20@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-guest state UP mode DEFAULT group default qlen 1000 link/ether 00:0d:b9:41:45:62 brd ff:ff:ff:ff:ff:ff promiscuity 1 vlan protocol 802.1Q id 20 <REORDER_HDR> bridge_slave state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on addrgenmode eui64 8: br-guest: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether 00:0d:b9:41:45:61 brd ff:ff:ff:ff:ff:ff promiscuity 0 bridge forward_delay 200 hello_time 200 max_age 2000 ageing_time 30000 stp_state 0 priority 32767 vlan_filtering 0 addrgenmode eui64
I can confirm this works as expected - for more info on VLANs on OpenWrt/LEDE, see my older post.
By default only the leftmost LED is active, and it just stays on. As of commit fe12d51, LEDE has a driver for the APU2's PCB LEDs and reset button. I have backported this to my LEDE 17.01 builds.
The GPIO headers are controlled by the Nuvoton NCT5104D chip. To use them, enable the
kmod-gpio-nct5104d package under
Kernel modules > Other modules.
AMD's cryptographic coprocessor ('CCP')
There's a few things you need to do before you can use the CCP. It can only work if you also enable the right cryptographic engine in OpenSSL; otherwise, you'll see an error like this in dmesg:
[ 9.700375] kmodloader: 1 module could not be probed [ 9.705437] kmodloader: - ccp-crypto - 0
To enable the CCP, do the following. The last two steps as necessary as of 19.07, since engine support has been split out.
- Enable the
Kernel modules > Cryptographic API modules.
Libraries > SSL > libopenssl, tick
Enable engine supportand enable
- Optionally, you can tick
Build chosen engines into libcrypto. Keep in mind you need to enable the engine(s) manually if you don't, and add
/etc/ssl/openssl.cnf to/etc/sysupgrade.conf`, if you'd like the settings to persist between upgrades.
kmod-sp5100-tco package under
Kernel modules > Other modules.
Optimising LEDE for the APU2
GCC has its own target for the Jaguar SoCs. You can optimise by using the
-march=btver2 -mtune=btver2 GCC options, with a recent GCC version. In the buildroot, enable
Advanced configuration options (for developers), then tick
Target Options and add the compiler flags to
Target Optimizations. Do keep in mind though this will greatly limit your testing capabilites - I haven't found a way to emulate a CPU supporting the same feature set yet, if you do, let me know, that would greatly simplify testing. For now I build a separate generic x86_64 build so I can test it in a VM, before deploying it.
One of the cooler things with the APU2 is it supports AES-NI, which is handy for applications relying on encryption (e.g. a VPN). LEDE enabled the necessary bits and bolts for that in the meantime. AES-NI is handy for e.g. IPsec and OpenVPN. Since a leaner solution like WireGuard does not rely on AES, it won't use the hardware acceleration the Jaguar SoC offers either.
If you like to use AES-NI hardware acceleration, you'll need to the
devcrypto engine in
/etc/ssl/openssl.cnf and make sure you have the appropriate packages installed (see the CCP section above). You can also add hardware acceleration support to e.g. GnuTLS as explained on the OpenWrt wiki, if you use that as well.
Chances are you'll be running conventional storage, so the default 4/48 configuration for the kernel and root partitions do not really use the space available. You can change this by setting the
CONFIG_TARGET_ROOTFS_PARTSIZE= (for the root file system) and
CONFIG_TARGET_KERNEL_PARTSIZE= for the kernel partition size.
I have preconfigured my LEDE builds for the APU2 to set up a WAN interface on
eth0 and a LAN bridge on the
eth2 interfaces, as well as some other tidbits. Imho, if you want to deploy the APU2 with LEDE/OpenWrt, prepping the network is a must.