Embedded Linux

Embedded Linux Device Emulation — Part 1: Docker & Buildx

Learn how to emulate ARM64 embedded Linux environments using Docker and Buildx for faster development without physical hardware

by Fabbio Protopapa
Also available in: Polski
#embedded-linux #docker #emulation #arm64

If you’ve worked on embedded device software, you know the pain: the hardware sits three desks away tangled in cables… or three continents away at a client’s site. And you still need to compile, test, and debug. Fun, right?

In this article series, we’ll tackle this problem. We’ll explore different methods for emulating embedded Linux environments on a typical development machine (still mostly x86-64, but more on that later). We’ll check out solutions like Docker and QEMU, and see how they perform for debugging, testing, with custom rootfs images, and for CI/CD use cases.

In this post, we’ll cover Docker and focus on setup and running applications.

Docker: The Simplest Starting Point

Quick Recap: What Actually Is Docker?

Docker lets you package applications and their dependencies into lightweight containers that use the host’s kernel. There’s no system virtualization here, no hypervisor or other tricks. A container is just a process (or group of processes) from the host’s perspective, utilizing Linux kernel features like namespaces and cgroups (for the curious).

In practice, this means a container built for ARM won’t run on a different architecture.

Alright, guess we can stop here :). Wait, wait, we have this cool thing called buildx, but more on that later.

First, let’s install Docker.

Installing Docker on Ubuntu 24.04

Installation should be straightforward and is well documented in the official documentation (documentation).

# Add Docker's official GPG key:
sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF

sudo apt update

If we look closely, we’ll spot buildx mentioned below.

sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

To test the installation, we can use this command:

$ docker run --rm hello-world

Building Our Own Container

Let’s write a tiny C program to work with:

// hello.c
#include <stdio.h>

int main(void)
{
    #if defined(__x86_64__)
        printf("Hello from x86_64!\n");
    #elif defined(__aarch64__)
        printf("Hello from ARM64 (aarch64)!\n");
    #else
        printf("Hello from unknown architecture!\n");
    #endif

    return 0;
}

Let’s create a simple Dockerfile:

FROM gcc:12

WORKDIR /usr/src/app

COPY hello.c .

# Compile the program with debug symbols and w/o optimization
RUN gcc -O0 -g -o hello hello.c

CMD ["./hello"]

And there we have it:

$ docker build -t hello-native .
$ docker run --rm hello-native
Hello from x86_64!

And now the most important question: why are we messing with this if we wanted to run the application on ARM64 architecture?

Fair point. Often we don’t need the hardware, the target kernel, or the same system architecture. The OS abstraction layer will let us run the application anyway. This means we can package our application, which we want to develop or test easily, into a container native to our host system.

Of course, there are situations where we can’t use the above solution, for example if our software depends on a custom kernel driver, or doesn’t support the host system’s architecture. Or other weird issues that aren’t so rare with embedded systems :).

So native Docker works great when we want:

  • environment isolation (e.g., we don’t install build tools on our host),
  • fast iteration during development,
  • application logic testing.

But what if we don’t want to maintain an additional architecture for an application we don’t really need? And that’s where buildx comes into play.

Buildx: Container “with Extra Steps”

Docker Buildx extends the standard builder with multi-arch support. Under the hood, it uses QEMU user-mode, which translates system calls from one architecture to another. This lets you “run” a binary compiled for ARM64 on an x86 machine. And without emulating an entire computer.

Buildx should already be available if we have a relatively new Docker installation. If not, you can install buildx as shown above in the installation section. Let’s check if we have buildx:

$ docker buildx version
github.com/docker/buildx v0.26.1 1a8287f

Using the default builder isn’t recommended, so we create a new one and enable it.

$ docker buildx create --name multiarch-builder --use
multiarch-builder

$ docker buildx inspect --bootstrap
[+] Building 8.3s (1/1) FINISHED
 => [internal] booting buildkit                                            8.3s
 => => pulling image moby/buildkit:buildx-stable-1                         7.3s
 => => creating container buildx_buildkit_arch64-builder0                  0.9s
Name:          multiarch-builder
Driver:        docker-container
Last Activity: 2025-11-28 20:33:57 +0000 UTC

Nodes:
Name:                  multiarch-builder0
Endpoint:              unix:///var/run/docker.sock
Status:                running
BuildKit daemon flags: --allow-insecure-entitlement=network.host
BuildKit version:      v0.26.2
Platforms:             linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386

If other architectures are missing, we can use the command below. It installs a cross-platform emulator packaged in a container (binfmt).

$ docker run --privileged --rm tonistiigi/binfmt --install all
Unable to find image 'tonistiigi/binfmt:latest' locally
latest: Pulling from tonistiigi/binfmt
f4700b809f99: Pull complete
2adec5d296ac: Pull complete
Digest: sha256:30cc9a4d03765acac9be2ed0afc23af1ad018aed2c28ea4be8c2eb9afe03fbd1
Status: Downloaded newer image for tonistiigi/binfmt:latest
installing: arm64 OK
installing: s390x OK
installing: ppc64le OK
installing: mips64le OK
installing: mips64 OK
installing: loong64 OK
installing: riscv64 OK
installing: arm OK
{
  "supported": [
    "linux/amd64",
    "linux/amd64/v2",
    "linux/amd64/v3",
    "linux/arm64",
    "linux/riscv64",
    "linux/ppc64le",
    "linux/s390x",
    "linux/386",
    "linux/mips64le",
    "linux/mips64",
    "linux/loong64",
    "linux/arm/v7",
    "linux/arm/v6"
  ],
  "emulators": [
    "llvm-18-runtime.binfmt",
    "python3.12",
    "qemu-aarch64",
    "qemu-arm",
    "qemu-loongarch64",
    "qemu-mips64",
    "qemu-mips64el",
    "qemu-ppc64le",
    "qemu-riscv64",
    "qemu-s390x"
  ]
}

$ docker buildx inspect --bootstrap
...
Platforms:             linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/mips64le, linux/mips64, linux/loong64, linux/arm/v7, linux/arm/v6
...

Rebuilding Our Example for ARM64

The Dockerfile and application file stay the same. Only the build and run commands change. Docker handles everything else for us.

docker buildx build \
	--platform linux/arm64 \
	-t hello-simple:arm64 \
	--load .

If we want to build a container for multiple architectures in one command, that’s possible. But in that case, we must immediately push to a container registry (or export to a tar archive). The Docker engine doesn’t understand multi-arch manifests, so we can only load a single container image directly.

$ docker run --platform linux/arm64 --rm hello-simple:arm64
Hello from ARM64 (aarch64)!

Success :).

What happened:

  • we compiled an ARM64 binary,
  • the gcc image supports multi-arch, so the correct image was pulled
  • we ran the container on x86,
  • QEMU user-mode translated instructions “on the fly”.

If this works so well, we can just use this solution and we’re done! Except this situation isn’t that simple. Besides the issues mentioned for regular Docker, buildx has its own specific drawbacks.

Compiling software through emulation is slow. With this small example it’s not noticeable, but with a large project - it can be a problem. The official documentation mentions it can be significantly slower (various threads mention 4-10 times slower). There are optimization possibilities, like using a native node, caching results, or cross-compilation (which we’ll look at later).

Also, non-typical syscalls might not work. That’ll be a topic for the next post.

Speeding Up Builds: Cross-Compilation

We need to modify the Dockerfile and build the container in two steps. In the first, we compile the application depending on the build platform. In the second, we copy the binary to the target image, which also supports ARM64 architecture.

FROM --platform=$BUILDPLATFORM gcc:12 AS builder

ARG TARGETPLATFORM

WORKDIR /usr/src/app

COPY hello.c .

RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
	apt-get update && apt-get install -y gcc-aarch64-linux-gnu && \
	aarch64-linux-gnu-gcc -O0 -g hello.c -o hello; \
	else \
	gcc -O0 -g hello.c -o hello; \
	fi

FROM debian:stable-slim

WORKDIR /usr/bin

COPY --from=builder /usr/src/app/hello hello

CMD ["./hello"]

Now we can build for amd64:

docker buildx build \
	  --platform linux/amd64 \
	  -t hello:amd64 \
	  --load \
	  .

And for ARM:

docker buildx build \
	--platform linux/arm64 \
	-t hello:arm64 \
	--load \
	.

And the final test:

$ docker run --rm hello:amd64
Hello from AMD64 (amd64)!

$ docker run --platform linux/arm64 --rm hello:arm64
Hello from ARM64 (aarch64)!

This solution requires a more complex Dockerfile, but it lets us fully utilize the host’s capabilities during compilation.

Summary

In this part, we tackled building with Docker and Buildx. We were able to cross-compile applications and run them on a foreign architecture.

When to use this approach?

When you want to:

  • build ARM images in CI,
  • test userland applications without special hardware requirements,
  • get repeatable builds and convenience.

Don’t use when you need:

  • kernel emulation,
  • kernel drivers,
  • faithful hardware reproduction,
  • performance and low-level tests.

What’s Next?

In the next part, we’ll tackle debugging applications in containers. We’ll check out the capabilities and limitations of using GDB.

Docker and Buildx are a great starting point, but it’s just the beginning of playing with embedded Linux emulation :).

Fabbio Protopapa

Fabbio Protopapa

Embedded Linux engineer. An enthusiast of open-source, IoT, and the internet.

Email GitHub LinkedIn Website

Related Posts