Thomas Jepp

Multi-platform builds with Gitlab CI

Posted on and revised on

I recently bought a M1 Mac Mini as a test machine, and I've been so impressed with it that it has become my main personal machine.

However, unlike older machines, it does not run Intel binaries natively - it runs arm64 binaries. This means that for best performance I need to rebuild my containers as arm64 containers.

Creating a Gitlab CI build that can build for multiple architectures is a bit more complicated than it might first appear - you need to support docker buildx.

Preparing the runner

  1. Install Debian on your runner. I used Debian 10 - buster.
  2. Install up to date Docker. Follow the official instructions.
  3. Install the Gitlab CI runner. Follow the official instructions.
  4. Register your runner. Follow the official instructions.
  5. Add some default environment variables to /etc/gitlab-runner/config.toml:
     concurrent = 1
     check_interval = 0
    
     [session_server]
     session_timeout = 1800
    
     [[runners]]
     name = "home"
     url = "*snip*"
     token = "*snip*"
     executor = "docker"
     [runners.custom_build_dir]
     [runners.cache]
         [runners.cache.s3]
         [runners.cache.gcs]
         [runners.cache.azure]
     [runners.docker]
         tls_verify = false
         image = "docker:stable"
         privileged = true
         disable_entrypoint_overwrite = false
         oom_kill_disable = false
         disable_cache = false
         volumes = ["/certs/client", "/cache"]
         shm_size = 0
         environment = [ "DOCKER_HOST=tcp://docker:2375", "DOCKER_TLS_CERTDIR=/certs", "DOCKER_CERT_PATH=/certs/client" ]
    
  6. Restart the gitlab runner:
    systemctl restart gitlab-runner
    

At this point, you have a working basic Gitlab CI runner.

Enabling QEMU for binary emulation

apt install qemu-user-static binfmt-support

On Debian buster, this enables executing different architecture binaries.

Creating a suitable docker buildx image

The default docker images don't include buildx. You'll need this to be able to do multi-platform builds.

I created my own docker container to use to build from:

FROM docker:latest

RUN apk add curl jq

RUN mkdir -vp ~/.docker/cli-plugins
RUN sh -c 'RELEASE=$(curl -s https://api.github.com/repos/docker/buildx/releases/latest | jq .name -r) && \
    echo "https://github.com/docker/buildx/releases/download/${RELEASE}/buildx-${RELEASE}.linux-amd64" && \
    curl -s -L https://github.com/docker/buildx/releases/download/${RELEASE}/buildx-${RELEASE}.linux-amd64 -o ~/.docker/cli-plugins/docker-buildx'
RUN chmod -v 755 ~/.docker/cli-plugins/docker-buildx
RUN docker buildx version

Making a multi-platform build

Starting from the example Gitlab CI Docker template:

docker-build-master:
  # Official docker image.
  image: docker:latest
  stage: build
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE" .
    - docker push "$CI_REGISTRY_IMAGE"
  only:
    - master

docker-build:
  # Official docker image.
  image: docker:latest
  stage: build
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" .
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG"
  except:
    - master

Replace with something like:

image: *your buildx image*

services:
  - docker:dind

before_script:
  - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  - docker context create tls-environment
  - docker buildx create --name multiarch-builder --use tls-environment

docker-build-master:
  stage: build
  script:
    - docker buildx build --push --platform linux/arm64,linux/amd64 -t "$CI_REGISTRY_IMAGE" .
  only:
    - master

docker-build:
  stage: build
  script:
    - docker buildx build --push --platform linux/arm64,linux/amd64 -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" .
  except:
    - master

At this point, you should have a working multi-platform image.

On my M1 Mac:

docker run --rm -it *your image* bash
root@3383d76e8c7b:/# arch
aarch64

On my x86_64 mac:

docker run --rm -it *your image* bash
root@94a61be1572c:/# arch
x86_64