Rust based applications become trends nowadays, as it got robustnest and minimal transition from C or any good reason out there, but cross compiling rust application also tricky sometimes, there are many platform out there in this write i will cover 6 of, x86_64 aarch64 arm32 for musl and glibc.

Installing the packages

First we need docker rust image and add things on top of it.

• Dockerfile x86_64

  FROM rust:1.65-slim-bullseye
  RUN apt update && \
  apt install -y --no-install-recommends curl git wget build-essential make perl pkg-config curl tar jq && \
  apt clean && \
  rm -rf /var/lib/apt/lists/* 

  Then you get image for building x86_64, then how about musl build? add musl-tools package, yes thats it.

• Dockerfile

  FROM rust:1.65-slim-bullseye
  RUN apt update && \
  apt install -y --no-install-recommends curl git wget build-essential make perl pkg-config curl tar jq musl-tools && \
  apt clean && \
  rm -rf /var/lib/apt/lists/* 

And don't forget to add target for x86_64 glibc and x86_64 musl, by default usually only x86_64-unknown-linux-gnu installed, as this image based on libc debian.

RUN rustup target add  x86_64-unknown-linux-gnu
RUN rustup target add x86_64-unknown-linux-musl

Now we got target for musl but we don't have compiler for it, then we manually download the musl-gcc

RUN wget --progress=dot:mega -t 10-c https://musl.cc/x86_64-linux-musl-cross.tgz && \
    tar zxf ./x86_64-linux-musl-cross.tgz -C /opt && \
    rm ./x86_64-linux-musl-cross.tgz

Pretty straightforward i said, for enabling glibc and musl on x86_64, now we are getting into tricky part. The steps for aarch64 or arm64 build also pretty much look alike with x86_64

RUN dpkg --add-architecture arm64 && \
    apt update && \
    apt install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libc6-arm64-cross libc6-dev-arm64-cross && \
    apt clean && \
    rm -rf /var/lib/apt/lists/*

Add the aarch64 target

RUN rustup target add aarch64-unknown-linux-gnu
RUN rustup target add aarch64-unknown-linux-musl

Download musl-gcc for aarch64

RUN wget --progress=dot:mega -t 10 -c https://musl.cc/aarch64-linux-musl-cross.tgz && \
    tar zxf ./aarch64-linux-musl-cross.tgz -C /opt && \
    rm ./aarch64-linux-musl-cross.tgz

And again the arm 32bit part also pretty similiar, add armhf compiler packages and add the target.

RUN dpkg --add-architecture armhf && \
    apt update && \
    apt install -y gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf libc6-armhf-cross libc6-dev-armhf-cross && \
    apt clean && \
    rm -rf /var/lib/apt/lists/*

add armhf target

RUN rustup target add armv7-unknown-linux-gnueabihf
RUN rustup target add arm-unknown-linux-musleabihf

and again we fetch the armhf musl-gcc

RUN wget --progress=dot:mega -t 10 -c https://musl.cc/armv7l-linux-musleabihf-cross.tgz && \
    tar zxf ./armv7l-linux-musleabihf-cross.tgz -C /opt && \
    rm ./armv7l-linux-musleabihf-cross.tgz

Lastly, dont forget export our musl-gcc to PATH

ENV PATH="/opt/aarch64-linux-musl-cross/:/opt/aarch64-linux-musl-cross/bin/:/opt/x86_64-linux-musl-cross/:/opt/x86_64-linux-musl-cross/bin/:$PATH"

Compiling

If i need to list all env that affect the compilation essentially only 2: OPENSSL and COMPILER PATH

OPENSSL_INCLUDE_DIR=/usr/include/openssl/
OPENSSL_LIB_DIR=/usr/lib/x86_64-linux-gnu/
OPENSSL_LIB_DIR=/usr/lib/arm64-linux-gnu/
OPENSSL_LIB_DIR=/usr/lib/arm-linux-gnueabihf/

CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-musl-gcc

CARGO_TARGET_ARMV7_UNKNOWN_LINUX_MUSLEABIHF_LINKER: arm-linux-gnueabihf-gcc
CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_LINKER: arm-linux-gnueabihf-gcc

CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-linux-musl-gcc

Notes:

• On building with gnu target, you may don't need to specify linker
• Building musl target must specify linker or it will throw error about musl-gcc not found
• From my experience building armv7 on musl is likely to fail due musl libs issue itself, i got this issue time64, yours may be different

Sample of building:

• Override openssl

  OPENSSL_INCLUDE_DIR=/usr/include/openssl/ OPENSSL_LIB_DIR=/usr/lib/x86_64-linux-gnu/ cargo build --target=x86_64-unknown-linux-gnu --release

• Override linkder

  CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc cargo build --target=aarch64-unknown-linux-gnu --release
  CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_LINKER: arm-linux-gnueabihf-gcc cargo build --target=aarch64-unknown-linux-gnu --release
  CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER: aarch64-linux-musl-gcc cargo build --target=aarch64-unknown-linux-gnu --release
  CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc cargo build --target=aarch64-unknown-linux-gnu --release

• Override gcc and g++, rarely needed.

  CXX=/opt/armv7l-linux-musleabihf-cross/bin/armv7l-linux-musleabihf-g++ CC=/opt/armv7l-linux-musleabihf-cross/bin/armv7l-linux-musleabihf-gcc cargo build --target=armv7-unknown-linux-musleabihf --release

  the ENV can be set directly or via export method

My personal rust builder image look like this

FROM rust:1.65-slim-bullseye
ENV PATH="/opt/aarch64-linux-musl-cross/:/opt/aarch64-linux-musl-cross/bin/:/opt/x86_64-linux-musl-cross/:/opt/x86_64-linux-musl-cross/bin/:$PATH"

# Install build deps and add sccache 
ARG SCCACHE_VERSION
ARG SCCACHE_FILE=sccache-$SCCACHE_VERSION-x86_64-unknown-linux-musl
ARG SCCACHE_URL=https://github.com/mozilla/sccache/releases/download
RUN apt update && \
    apt install -y --no-install-recommends curl git wget build-essential make perl pkg-config curl tar jq musl-tools && \
    apt clean && \
    rm -rf /var/lib/apt/lists/* && \
    curl -L $SCCACHE_URL/$SCCACHE_VERSION/$SCCACHE_FILE.tar.gz | tar xz && \
    mv $SCCACHE_FILE/sccache /usr/local/bin/sccache && \
    chmod +x /usr/local/bin/sccache
### add aarch64 build and aarch64 target
RUN dpkg --add-architecture arm64 && \
    apt update && \
    apt install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libc6-arm64-cross libc6-dev-arm64-cross && \
    apt clean && \
    rm -rf /var/lib/apt/lists/* && \
    rustup target add aarch64-unknown-linux-gnu
### armhf build
#RUN dpkg --add-architecture arm64 && \
#    apt update && \
#    apt install -y gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf libc6-armhf-cross libc6-dev-armhf-cross && \
#    apt clean && \
#    rm -rf /var/lib/apt/lists/*
### add armhf target
#RUN rustup target add rustup target add armv7-unknown-linux-gnueabihf

### Add musl-gcc aarch64 and x86_64
RUN wget --progress=dot:mega -t 10-c https://musl.cc/x86_64-linux-musl-cross.tgz && \
    tar zxf ./x86_64-linux-musl-cross.tgz -C /opt && \
    wget --progress=dot:mega -t 10 -c https://musl.cc/aarch64-linux-musl-cross.tgz && \
    tar zxf ./aarch64-linux-musl-cross.tgz -C /opt && \
    rm ./x86_64-linux-musl-cross.tgz && \
    rm ./aarch64-linux-musl-cross.tgz

## Install nodejs, follow lts path
RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash - && \
    apt update && \
    apt install -y --no-install-recommends nodejs && \
    apt clean && \
    rm -rf /var/lib/apt/lists/* && \
    npm install -g npm && \
    npm install -g yarn && \
    npm install -g pnpm && \
    rm -rf $HOME/.config

CMD ["bash"]

Only x86_64 and aarch64

sources:

• musl cc
• rust openssl build
• CI/CD rust gitlab
• sample config gitlab-ci
• musl time64 issue