Building rust with openssl was tricky this error due to missing openssl header files and mismatch target. On GNU target it pretty straighforward by install libssl-dev packageon other hand MUSL target a whole different story from penssl development package not detected, or architecture mismatch, because cross compiling. Then i had it enough, I resort to openssl static build, with static openssl we ensure that the openssl was there and can be statically linked that useful on MUSL target.

Depends on your desired target you may want different openssl compiler

ARG SSL_VER="3.1.0"
ARG SSL_DIR=/opt/lib/openssl-static-gnu
ARG SSL_AARCH64_DIR=/opt/lib/openssl-static-aarch64-gnu
ARG SSL_MUSL_DIR=/opt/lib/openssl-static-musl
ARG SSL_AARCH64_MUSL_DIR=/opt/lib/openssl-static-aarch64-musl

RUN curl -sSL https://www.openssl.org/source/openssl-$SSL_VER.tar.gz | tar xz && \
    cd openssl-$SSL_VER && \
    ./Configure no-zlib no-shared -fPIC --prefix=$SSL_DIR --openssldir=$SSL_DIR/openssl linux-x86_64 && \
    env C_INCLUDE_PATH=$SSL_DIR/include make depend 2> /dev/null && \
    make -j$(nproc) && sudo make install_sw && cd ../ && rm -rf openssl-$SSL_VER

RUN curl -sSL https://www.openssl.org/source/openssl-$SSL_VER.tar.gz | tar xz && \
    cd openssl-$SSL_VER && \
    ./Configure no-zlib no-shared -fPIC --prefix=$SSL_AARCH64_DIR --openssldir=$SSL_AARCH64_DIR/openssl linux-aarch64 && \
    env C_INCLUDE_PATH=$SSL_AARCH64_DIR/include make depend 2> /dev/null && \
    make CC=aarch64-linux-gnu-gcc -j$(nproc) && sudo make install_sw && cd ../ && rm -rf openssl-$SSL_VER

RUN curl -sSL https://www.openssl.org/source/openssl-$SSL_VER.tar.gz | tar xz && \
    cd openssl-$SSL_VER && \
    ./Configure no-zlib no-shared -fPIC --prefix=$SSL_MUSL_DIR --openssldir=$SSL_MUSL_DIR/openssl linux-x86_64 && \
    env C_INCLUDE_PATH=$SSL_MUSL_DIR/include make depend 2> /dev/null && \
    make CC=x86_64-linux-musl-gcc -j$(nproc) && sudo make install_sw && cd ../ && rm -rf openssl-$SSL_VER

RUN curl -sSL https://www.openssl.org/source/openssl-$SSL_VER.tar.gz | tar xz && \
    cd openssl-$SSL_VER && \
    ./Configure no-zlib no-shared -fPIC --prefix=$SSL_AARCH64_MUSL_DIR --openssldir=$SSL_AARCH64_MUSL_DIR/openssl linux-aarch64 && \
    env C_INCLUDE_PATH=$SSL_AARCH64_MUSL_DIR/include make depend 2> /dev/null && \
    make CC=aarch64-linux-musl-gcc -j$(nproc) && sudo make install_sw && cd ../ && rm -rf openssl-$SSL_VER

Notice the CC, it our compiler, for musl compiler get from https://musl.cc , extract and export to PATH.

After building success, before trigger cargo build, we need to set cargo env for each target

# Set openssl envenv per arch split to minimize confusion
ENV X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR=/opt/lib/openssl-static-gnu \
    X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR=/opt/lib/openssl-static-gnu/include \
    X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR=/opt/lib/openssl-static-gnu/lib64 \
    X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_DIR=/opt/lib/openssl-static-musl \
    X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_INCLUDE_DIR=/opt/lib/openssl-static-musl/include \
    X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR=/opt/lib/openssl-static-musl/lib64

ENV AARCH64_UNKNOWN_LINUX_GNU_OPENSSL_DIR=/opt/lib/openssl-static-aarch64-gnu \
    AARCH64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR=/opt/lib/openssl-static-aarch64-gnu/include \
    AARCH64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR=/opt/lib/openssl-static-aarch64-gnu/lib \
    AARCH64_UNKNOWN_LINUX_MUSL_OPENSSL_DIR=/opt/lib/openssl-static-aarch64-musl \
    AARCH64_UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR=/opt/lib/openssl-static-aarch64-musl/lib \
    AARCH64_UNKNOWN_LINUX_MUSL_OPENSSL_INCLUDE_DIR=/opt/lib/openssl-static-aarch64-musl/include

Then we can trigger cargo build without annoyance of openssl-sys, no more missing openssl libaries.