
Using BuildKit for Container Image Building in GitLab CI And Pushing To ECR
Aron Schüler Published
Intro
If you’re still using Kaniko for building container images in your GitLab CI pipelines, it’s time to make a change. Kaniko has been deprecated, and BuildKit has emerged as the modern, efficient alternative for building container images in CI/CD environments. BuildKit offers superior caching capabilities, better performance, and enhanced security features compared to its predecessor. While Gitlab Docs offer examples, I had to spend some time getting our pipelines authenticated for AWS and allowing ECR pushes to work flawlessly.
In this post, I’ll walk you through migrating your GitLab pipelines from Kaniko to BuildKit, specifically focusing on building images, authenticating against AWS, and then pushing them to AWS Elastic Container Registry (ECR). The migration makes sure your pipelines stay up to date with industry standards. It should also bring improvements in build speed and reliability!
Step-by-Step Guide
1. Understanding the BuildKit Setup
BuildKit runs in rootless mode for enhanced security, which means it doesn’t require privileged access to build container images. The key components of our setup include:
- BuildKit rootless image:
moby/buildkit:rootless
- ECR credential helper: For seamless authentication with AWS ECR
- BuildKit daemon: Running in daemonless mode for GitLab CI compatibility
2. Setting Up the Base Job Configuration
First, let’s create a reusable job template that contains all the common configuration. You can reuse this later for different kind of build, e.g. production vs QA builds, or maybe E2E builds.
.build backend:
stage: build
tags:
- medium-runner
variables:
GIT_STRATEGY: fetch
AWS_DEFAULT_REGION: eu-central-1
ECR_REPOSITORY: $ECR_REPOSITORY
ECR_REGISTRY: $ECR_REGISTRY
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
BUILDKITD_FLAGS: --oci-worker-no-process-sandbox
image:
name: moby/buildkit:rootless
entrypoint: [""]
The BUILDKITD_FLAGS: --oci-worker-no-process-sandbox
flag is necessary for running BuildKit in GitLab’s shared runners without requiring additional privileges and something I had missing in my first jobs.
3. Installing and Configuring ECR Authentication
The before_script
section handles the ECR authentication setup:
before_script:
# Install ecr login helper
- mkdir credentials-binaries
- wget -O credentials-binaries/docker-credential-ecr-login https://amazon-ecr-credential-helper-releases.s3.us-east-2.amazonaws.com/0.10.0/linux-amd64/docker-credential-ecr-login
- chmod +x credentials-binaries/docker-credential-ecr-login
- chown 1000:1000 credentials-binaries/docker-credential-*
- export PATH="${CI_PROJECT_DIR}/credentials-binaries:$PATH"
# Create docker config for ECR authentication
- mkdir -p ~/.docker
- |
cat << EOF > ~/.docker/config.json
{
"credHelpers": {
"$ECR_REGISTRY": "ecr-login"
}
}
EOF
This setup downloads the ECR credential helper and configures Docker to use it for authentication, using your AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
for that.
The credential helper also handles token refresh, so you don’t need manual docker login
commands.
4. Building and Pushing Images with BuildKit
The main build logic uses buildctl-daemonless.sh
, which is perfect for CI environments
that don’t have the docker service available:
script:
# Build and push image using BuildKit
- |
buildctl-daemonless.sh build \
--frontend dockerfile.v0 \
--local context=$CI_PROJECT_DIR/backend \
--local dockerfile=$CI_PROJECT_DIR/backend \
--export-cache type=registry,ref=$ECR_REPOSITORY:buildcache \
--import-cache type=registry,ref=$ECR_REPOSITORY:buildcache \
--output type=image,name=$ECR_REPOSITORY:$IMAGE_TAG,push=true
The build uses registry-based caching for subsequent builds and builds+pushes to ECR in a single operation,
based on the backend/
folder in this example and the backend/Dockerfile
.
Of course, this might need adaption in your usecase.
5. Creating Environment-Specific Jobs
Extend the base job for different environments:
build backend test:
extends:
- .build backend
environment:
name: test
variables:
IMAGE_TAG: backend-latest
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- when: never
This pattern allows you to create jobs for different environments (staging, production) or usecases by extending the base job and overriding necessary variables.
6. Required GitLab CI Variables
Ensure these variables are configured in your GitLab project settings:
ECR_REGISTRY
: ECR registry domain (e.g.,123456789.dkr.ecr.eu-central-1.amazonaws.com
)ECR_REPOSITORY
: Full ECR repository URI (e.g.,123456789.dkr.ecr.eu-central-1.amazonaws.com/my-app
) (could reuse $ECR_REGISTRY lol)AWS_ACCESS_KEY_ID
: AWS credentials with ECR push permissionsAWS_SECRET_ACCESS_KEY
: Corresponding AWS secret key
Conclusion
The setup shown here provides a solid foundation that you can customize for your specific needs. Migrating from Kaniko is pretty straightforward with this. BuildKit brings your CI/CD pipeline up to date with current best practices. It also brings benefits like improved build speeds through better caching, enhanced security with rootless builds, and more reliable image building. The ECR integration step is something that I needed to pierce together from different sources, but I’m very happy with the setup and the speed. As you implement this in your own projects, you might want to explore additional BuildKit features like multi-platform builds or custom frontend implementations.
As always, feel free to share your experiences or ask questions about this migration approach down below :-)