User defined HCL functions

Using interpolation to tag an image with the git sha

As shown in the File definition page, bake supports variable blocks which are assigned to matching environment variables or default values:

# docker-bake.hcl
variable "TAG" {
  default = "latest"
}

group "default" {
  targets = ["webapp"]
}

target "webapp" {
  tags = ["docker.io/username/webapp:${TAG}"]
}

alternatively, in json format:

{
  "variable": {
    "TAG": {
      "default": "latest"
    }
  },
  "group": {
    "default": {
      "targets": ["webapp"]
    }
  },
  "target": {
    "webapp": {
      "tags": ["docker.io/username/webapp:${TAG}"]
    }
  }
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "docker.io/username/webapp:latest"
      ]
    }
  }
}

$ TAG=$(git rev-parse --short HEAD) docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "docker.io/username/webapp:985e9e9"
      ]
    }
  }
}

Using the add function

You can use go-cty stdlib functions. Here we are using the add function.

# docker-bake.hcl
variable "TAG" {
  default = "latest"
}

group "default" {
  targets = ["webapp"]
}

target "webapp" {
  args = {
    buildno = "${add(123, 1)}"
  }
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "buildno": "124"
      }
    }
  }
}

Defining an increment function

It also supports user defined functions. The following example defines a simple an increment function.

# docker-bake.hcl
function "increment" {
  params = [number]
  result = number + 1
}

group "default" {
  targets = ["webapp"]
}

target "webapp" {
  args = {
    buildno = "${increment(123)}"
  }
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "buildno": "124"
      }
    }
  }
}

Only adding tags if a variable is not empty using an notequal

Here we are using the conditional notequal function which is just for symmetry with the equal one.

# docker-bake.hcl
variable "TAG" {default="" }

group "default" {
  targets = [
    "webapp",
  ]
}

target "webapp" {
  context="."
  dockerfile="Dockerfile"
  tags = [
    "my-image:latest",
    notequal("",TAG) ? "my-image:${TAG}": "",
  ]
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "my-image:latest"
      ]
    }
  }
}

Using variables in functions

You can refer variables to other variables like the target blocks can. Stdlib functions can also be called but user functions can’t at the moment.

# docker-bake.hcl
variable "REPO" {
  default = "user/repo"
}

function "tag" {
  params = [tag]
  result = ["${REPO}:${tag}"]
}

target "webapp" {
  tags = tag("v1")
}

$ docker buildx bake --print webapp

{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "user/repo:v1"
      ]
    }
  }
}

Using typed variables

Non-string variables are also accepted. The value passed with env is parsed into suitable type first.

# docker-bake.hcl
variable "FOO" {
  default = 3
}

variable "IS_FOO" {
  default = true
}

target "app" {
  args = {
    v1 = FOO > 5 ? "higher" : "lower" 
    v2 = IS_FOO ? "yes" : "no"
  }
}

$ docker buildx bake --print app

{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "lower",
        "v2": "yes"
      }
    }
  }
}

High-level builds with Bake

This command is experimental.

The design of bake is in early stages, and we are looking for feedback from users.

Buildx also aims to provide support for high-level build concepts that go beyond invoking a single build command. We want to support building all the images in your application together and let the users define project specific reusable build flows that can then be easily invoked by anyone.

BuildKit efficiently handles multiple concurrent build requests and de-duplicating work. The build commands can be combined with general-purpose command runners (for example, make ). However, these tools generally invoke builds in sequence and therefore cannot leverage the full potential of BuildKit parallelization, or combine BuildKit’s output for the user. For this use case, we have added a command called docker buildx bake .

The bake command supports building images from HCL, JSON and Compose files. This is similar to docker compose build , but allowing all the services to be built concurrently as part of a single request. If multiple files are specified they are all read and configurations are combined.

We recommend using HCL files as its experience is more aligned with buildx UX and also allows better code reuse, different target groups and extended features.

Next steps

• File definition
• Configuring builds
• User defined HCL functions
• Defining additional build contexts and linking targets
• Building from Compose file

Create a base image

Most Dockerfiles start from a parent image. If you need to completely control the contents of your image, you might need to create a base image instead. Here’s the difference:

• A parent image is the image that your image is based on. It refers to the contents of the FROM directive in the Dockerfile. Each subsequent declaration in the Dockerfile modifies this parent image. Most Dockerfiles start from a parent image, rather than a base image. However, the terms are sometimes used interchangeably.

• A base image has FROM scratch in its Dockerfile.

This topic shows you several ways to create a base image. The specific process will depend heavily on the Linux distribution you want to package. We have some examples below, and you are encouraged to submit pull requests to contribute new ones.

Create a full image using tar

In general, start with a working machine that is running the distribution you’d like to package as a parent image, though that is not required for some tools like Debian’s Debootstrap, which you can also use to build Ubuntu images.

It can be as simple as this to create an Ubuntu parent image:

$ sudo debootstrap focal focal > /dev/null
$ sudo tar -C focal -c . | docker import - focal

sha256:81ec9a55a92a5618161f68ae691d092bf14d700129093158297b3d01593f4ee3

$ docker run focal cat /etc/lsb-release

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS"

There are more example scripts for creating parent images in the Docker GitHub repository.

Create a simple parent image using scratch

You can use Docker’s reserved, minimal image, scratch , as a starting point for building containers. Using the scratch “image” signals to the build process that you want the next command in the Dockerfile to be the first filesystem layer in your image.

While scratch appears in Docker’s repository on the hub, you can’t pull it, run it, or tag any image with the name scratch . Instead, you can refer to it in your Dockerfile . For example, to create a minimal container using scratch :

# syntax=docker/dockerfile:1
FROM scratch
ADD hello /
CMD ["/hello"]

Assuming you built the “hello” executable example by using the source code at https://github.com/docker-library/hello-world, and you compiled it with the -static flag, you can build this Docker image using this docker build command:

$ docker build --tag hello .

Don’t forget the . character at the end, which sets the build context to the current directory.

Note : Because Docker Desktop for Mac and Docker Desktop for Windows use a Linux VM, you need a Linux binary, rather than a Mac or Windows binary. You can use a Docker container to build it:

$ docker run --rm -it -v $PWD:/build ubuntu:20.04

container# apt-get update && apt-get install build-essential
container# cd /build
container# gcc -o hello -static hello.c

To run your new image, use the docker run command:

$ docker run --rm hello

This example creates the hello-world image used in the tutorials. If you want to test it out, you can clone the image repo.

More resources

There are lots of resources available to help you write your Dockerfile .

• There’s a complete guide to all the instructions available for use in a Dockerfile in the reference section.
• To help you write a clear, readable, maintainable Dockerfile , we’ve also written a Dockerfile best practices guide.
• If your goal is to create a new Docker Official Image, read Docker Official Images.

Build context

The docker build or docker buildx build commands build Docker images from a Dockerfile and a “context”.

A build’s context is the set of files located at the PATH or URL specified as the positional argument to the build command:

$ docker build [OPTIONS] PATH | URL | -
                         ^^^^^^^^^^^^^^

The build process can refer to any of the files in the context. For example, your build can use a COPY instruction to reference a file in the context or a RUN --mount=type=bind instruction for better performance with BuildKit. The build context is processed recursively. So, a PATH includes any subdirectories and the URL includes the repository and its submodules.

PATH context

This example shows a build command that uses the current directory ( . ) as a build context:

$ docker build .
...
#16 [internal] load build context
#16 sha256:23ca2f94460dcbaf5b3c3edbaaa933281a4e0ea3d92fe295193e4df44dc68f85
#16 transferring context: 13.16MB 2.2s done
...

With the following Dockerfile:

# syntax=docker/dockerfile:1
FROM busybox
WORKDIR /src
COPY foo .

And this directory structure:

.
├── Dockerfile
├── bar
├── foo
└── node_modules

The legacy builder sends the entire directory to the daemon, including bar and node_modules directories, even though the Dockerfile does not use them. When using BuildKit, the client only sends the files required by the COPY instructions, in this case foo .

In some cases you may want to send the entire context:

# syntax=docker/dockerfile:1
FROM busybox
WORKDIR /src
COPY . .

You can use a .dockerignore file to exclude some files or directories from being sent:

# .dockerignore
node_modules
bar

Warning

Avoid using your root directory, / , as the PATH for your build context, as it causes the build to transfer the entire contents of your hard drive to the daemon.

URL context

The URL parameter can refer to three kinds of resources:

• Git repositories
• Pre-packaged tarball contexts
• Plain text files

Git repositories

When the URL parameter points to the location of a Git repository, the repository acts as the build context. The builder recursively pulls the repository and its submodules. A shallow clone is performed and therefore pulls down just the latest commits, not the entire history. A repository is first pulled into a temporary directory on your host. After that succeeds, the directory is sent to the daemon as the context. Local copy gives you the ability to access private repositories using local user credentials, VPN’s, and so forth.

Note

If the URL parameter contains a fragment the system will recursively clone the repository and its submodules using a git clone --recursive command.

Git URLs accept a context configuration parameter in the form of a URL fragment, separated by a colon ( : ). The first part represents the reference that Git will check out, and can be either a branch, a tag, or a remote reference. The second part represents a subdirectory inside the repository that will be used as a build context.

For example, run this command to use a directory called docker in the branch container :

$ docker build https://github.com/user/myrepo.git#container:docker

The following table represents all the valid suffixes with their build contexts:

By default .git directory is not kept on Git checkouts. You can set the BuildKit built-in arg BUILDKIT_CONTEXT_KEEP_GIT_DIR=1 to keep it. It can be useful to keep it around if you want to retrieve Git information during your build:

# syntax=docker/dockerfile:1
FROM alpine
WORKDIR /src
RUN --mount=target=. \
  make REVISION=$(git rev-parse HEAD) build

$ docker build --build-arg BUILDKIT_CONTEXT_KEEP_GIT_DIR=1 https://github.com/user/myrepo.git#main

Tarball contexts

If you pass a URL to a remote tarball, the URL itself is sent to the daemon:

$ docker build http://server/context.tar.gz
#1 [internal] load remote build context
#1 DONE 0.2s

#2 copy /context /
#2 DONE 0.1s
...

The download operation will be performed on the host the daemon is running on, which is not necessarily the same host from which the build command is being issued. The daemon will fetch context.tar.gz and use it as the build context. Tarball contexts must be tar archives conforming to the standard tar UNIX format and can be compressed with any one of the xz , bzip2 , gzip or identity (no compression) formats.

Text files

Instead of specifying a context, you can pass a single Dockerfile in the URL or pipe the file in via STDIN . To pipe a Dockerfile from STDIN :

$ docker build - < Dockerfile

With Powershell on Windows, you can run:

Get-Content Dockerfile | docker build -

If you use STDIN or specify a URL pointing to a plain text file, the system places the contents into a file called Dockerfile , and any -f , --file option is ignored. In this scenario, there is no context.

The following example builds an image using a Dockerfile that is passed through stdin. No files are sent as build context to the daemon.

docker build -t myimage:latest -<<EOF
FROM busybox
RUN echo "hello world"
EOF

Omitting the build context can be useful in situations where your Dockerfile does not require files to be copied into the image, and improves the build-speed, as no files are sent to the daemon.

Multi-platform images

Docker images can support multiple platforms, which means that a single image may contain variants for different architectures, and sometimes for different operating systems, such as Windows.

When running an image with multi-platform support, docker automatically selects the image that matches your OS and architecture.

Most of the Docker Official Images on Docker Hub provide a variety of architectures. For example, the busybox image supports amd64 , arm32v5 , arm32v6 , arm32v7 , arm64v8 , i386 , ppc64le , and s390x . When running this image on an x86_64 / amd64 machine, the amd64 variant is pulled and run.

Building multi-platform images

Docker is now making it easier than ever to develop containers on, and for Arm servers and devices. Using the standard Docker tooling and processes, you can start to build, push, pull, and run images seamlessly on different compute architectures. In most cases, you don’t have to make any changes to Dockerfiles or source code to start building for Arm.

BuildKit with Buildx is designed to work well for building for multiple platforms and not only for the architecture and operating system that the user invoking the build happens to run.

When you invoke a build, you can set the --platform flag to specify the target platform for the build output, (for example, linux/amd64 , linux/arm64 , or darwin/amd64 ).

When the current builder instance is backed by the docker-container driver, you can specify multiple platforms together. In this case, it builds a manifest list which contains images for all specified architectures. When you use this image in docker run or docker service , Docker picks the correct image based on the node’s platform.

You can build multi-platform images using three different strategies that are supported by Buildx and Dockerfiles:

1. Using the QEMU emulation support in the kernel
2. Building on multiple native nodes using the same builder instance
3. Using a stage in Dockerfile to cross-compile to different architectures

QEMU is the easiest way to get started if your node already supports it (for example. if you are using Docker Desktop). It requires no changes to your Dockerfile and BuildKit automatically detects the secondary architectures that are available. When BuildKit needs to run a binary for a different architecture, it automatically loads it through a binary registered in the binfmt_misc handler.

For QEMU binaries registered with binfmt_misc on the host OS to work transparently inside containers, they must be statically compiled and registered with the fix_binary flag. This requires a kernel >= 4.8 and binfmt-support >= 2.1.7. You can check for proper registration by checking if F is among the flags in /proc/sys/fs/binfmt_misc/qemu-* . While Docker Desktop comes preconfigured with binfmt_misc support for additional platforms, for other installations it likely needs to be installed using tonistiigi/binfmt image.

$ docker run --privileged --rm tonistiigi/binfmt --install all

Using multiple native nodes provide better support for more complicated cases that are not handled by QEMU and generally have better performance. You can add additional nodes to the builder instance using the --append flag.

Assuming contexts node-amd64 and node-arm64 exist in docker context ls ;

$ docker buildx create --use --name mybuild node-amd64
mybuild
$ docker buildx create --append --name mybuild node-arm64
$ docker buildx build --platform linux/amd64,linux/arm64 .

Finally, depending on your project, the language that you use may have good support for cross-compilation. In that case, multi-stage builds in Dockerfiles can be effectively used to build binaries for the platform specified with --platform using the native architecture of the build node. A list of build arguments like BUILDPLATFORM and TARGETPLATFORM is available automatically inside your Dockerfile and can be leveraged by the processes running as part of your build.

# syntax=docker/dockerfile:1
FROM --platform=$BUILDPLATFORM golang:alpine AS build
ARG TARGETPLATFORM
ARG BUILDPLATFORM
RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log
FROM alpine
COPY --from=build /log /log

Getting started

Run the docker buildx ls command to list the existing builders:

$ docker buildx ls
NAME/NODE  DRIVER/ENDPOINT  STATUS   BUILDKIT PLATFORMS
default *  docker
  default  default          running  20.10.17 linux/amd64, linux/arm64, linux/arm/v7, linux/arm/v6

This displays the default builtin driver, that uses the BuildKit server components built directly into the docker engine, also known as the docker driver.

Create a new builder using the docker-container driver which gives you access to more complex features like multi-platform builds and the more advanced cache exporters, which are currently unsupported in the default docker driver:

$ docker buildx create --name mybuilder --driver docker-container --bootstrap
mybuilder

Switch to the new builder:

$ docker buildx use mybuilder

Note

Alternatively, run docker buildx create --name mybuilder --driver docker-container --bootstrap --use to create a new builder and switch to it using a single command.

And inspect it:

$ docker buildx inspect
Name:   mybuilder
Driver: docker-container

Nodes:
Name:      mybuilder0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Buildkit:  v0.10.4
Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

Now listing the existing builders again, we can see our new builder is registered:

$ docker buildx ls
NAME/NODE     DRIVER/ENDPOINT              STATUS   BUILDKIT PLATFORMS
mybuilder     docker-container
  mybuilder0  unix:///var/run/docker.sock  running  v0.10.4  linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default *     docker
  default     default                      running  20.10.17 linux/amd64, linux/arm64, linux/arm/v7, linux/arm/v6

Example

Test the workflow to ensure you can build, push, and run multi-platform images. Create a simple example Dockerfile, build a couple of image variants, and push them to Docker Hub.

The following example uses a single Dockerfile to build an Alpine image with cURL installed for multiple architectures:

# syntax=docker/dockerfile:1
FROM alpine:3.16
RUN apk add curl

Build the Dockerfile with buildx, passing the list of architectures to build for:

$ docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t <username>/<image>:latest --push .
...
#16 exporting to image
#16 exporting layers
#16 exporting layers 0.5s done
#16 exporting manifest sha256:71d7ecf3cd12d9a99e73ef448bf63ae12751fe3a436a007cb0969f0dc4184c8c 0.0s done
#16 exporting config sha256:a26f329a501da9e07dd9cffd9623e49229c3bb67939775f936a0eb3059a3d045 0.0s done
#16 exporting manifest sha256:5ba4ceea65579fdd1181dfa103cc437d8e19d87239683cf5040e633211387ccf 0.0s done
#16 exporting config sha256:9fcc6de03066ac1482b830d5dd7395da781bb69fe8f9873e7f9b456d29a9517c 0.0s done
#16 exporting manifest sha256:29666fb23261b1f77ca284b69f9212d69fe5b517392dbdd4870391b7defcc116 0.0s done
#16 exporting config sha256:92cbd688027227473d76e705c32f2abc18569c5cfabd00addd2071e91473b2e4 0.0s done
#16 exporting manifest list sha256:f3b552e65508d9203b46db507bb121f1b644e53a22f851185d8e53d873417c48 0.0s done
#16 ...

#17 [auth] <username>/<image>:pull,push token for registry-1.docker.io
#17 DONE 0.0s

#16 exporting to image
#16 pushing layers
#16 pushing layers 3.6s done
#16 pushing manifest for docker.io/<username>/<image>:latest@sha256:f3b552e65508d9203b46db507bb121f1b644e53a22f851185d8e53d873417c48
#16 pushing manifest for docker.io/<username>/<image>:latest@sha256:f3b552e65508d9203b46db507bb121f1b644e53a22f851185d8e53d873417c48 1.4s done
#16 DONE 5.6s

Note

• <username> must be a valid Docker ID and <image> and valid repository on Docker Hub.
• The --platform flag informs buildx to create Linux images for AMD 64-bit, Arm 64-bit, and Armv7 architectures.
• The --push flag generates a multi-arch manifest and pushes all the images to Docker Hub.

Inspect the image using docker buildx imagetools command:

$ docker buildx imagetools inspect <username>/<image>:latest
Name:      docker.io/<username>/<image>:latest
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:f3b552e65508d9203b46db507bb121f1b644e53a22f851185d8e53d873417c48

Manifests:
  Name:      docker.io/<username>/<image>:latest@sha256:71d7ecf3cd12d9a99e73ef448bf63ae12751fe3a436a007cb0969f0dc4184c8c
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64

  Name:      docker.io/<username>/<image>:latest@sha256:5ba4ceea65579fdd1181dfa103cc437d8e19d87239683cf5040e633211387ccf
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64

  Name:      docker.io/<username>/<image>:latest@sha256:29666fb23261b1f77ca284b69f9212d69fe5b517392dbdd4870391b7defcc116
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7

The image is now available on Docker Hub with the tag <username>/<image>:latest . You can use this image to run a container on Intel laptops, Amazon EC2 Graviton instances, Raspberry Pis, and on other architectures. Docker pulls the correct image for the current architecture, so Raspberry PIs run the 32-bit Arm version and EC2 Graviton instances run 64-bit Arm.

The digest identifies a fully qualified image variant. You can also run images targeted for a different architecture on Docker Desktop. For example, when you run the following on a macOS:

$ docker run --rm docker.io/<username>/<image>:latest@sha256:2b77acdfea5dc5baa489ffab2a0b4a387666d1d526490e31845eb64e3e73ed20 uname -m
aarch64

$ docker run --rm docker.io/<username>/<image>:latest@sha256:723c22f366ae44e419d12706453a544ae92711ae52f510e226f6467d8228d191 uname -m
armv7l

In the above example, uname -m returns aarch64 and armv7l as expected, even when running the commands on a native macOS or Windows developer machine.

Support on Docker Desktop

Docker Desktop provides binfmt_misc multi-architecture support, which means you can run containers for different Linux architectures such as arm , mips , ppc64le , and even s390x .

This does not require any special configuration in the container itself as it uses qemu-static from the Docker for Mac VM . Because of this, you can run an ARM container, like the arm32v7 or ppc64le variants of the busybox image.