Welcome to Knowledge Base!

KB at your finger tips

Book a Meeting to Avail the Services of Docker overtime

This is one stop global knowledge base where you can learn about all the products, solutions and support features.

Categories
All

Docker

Introduction to GitHub Actions

Introduction to GitHub Actions

GitHub Actions is a popular CI/CD platform for automating your build, test, and deployment pipeline. Docker provides a set of official GitHub Actions for you to use in your workflows. These official actions are reusable, easy-to-use components for building, annotating, and pushing images.

The following GitHub Actions are available:

  • Build and push Docker images: build and push Docker images with BuildKit.
  • Docker Login: sign in to a Docker registry.
  • Docker Setup Buildx: initiates a BuildKit builder.
  • Docker Metadata action: extracts metadata from Git reference and GitHub events.
  • Docker Setup QEMU: installs QEMU static binaries for multi-arch builds.
  • Docker Buildx Bake: enables using high-level builds with Bake.

Using Docker’s actions provides an easy-to-use interface, while still allowing flexibility for customizing build parameters.

Get started with GitHub Actions

This tutorial walks you through the process of setting up and using Docker GitHub Actions for building Docker images, and pushing images to Docker Hub. You will complete the following steps:

  1. Create a new repository on GitHub.
  2. Define the GitHub Actions workflow.
  3. Run the workflow.

To follow this tutorial, you need a Docker ID and a GitHub account.

Step one: Create the repository

Create a GitHub repository and configure the Docker Hub secrets.

  1. Create a new GitHub repository using this template repository.

    The repository contains a simple Dockerfile, and nothing else. Feel free to use another repository containing a working Dockerfile if you prefer.

  2. Open the repository Settings , and go to Secrets > Actions .

  3. Create a new secret named DOCKER_HUB_USERNAME and your Docker ID as value.

  4. Create a new Personal Access Token (PAT) for Docker Hub. You can name this token clockboxci .

  5. Add the PAT as a second secret in your GitHub repository, with the name DOCKER_HUB_ACCESS_TOKEN .

With your repository created, and secrets configured, you’re now ready for action!

Step two: Set up the workflow

Set up your GitHub Actions workflow for building and pushing the image to Docker Hub.

  1. Go to your repository on GitHub and then select the Actions tab.
  2. Select set up a workflow yourself .

    This takes you to a page for creating a new GitHub actions workflow file in your repository, under .github/workflows/main.yml by default.

  3. In the editor window, copy and paste the following YAML configuration.

    name: ci
    
    on:
      push:
        branches:
          - "main"
    
    jobs:
      build:
        runs-on: ubuntu-latest
    
    • name : the name of this workflow.
    • on.push.branches : specifies that this workflow should run on every push event for the branches in the list.
    • jobs : creates a job ID ( build ) and declares the type of machine that the job should run on.

For more information about the YAML syntax used here, see Workflow syntax for GitHub Actions.

Step three: Define the workflow steps

Now the essentials: what steps to run, and in what order to run them.

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      -
        name: Checkout
        uses: actions/checkout@v3
      -
        name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      -
        name: Build and push
        uses: docker/build-push-action@v3
        with:
          context: .
          file: ./Dockerfile
          push: true
          tags: ${{ secrets.DOCKER_HUB_USERNAME }}/clockbox:latest

The previous YAML snippet contains a sequence of steps that:

  1. Checks out the repository on the build machine.
  2. Signs in to Docker Hub, using the Docker Login action and your Docker Hub credentials.
  3. Creates a BuildKit builder instance using the Docker Setup Buildx action.
  4. Builds the container image and pushes it to the Docker Hub repository, using Build and push Docker images.

    The with key lists a number of input parameters that configures the step:

    • context : the build context.
    • file : filepath to the Dockerfile.
    • push : tells the action to upload the image to a registry after building it.
    • tags : tags that specify where to push the image.

Add these steps to your workflow file. The full workflow configuration should look as follows:

name: ci

on:
  push:
    branches:
      - "main"

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      -
        name: Checkout
        uses: actions/checkout@v3
      -
        name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      -
        name: Build and push
        uses: docker/build-push-action@v3
        with:
          context: .
          file: ./Dockerfile
          push: true
          tags: ${{ secrets.DOCKER_HUB_USERNAME }}/clockbox:latest

Run the workflow

Save the workflow file and run the job.

  1. Select Start commit and push the changes to the main branch.

    After pushing the commit, the workflow starts automatically.

  2. Go to the Actions tab. It displays the workflow.

    Selecting the workflow shows you the breakdown of all the steps.

  3. When the workflow is complete, go to your repositories on Docker Hub.

    If you see the new repository in that list, it means the GitHub Actions successfully pushed the image to Docker Hub!

Next steps

This tutorial has shown you how to create a simple GitHub Actions workflow, using the official Docker actions, to build and push an image to Docker Hub.

There are many more things you can do to customize your workflow to better suit your needs. To learn more about some of the more advanced use cases, take a look at the advanced examples, such as building multi-platform images, or using cache storage backends and also how to configure your builder.


Stay Ahead in Today’s Competitive Market!
Unlock your company’s full potential with a Virtual Delivery Center (VDC). Gain specialized expertise, drive seamless operations, and scale effortlessly for long-term success.

Book a Meeting to Avail the Services of Dockerovertime

Docker driver

Docker driver

The Buildx Docker driver is the default driver. It uses the BuildKit server components built directly into the Docker engine. The Docker driver requires no configuration.

Unlike the other drivers, builders using the Docker driver can’t be manually created. They’re only created automatically from the Docker context.

Images built with the Docker driver are automatically loaded to the local image store.

Synopsis

# The Docker driver is used by buildx by default
docker buildx build .

It’s not possible to configure which BuildKit version to use, or to pass any additional BuildKit parameters to a builder using the Docker driver. The BuildKit version and parameters are preset by the Docker engine internally.

If you need additional configuration and flexibility, consider using the Docker container driver.

Further reading

For more information on the Docker driver, see the buildx reference.

Read article

Local and tar exporters

Local and tar exporters

The local and tar exporters output the root filesystem of the build result into a local directory. They’re useful for producing artifacts that aren’t container images.

  • local exports files and directories.
  • tar exports the same, but bundles the export into a tarball.

Synopsis

Build a container image using the local exporter:

$ docker buildx build --output type=local[,parameters] .
$ docker buildx build --output type=tar[,parameters] .

The following table describes the available parameters:

Parameter Type Default Description
dest String  Path to copy files to

Further reading

For more information on the local or tar exporters, see the BuildKit README.

Read article

Continuous integration with Docker

Continuous integration with Docker

Continuous Integration (CI) is the part of the development process where you’re looking to get your code changes merged with the main branch of the project. At this point, development teams run tests and builds to vet that the code changes don’t cause any unwanted or unexpected behaviors.

Git branches about to get merged

There are several uses for Docker at this stage of development, even if you don’t end up packaging your application as a container image.

Docker as a build environment

Containers are reproducible, isolated environments that yield predictable results. Building and testing your application in a Docker container makes it easier to prevent unexpected behaviors from occurring. Using a Dockerfile, you define the exact requirements for the build environment, including programming runtimes, operating system, binaries, and more.

Using Docker to manage your build environment also eases maintenance. For example, updating to a new version of a programming runtime can be as simple as changing a tag or digest in a Dockerfile. No need to SSH into a pet VM to manually reinstall a newer version and update the related configuration files.

Additionally, just as you expect third-party open source packages to be secure, the same should go for your build environment. You can scan and index a builder image, just like you would for any other containerized application.

The following links provide instructions for how you can get started using Docker for building your applications in CI:

  • GitHub Actions
  • GitLab
  • Circle CI
  • Render

Docker in Docker

You can also use a Dockerized build environment to build container images using Docker. That is, your build environment runs inside a container which itself is equipped to run Docker builds. This method is referred to as “Docker in Docker”.

Docker provides an official Docker image that you can use for this purpose.

What’s next

Docker maintains a set of official GitHub Actions that you can use to build, annotate, and push container images on the GitHub Actions platform. See Introduction to GitHub Actions to learn more and get started.

Read article

Docker container driver

Docker container driver

The buildx Docker container driver allows creation of a managed and customizable BuildKit environment in a dedicated Docker container.

Using the Docker container driver has a couple of advantages over the default Docker driver. For example:

  • Specify custom BuildKit versions to use.
  • Build multi-arch images, see QEMU
  • Advanced options for cache import and export

Synopsis

Run the following command to create a new builder, named container , that uses the Docker container driver:

$ docker buildx create \
  --name container \
  --driver=docker-container \
  --driver-opt=[key=value,...]
container

The following table describes the available driver-specific options that you can pass to --driver-opt :

Parameter Type Default Description
image String  Sets the image to use for running BuildKit.
network String  Sets the network mode for running the BuildKit container.
cgroup-parent String /docker/buildx Sets the cgroup parent of the BuildKit container if Docker is using the cgroupfs driver.
env.<key> String  Sets the environment variable key to the specified value in the BuildKit container.

Usage

When you run a build, Buildx pulls the specified image (by default, moby/buildkit ){:target=”blank” rel=”noopener” class=””}. When the container has started, Buildx submits the build submitted to the containerized build server.

$ docker buildx build -t <image> --builder=container .
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 1.9s done
#1 creating container buildx_buildkit_container0
#1 creating container buildx_buildkit_container0 0.5s done
#1 DONE 2.4s
...

Loading to local image store

Unlike when using the default docker driver, images built with the docker-container driver must be explicitly loaded into the local image store. Use the --load flag:

$ docker buildx build --load -t <image> --builder=container .
...
 => exporting to oci image format                                                                                                      7.7s
 => => exporting layers                                                                                                                4.9s
 => => exporting manifest sha256:4e4ca161fa338be2c303445411900ebbc5fc086153a0b846ac12996960b479d3                                      0.0s
 => => exporting config sha256:adf3eec768a14b6e183a1010cb96d91155a82fd722a1091440c88f3747f1f53f                                        0.0s
 => => sending tarball                                                                                                                 2.8s
 => importing to docker

The image becomes available in the image store when the build finishes:

$ docker image ls
REPOSITORY                       TAG               IMAGE ID       CREATED             SIZE
<image>                          latest            adf3eec768a1   2 minutes ago       197MB

Cache persistence

The docker-container driver supports cache persistence, as it stores all the BuildKit state and related cache into a dedicated Docker volume.

To persist the docker-container driver’s cache, even after recreating the driver using docker buildx rm and docker buildx create , you can destroy the builder using the --keep-state flag:

For example, to create a builder named container and then remove it while persisting state:

# setup a builder
$ docker buildx create --name=container --driver=docker-container --use --bootstrap
container
$ docker buildx ls
NAME/NODE       DRIVER/ENDPOINT              STATUS   BUILDKIT PLATFORMS
container *     docker-container
  container0    desktop-linux                running  v0.10.5  linux/amd64
$ docker volume ls
DRIVER    VOLUME NAME
local     buildx_buildkit_container0_state

# remove the builder while persisting state
$ docker buildx rm --keep-state container
$ docker volume ls
DRIVER    VOLUME NAME
local     buildx_buildkit_container0_state

# the newly created driver with the same name will have all the state of the previous one!
$ docker buildx create --name=container --driver=docker-container --use --bootstrap
container

QEMU

The docker-container driver supports using QEMU (user mode) to build non-native platforms. Use the --platform flag to specify which architectures that you want to build for.

For example, to build a Linux image for amd64 and arm64 :

$ docker buildx build \
  --builder=container \
  --platform=linux/amd64,linux/arm64 \
  -t <registry>/<image> \
  --push .

Warning

QEMU performs full-system emulation of non-native platforms, which is much slower than native builds. Compute-heavy tasks like compilation and compression/decompression will likely take a large performance hit.

Custom network

You can customize the network that the builder container uses. This is useful if you need to use a specific network for your builds.

For example, let’s create a network named foonet :

$ docker network create foonet

Now create a docker-container builder that will use this network:

$ docker buildx create --use \
  --name mybuilder \
  --driver docker-container \
  --driver-opt "network=foonet"

Boot and inspect mybuilder :

$ docker buildx inspect --bootstrap

Inspect the builder container and see what network is being used:

$ docker inspect buildx_buildkit_mybuilder0 --format={{.NetworkSettings.Networks}}
map[foonet:0xc00018c0c0]

Further reading

For more information on the Docker container driver, see the buildx reference.

Read article

Drivers overview

Drivers overview

Buildx drivers are configurations for how and where the BuildKit backend runs. Driver settings are customizable and allows fine-grained control of the builder. Buildx supports the following drivers:

  • docker : uses the BuildKit library bundled into the Docker daemon.
  • docker-container : creates a dedicated BuildKit container using Docker.
  • kubernetes : creates BuildKit pods in a Kubernetes cluster.
  • remote : connects directly to a manually managed BuildKit daemon.

Different drivers support different use cases. The default docker driver prioritizes simplicity and ease of use. It has limited support for advanced features like caching and output formats, and isn’t configurable. Other drivers provide more flexibility and are better at handling advanced scenarios.

The following table outlines some differences between drivers.

Feature docker docker-container kubernetes remote
Automatically load image ✠  Â
Cache export Inline only ✠✠âœ
Tarball output  ✠✠âœ
Multi-arch images  ✠✠âœ
BuildKit configuration  ✠✠Managed externally

List available builders

Use docker buildx ls to see builder instances available on your system, and the drivers they’re using.

$ docker buildx ls
NAME/NODE       DRIVER/ENDPOINT      STATUS   BUILDKIT PLATFORMS
default         docker
  default       default              running  20.10.17 linux/amd64, linux/386

Depending on your setup, you may find multiple builders in your list that use the Docker driver. For example, on a system that runs both a manually installed version of dockerd, as well as Docker Desktop, you might see the following output from docker buildx ls :

NAME/NODE       DRIVER/ENDPOINT STATUS  BUILDKIT PLATFORMS
default         docker
  default       default         running 20.10.17 linux/amd64, linux/386
desktop-linux * docker
  desktop-linux desktop-linux   running 20.10.17 linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

This is because the Docker driver builders are automatically pulled from the available Docker Contexts. When you add new contexts using docker context create , these will appear in your list of buildx builders.

The asterisk ( * ) next to the builder name indicates that this is the selected builder which gets used by default, unless you specify a builder using the --builder option.

Create a new builder

Use the docker buildx create command to create a builder, and specify the driver using the --driver option.

$ docker buildx create --name=<builder-name> --driver=<driver> --driver-opt=<driver-options>

This creates a new builder instance with a single build node. After creating a new builder you can also append new nodes to it.

To use a remote node for your builders, you can set the DOCKER_HOST environment variable or provide a remote context name when creating the builder.

Switch between builders

To switch between different builders, use the docker buildx use <name> command. After running this command, the build commands will automatically use this builder.

What’s next

Read about each of the Buildx drivers to learn about how they work and how to use them:

  • Docker driver
  • Docker container driver
  • Kubernetes driver
  • Remote driver
Read article