This the multi-page printable view of this section.Click here to print.

Return to the regular view of this page.

Docker Mirror

A tool to mirror docker images

Table of Contents

docker-mirror is a simple tool to mirror docker images between repositories, usually from the public docker hub to a local internal repository.

You would do this if you use an image frequently. For example I mirror the golang:alpine as I run a lot of builds using golang to compile multiple projects, so it makes sense to have a local mirror of that image.

This has multiple benefits:

  1. Builds are faster as any pull's occur locally removing the internet or your, usually slower than Ethernet internet connection.
  2. Docker now have limits on how often you can pull an image over a set period of time. If you have an image used across multiple machines this can quickly push you over the limit.
  3. If your internet connection goes down, or the public repository is unavailable you can still work

Prerequisites

Because this utility uses the docker manifest command you need to enable experimental features to the Docker CLI.

You also need to be logged in to the destination repository when running the tool with an account that allows you to push to that repository.

Isn't this the same as docker pull?

In most instances you can do exactly the same thing with the following 3 docker commands: 

1docker pull golang:alpine
2docker tag library/golang:alpine docker.example.com/library/golang:alpine
3docker push docker.example.com/library/golang:alpine

However this works only for images built for the same architecture as the machine running those commands.

If you have a muti-architecture environment, or need to support multiple architectures like amd64 & arm then this is where docker-mirror becomes useful as it will automatically mirror all architectures an image supports.

Doesn't docker already support a local mirror?

Yes it does, however it doesn't work due to a bug with docker #30880 causes it to fail for non-root users.

Where's the source?

The source for this tool is on github at https://github.com/peter-mount/docker-mirror.

1 - Mirroring an Image

How to mirror an image to a local docker repository

To mirror an image simply run the command 

1docker-mirror -d docker.example.com golang:alpine

Where docker.example.com is the url to your local repository and golang:alpine is the image you want to mirror.

Once complete your mirror of the image can be retrieved with the tag docker.example.com/library/golang:alpine

Note about libraries

In the above example the golang:alpine image is pushed as library/golang:alpine. This is because in docker the source image is in the library project, so if an image does not contain a project then it gets the library project inserted during the pull.

If you pulled from a different project then you would get a path you would expect.

For example, mirroring area51/example:latest would mirror it as docker.example.com/area51/example:latest

Flowchart of mirror process
Flow chart showing what docker-mirror does to perform the mirror

2 - Using a mirrored image

How to use a mirrored image

If you've mirrored an image to your local repository then you can just use that instead of the public reference.

For example, if you mirrored the golang:alpine image to docker.example.com then just reference it as docker.example.com/library/golang:alpine

How to use both public & mirrored images in the same Dockerfile

If, like myself, you have a lot of Open Sourced projects you want to have a single Dockerfile which anyone can run without having access to, or even knowledge of your private repository.

The solution to this is to use build arguments.

Here's an example from one of my Dockerfiles (actually from the Dockerfile that builds this actual site): 

1ARG prefix
2FROM ${prefix}golang:alpine AS build

Here we have a build argument prefix and in the FROM line we've prefixed the source image name, so where we want to use golang:alpine we've defined it as ${prefix}golang:alpine.

When someone runs the build, prefix defaults to "" so they get the public golang:alpine image.

However, locally you can run docker build passing the prefix to your local repository: 

1docker build --build-arg prefix=docker.example.com/library/ -t myproject:latest .

As you've now set the prefix to point to your local repository then the build will now use the local docker.example.com/library/golang:alpine image instead of the public one.

3 - Using docker-mirror in a Jenkins pipeline

How to use docker-mirror in a Jenkins pipeline

The following is an example pipeline for Jenkins to perform regular mirrors of common repositories: 

 1node( "docker" ) {
 2    [
 3    "alpine:latest",
 4    "debian:11-slim",
 5    "debian:latest",
 6    "golang:alpine",
 7    "openjdk:11"
 8    ].each {
 9        name -> stage( name ) {
10            sh "docker-mirror -d docker.example.com " + name
11        }
12    }
13}

This pipeline will use a jenkins node called "docker" and will run docker-mirror against each one of the listed images, ensuring that the version on docker.example.com is in sync.