This the multi-page printable view of this section.Click here to print.

Return to the regular view of this page.

Setting up a local package repository

Installing and configuring Nexus 3 as a local package repository

Table of Contents

If you have a large number of machines locally, or use Docker builds a lot then running a local package repository can help in reducing both the download times for packages and reduce the bandwidth used both on your internet connection and at the remote repositories.

In this guide I will show how to setup Sonatype Nexus Repository OSS in a docker container and use it as a target for all of your Linux boxes so that when they perform an update they use a local cached version of packages, leaving the retrieval & updating of the cache to Nexus.

1 - Installation

How to install Nexus into docker
First ensure you have docker up and running on the host you want to run this on.

Here we will build a container based on the official sonatype/nexus3 image and then configure systemd to ensure that it is started on startup.

We build our own image as we need to modify how it logs. If we don't do this then nexus's logs will go to the system's own logging and not it's own.

Building the container

Create a new directory and copy the following into a Dockerfile file:

1ARG nexusVersion
2FROM sonatype/nexus3:${nexusVersion}
3
4WORKDIR /opt/sonatype/nexus
5USER root
6RUN echo "-Djava.util.prefs.userRoot=/opt/sonatype/nexus" >>bin/nexus.vmoptions &&\
7    sed -i 's|<appender-ref ref="console"/>||g' etc/logback/logback.xml
8USER nexus

Next run the following to build the image: Substitute 3.36.0 with the current version

1docker build -t nexus:latest --build-arg nexusVersion=3.36.0

What this does is pull the official image for the specified version, modify the userRoot to /opt/sonatype/nexus inside the container and remove the console logging appender. Without the latter the Nexus logs would be echoed to systemd and the host's logging.

Configure SystemD

Next we need to configure systemd, so create /etc/systemd/system/nexus.service with the following:

 1[Unit]
 2Description=Nexus
 3After=network.target, docker.service
 4Require=docker.service
 5
 6[Service]
 7TimeoutStartSec=0
 8Restart=always
 9LimitNOFILE=32767
10LimitNPROC=32767
11Environment=IMAGE=nexus:latest
12ExecStartPre=-/usr/bin/docker stop %n
13ExecStartPre=-/usr/bin/docker rm %n
14ExecStart=/usr/bin/docker run \
15  --name %n \
16  -v /usr/local/nexus:/opt/sonatype/sonatype-work \
17  -e NEXUS_CONTEXT= \
18  -p 8081-8083:8081-8083 \
19  ${IMAGE}
20ExecStop=/usr/bin/docker stop %n
21
22[Install]
23WantedBy=multi-user.target

Here the main parts are:

Line 11 is the image name you used when building the image

Line 16 defines the volume. Here we are going to use /usr/local/nexus on the host to store the repositories and configuration. You can change this to anything you like. Usually it's under /usr/local but put it where you have plenty of space, or on a dedicated volume.

Line 18 lists the ports we want to expose to the network. Normally 8081 is enough for httpd but as we also use Nexus as a docker repository we need additional ports.

Make certain the directory on the host defined on Line 16 exists, then start nexus for the first time:

1sudo mkdir -p /usr/local/nexus
2sudo systemctl daemon-reload
3sudo systemctl start nexus
4tail -F /usr/local/nexus/nexus3/log/nexus.log

If all goes well you should see nexus logging in the nexus3/log/nexus.log file within the volume.

You should then be able to connect to port 8081 on your machine, e.g. http://10.11.12.13:8081/

You must now change the default password! The default user is admin and the password admin123.

To start nexus after a reboot

When you are happy with the setup simply run sudo systemctl enable nexus then nexus will start automatically after a reboot.

Configure for HTTPS

What's described above is good enough for most home lab networks, however ideally you should also configure it for https.

I'm not going to describe that here but I run Traefik as a docker container with Lets Encrypt, IPv6 and access rules to get proper HTTPS certificates working.

The configuration is almost identical as above - as long as the Traefik container can see the ports of the Nexus container then the -p line in the systemd/nexus.service file can be removed so access is only via the proxy.

Other alternatives to Traefik like HAProxy or even Apache HTTPD are available.

2 - Configure Repository mirror

Configure repository mirrors

Now we have a working installation it's time to set up mirrors for package repositories.

Naming conventions

Although you can call the repositories anything you want, it's advisable to keep to a naming convention. The number of repositories will grow over time, and it will become obvious later which repository is for which package manager.

Because of this I advise the following:

  • Keep repository names lower case and use - as a word separator. Never use a space.
  • The first word in a repository name should be the type,
    e.g. apt for an apt repository, npm for a node/npm repository etc.
  • The second word in a repository name should be the distribution.
    For example, apt-debian for Debian, apt-ubuntu for Ubuntu or apt-raspbian for Raspberry PI OS.
  • Any subsequent words in the name would be for specific repository names.
    e.g. apt-debian-security for the Debian security repository.

Keeping to this scheme will make it easier to maintain as well as keeping like repositories next to each other within the Nexus UI.

Object Stores

Each repository mirror requires an object store to store the cached artefacts. It's advisable to create a new object store for each type of repository.

So for any mirror whose name starts with apt-debian then create a keystore called debian and use it as the backing store for all of them. Do the same for Ubuntu, Raspberry PI OS etc.

If you are mirroring third party repositories like NVidia or OBS then you can either create an object store for each or create one called thirdparty and use that for those smaller repositories. The naming is up to you, but it can help in keeping artefacts separate & not have individual large stores.

Configuring a repository mirror

The following subsections list how to configure each repository type:

2.1 - APT repository mirrors

How to configure a repository mirror for APT

APT repository mirrors are easy to configure.

Here we will create a repository for the default Debian repository located at http://deb.debian.org/debian

Create blob store

You only need to do this once per group of repositories.

So for Debian then create a blob store called debian, for Ubuntu then create ubuntu etc. Then for each mirror you create then reuse the same blob store for each group.

  1. Log in to Nexus and go to the admin page via the icon in the top nav bar.
  2. Select blob store, then Create blob store
  3. Under Type select File
  4. Under Name enter debian
  5. Select Create blob store to create it.
Blank create blob panel

Create mirror

  1. Log in to Nexus and go to the admin page via the icon in the top nav bar.
  2. Select Repositories then the Create Repository
  3. Select apt (proxy) as the repository type.
  4. Now fill in the required fields.

    Here we will use Debian as an example:

    1. For name enter apt-debian
    2. For Distribution buster - note this isn't as important as you might think but it is required
    3. For Remote storage enter the repository url http://deb.debian.org/debian
    4. For Blob store select the one you created earlier, debian if you are following my naming conventions.
  5. Once it's all filled in select Save and the Repositories list should now show it.
Blank APT config panel

Common repositories to mirror

The following table lists the repositories mirror covering Debian, Ubuntu and Raspberry PI OS:

Distribution Name Remote storage Blob store Distribution
Debian apt-debian http://deb.debian.org/debian debian buster
apt-debian-security http://security.debian.org/debian-security
Raspian /
Raspberry PI OS
apt-raspbian http://raspbian.raspberrypi.org/raspbian/ raspbian buster
apt-raspbian-archive http://archive.raspberrypi.org/debian/
Ubuntu apt-ubuntu-archive http://gb.archive.ubuntu.com/ubuntu ubuntu focal
apt-ubuntu-ports http://ports.ubuntu.com/ubuntu-ports
For ubuntu my apt-ubuntu-archive proxy goes to gb.archive.ubuntu.com as I'm in the UK. If you are using a different endpoint due to being elsewhere in the world you should be using the same one instead of the GB one.

Using the mirror

Once you have configured your repositories, you can then proceed to configure your machines to use them. This is described in the Using Repository mirrors section.

2.2 - NPM mirror

How to configure a mirror for NPM

Although NPM is used for NodeJS packages rather than operating system ones, I'm including it here as Nexus will happily proxy them as well, improving build times as any downloads are local and not over your internet connection.

Create blob store

Just like for APT we'll create a blobstore just for NPM. This will be called npm to follow the same guidelines.

  1. Log in to Nexus and go to the admin page via the icon in the top nav bar.
  2. Select blob store, then Create blob store
  3. Under Type select File
  4. Under Name enter npm
  5. Select Create blob store to create it.

Create npm mirror

  1. Log in to Nexus and go to the admin page via the icon in the top nav bar.
  2. Select Repositories then the Create Repository
  3. Select npm (proxy) as the repository type.
  4. Now fill in the required fields.

    1. For name enter npm-proxy
    2. For Remote storage enter the repository url https://registry.npmjs.org
    3. For Blob store select the one you created earlier, npm if you are following my naming conventions.
  5. Once it's all filled in select Save and the Repositories list should now show it.

Create npm group

This is optional. If you are going to just mirror the core npm repository then you can skip this and just use npm-proxy directly.

However, Nexus supports grouping multiple repositories together with a group, where if it cannot find an artefact in the first one it would check the next one. This is seamless to the client as all it knows about is there's a single repository.

One use-case for this is if you have a hosted repository where you store your own private artefacts. In this instance you'd create the proxy as above, then an npm-hosted repository. The group would then include both of them as if it's just one.

I'd advise you should create the group with just npm-proxy present as then your npm clients will use the group, and you can then seamlessly change the underlying proxies separately. For example, using an alternate upstream npm repository instead of the default one. You can do the switch in nexus without changing any client.

  1. Log in to Nexus and go to the admin page via the icon in the top nav bar.
  2. Select Repositories then the Create Repository
  3. Select npm (group) as the repository type.
  4. Now fill in the required fields:

    1. For name enter npm-group
    2. Under member repositories you will see two lists.

      On the left hand side select the npm repositories you want to be a member of this group and select >. It should then appear on the right hand side.

      You can change the priority of the entries by selecting the repository name and clicking the up or down arrows. The higher a repository is in the list then they are checked first.

  5. Once it's all filled in select Save and the Repositories list should now show it.

Using the mirror

Once you have configured your repositories, you can then now point npm to use it. This is described in the Using Repository mirrors section.

2.3 - PIP mirror

How to configure a mirror for PIP

PIP is the package manager for Python and Nexus does support acting as a mirror.

Create blob store

Just like for APT and NPM we'll create a blobstore just for PIP. This will be called pip to follow the same guidelines.

  1. Log in to Nexus and go to the admin page via the icon in the top nav bar.
  2. Select blob store, then Create blob store
  3. Under Type select File
  4. Under Name enter pip
  5. Select Create blob store to create it.

Create pip/pypi mirror

  1. Log in to Nexus and go to the admin page via the icon in the top nav bar.
  2. Select Repositories then the Create Repository
  3. Select pypi (proxy) as the repository type.
  4. Now fill in the required fields.

    1. For name enter pypi-proxy
    2. For Remote storage enter the repository url https://pypi.org
    3. For Blob store select the one you created earlier, pip if you are following my naming conventions.
  5. Once it's all filled in select Save and the Repositories list should now show it.

Create pip/pypi group

This is optional. If you are going to just mirror the core pypi repository then you can skip this and just use pypi-proxy directly.

However, Nexus supports grouping multiple repositories together with a group, where if it cannot find an artefact in the first one it would check the next one. This is seamless to the client as all it knows about is there's a single repository.

One use-case for this is if you have a hosted repository where you store your own private artefacts. In this instance you'd create the proxy as above, then a pypi-hosted repository. The group would then include both of them as if it's just one.

I'd advise you should create the group with just pypi-proxy present as then your pip clients will use the group, and you can then seamlessly change the underlying proxies separately. For example, using an alternate upstream pypi repository instead of the default one. You can do the switch in nexus without changing any client.

  1. Log in to Nexus and go to the admin page via the icon in the top nav bar.
  2. Select Repositories then the Create Repository
  3. Select pypi (group) as the repository type.
  4. Now fill in the required fields:

    1. For name enter pypi-group
    2. Under member repositories you will see two lists.

      On the left hand side select the npm repositories you want to be a member of this group and select >. It should then appear on the right hand side.

      You can change the priority of the entries by selecting the repository name and clicking the up or down arrows. The higher a repository is in the list then they are checked first.

  5. Once it's all filled in select Save and the Repositories list should now show it.

Using the mirror

Once you have configured your repositories, you can then now point pip to use it. This is described in the Using Repository mirrors section.

3 - Using Repository mirrors

How to use the repository mirrors

3.1 - APT repository mirrors

How to configure a Linux machine to use APT repository mirrors

Configuring APT to use them

Make certain you make a backup of the files you will be editing below in-case you make a mistake.

Not doing so could cause you not to be able to make any further updates to your machine!

To switch over to using your new proxy you need to edit the apt sources files under /etc/apt/ then run apt-get update.

If you have followed the naming conventions above you can go in to /etc/apt/sources.list and replace any reference to the remote storage url in nexus with the new path.

You should also do this for anything in the /etc/apt/sources.list.d/ directory - but only those you have configured a proxy for!

For example, if your local nexus is accessible at http://10.11.12.13:8081/ then for the apt-ubuntu-archive proxy changing http://gb.archive.ubuntu.com/ubuntu in sources.list with http://10.11.12.13:8081/repository/apt-ubuntu-archive/

For example:

1deb http://gb.archive.ubuntu.com/ubuntu focal main restricted
2deb http://gb.archive.ubuntu.com/ubuntu focal-updates main restricted

becomes:

1deb http://10.11.12.13:8081/repository/apt-ubuntu-archive/ focal main restricted
2deb http://10.11.12.13:8081/repository/apt-ubuntu-archive/ focal-updates main restricted
Do not change the distribution field, e.g. focal-updates above. They should match that of the remote repository. Do not replace them with the value in the proxy!

If like me, you have https configured, then you should be using the correct https address and not the 8081 port.

Now run an update:

1sudo apt-get update

Apt should now access your local proxy. If this completes without any errors then any future upgrade or install will now use the proxy instead.

3.2 - NPM repository mirrors

How to configure NPM use an NPM repository mirror

Configuring NPM to use Nexus as a proxy is simple.

First ensure you've created the NPM proxy.

Next, edit or create the file .npmrc in your home directory with the following line:

1registry=http://10.11.12.13:8081/repository/npm-proxy

Where 10.11.12.13 is where you have deployed Nexus and npm-proxy is what you called your NPM proxy. If you've configured a group then use the group's name instead, e.g. npm-group

Obviously if you've configured https and dns then use that and not port 8081.

Now any time you run npm it will use your proxy.

3.3 - PyPi repository mirrors

How to configure PIP use a PyPi repository mirror

Configuring PIP to use Nexus as a proxy is simple.

First ensure you've created the PyPi proxy.

Next, we need to edit or create pip.conf to hold the new index position. This file can be in one of three locations:

Global
If it exists then ${XDG_CONFIG_DIRS}/pip.conf will be loaded, followed by /etc/pip.conf.
User
${HOME}/.config/pip/pip.conf for the specific user. There is a legacy option of ${HOME}/.pip/pip.conf which will be used if it exists.
Site
For virtual environments then ${VIRTUAL_ENV}/pip.conf

pip.conf

pip.conf should contain the following:

1[global]
2index = http://10.11.12.13:8081/repository/pypi-group/pypi
3index-url = http://10.11.12.13:8081/repository/pypi-group/simple

Where 10.11.12.13 is where you have deployed Nexus and pypi-proxy is what you called your PyPi proxy. If you've configured a group then use the group's name instead, e.g. pypi-group

Obviously if you've configured https and dns then use that and not port 8081.

Now any time you run pip it will use your proxy.

Further reading