Maintenance

Updates

Installing updates is relatively straight forward if you are deploying using docker-compose and our container registry. If not, you are kinda on your own. The update can be done live with minimal downtime!

# Pull updates from the container registry
docker compose pull
# Redeploy the containers
docker compose up -d --force-recreate

This required you the have the latest tag in your docker-compose.yml file for all images. Generally, this is currently the default and recommended way to deploy the services.

Automatic updates

If you want you may also automatically update the services by using our container registry. This can e.g. be done using watchtower, kubernetes or a cronjob.

If you want to use watchtower you can add the following to your docker-compose.yml file:

services:
watchtower:
    image: containrrr/watchtower
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock

Backups and Restore

We supply you with two small scripts which allow you to incrementally backup and restore your database. These can also be run by an automatic workflow or in a cronjob. For the main instance we run a scheduled gitlab workflow.

• ./database/mariaDB_backup.sh - creates a backup of the database and stores it in the backups folder in the mariadb container! This is mounted to the /tmp/backup folder on your local machine. Make sure to mount the folder to a persistent storage in the docker compose file!

• ./database/mariaDB_restore.sh - restores the database from the latest backup. Reads the tmp/backup folder.

This script uses the mariabackup tool which allows for incremental backups.

For an example you can see the .gitlab/backup.yml file in the root directory of the project. This file is used to create a backup of our main instance.

Alternatively you may backup the full docker volume but this will increase the size of the backup significantly, and we do not recommend it.

Logging and metrics

By default each of our containers records its own usage metrics. You may find them within the /logs folder inside the containers. The files are in jsonl format, i.e. contain

Additionally, the nginx container also writes the access log to te logs folder.

We rotate all log files every day and keep the files for 31 days. At the moment this is not configurable. If you have a need to change this let us know.

You may mount the /logs folder to a persistent storage to keep the logs or further process them.

# e.g.
services:
    nginx:
        volumes:
        - ./logs:/logs
    ...

We choose this approach as it is quite lightweight and doesn't require another container running for the only purpose of logging and monitoring.