GitLab

Configure the GitLab chart with UBI-based images | GitLab

Sample values
Known Limitations

Configure the GitLab chart with UBI-based images

GitLab offers Red Hat UBI
versions of its images, allowing you to replace standard images with UBI-based
images. These images use the same tag as standard images with -ubi8 extension.

The GitLab chart uses third-party images that are not based on UBI. These images
are mostly offer external services to GitLab, such as Redis, PostgreSQL, and so on.
If you wish to deploy a GitLab instance that purely based on UBI you must
disable the internal services, and use external deployments or services.

The services that must be disabled and provided externally are:

PostgreSQL
MinIO (Object Store)
Redis

The services must be disabled are:

CertManager (Let’s Encrypt integration)
Prometheus
Grafana
GitLab Runner

Sample values

We provide an example for GitLab chart values in examples/ubi/values.yaml
which can help you to build a pure UBI GitLab deployment.

Known Limitations

Currently there is no UBI version of GitLab Runner. Therefore we disable it.
However, that does not prevent attaching your own runner to your UBI-based
GitLab deployment.
GitLab relies on the official image of Docker Registry which is based on alpine .
At the moment we do not maintain or release a UBI-based version of Registry. Since
this functionality is crucial , we do not disable this service.

Stay Ahead in Today’s Competitive Market!
Unlock your company’s full potential with a Virtual Delivery Center (VDC). Gain specialized expertise, drive seamless operations, and scale effortlessly for long-term success.

Book a Meeting to Avail the Services of GitLab

Goals | GitLab

Scheduler
Helm charts

Goals

We have a few core goals with this initiative:

Easy to scale horizontally
Easy to deploy, upgrade, maintain
Wide support of cloud service providers
Initial support for Kubernetes and Helm, with flexibility to support other
schedulers in the future

Scheduler

We will launch with support for Kubernetes, which is mature and widely supported
across the industry. As part of our design however, we will try to avoid decisions
which will preclude the support of other schedulers. This is especially true for
downstream Kubernetes projects like OpenShift and Tectonic. In the future other
schedulers may also be supported like Docker Swarm and Mesosphere.

We aim to support the scaling and self-healing capabilities of Kubernetes:

Readiness and Health checks to ensure pods are functioning, and if not to recycle them
Tracks to support canary and rolling deployments
Auto-scaling

We will try to leverage standard Kubernetes features:

ConfigMaps for managing configuration. These will then get mapped or passed to
Docker containers
Secrets for sensitive data

Since we might be also using Consul, this may be utilized instead for consistency with other installation methods.

Helm charts

A Helm chart will be created to manage the deployment of each GitLab specific container/service. We will then also include bundled charts to make the overall deployment easier. This is particularly
important for this effort, as there will be significantly more complexity in
the Docker and Kubernetes layers than the all-in-one Omnibus based solutions.
Helm can help to manage this complexity, and provide an easy top level interface
to manage settings via the values.yaml file.

We plan to offer a three tiered set of Helm charts:

Helm chart Structure

Read article

Architecture of Cloud native GitLab Helm charts | GitLab

Architecture of Cloud native GitLab Helm charts

Documentation Organization:

Goals
Architecture
Design Decisions
Resource Usage

Read article

Resource usage | GitLab

Resource Requests
- GitLab Shell
- Webservice
- Sidekiq
- KAS

Resource usage

Resource Requests

All of our containers include predefined resource request values. By default we
have not put resource limits into place. If your nodes do not have excess memory
capacity, one option is to apply memory limits, though adding more memory (or nodes)
would be preferable. (You want to avoid running out of memory on any of your
Kubernetes nodes, as the Linux kernel’s out of memory manager may end essential Kube processes)

In order to come up with our default request values, we run the application, and
come up with a way to generate various levels of load for each service. We monitor the
service, and make a call on what we think is the best default value.

We will measure:

Idle Load - No default should be below these values, but an idle process
isn’t useful, so typically we will not set a default based on this value.
Minimal Load - The values required to do the most basic useful amount of work.
Typically, for CPU, this will be used as the default, but memory requests come with
the risk of the Kernel reaping processes, so we will avoid using this as a memory default.
Average Loads - What is considered average is highly dependent on the installation,
for our defaults we will attempt to take a few measurements at a few of what we
consider reasonable loads. (we will list the loads used). If the service has a pod
autoscaler, we will typically try to set the scaling target value based on these.
And also the default memory requests.
Stressful Task - Measure the usage of the most stressful task the service
should perform. (Not necessary under load). When applying resource limits, try and
set the limit above this and the average load values.
Heavy Load - Try and come up with a stress test for the service, then measure
the resource usage required to do it. We currently don’t use these values for any
defaults, but users will likely want to set resource limits somewhere between the
average loads/stress task and this value.

GitLab Shell

Load was tested using a bash loop calling nohup git clone <project> <random-path-name> in order to have some concurrency.
In future tests we will try to include sustained concurrent load, to better match the types of tests we have done for the other services.

Idle values
- 0 tasks, 2 pods
  - cpu: 0
  - memory: 5M
Minimal Load
- 1 tasks (one empty clone), 2 pods
  - cpu: 0
  - memory: 5M
Average Loads
- 5 concurrent clones, 2 pods
  - cpu: 100m
  - memory: 5M
- 20 concurrent clones, 2 pods
  - cpu: 80m
  - memory: 6M
Stressful Task
- SSH clone the Linux kernel (17MB/s)
  - cpu: 280m
  - memory: 17M
- SSH push the Linux kernel (2MB/s)
  - cpu: 140m
  - memory: 13M
  - Upload connection speed was likely a factor during our tests
Heavy Load
- 100 concurrent clones, 4 pods
  - cpu: 110m
  - memory: 7M
Default Requests
- cpu: 0 (from minimal load)
- memory: 6M (from average load)
- target CPU average: 100m (from average loads)
Recommended Limits
- cpu: > 300m (greater than stress task)
- memory: > 20M (greater than stress task)

Check the troubleshooting documentation
for details on what might happen if gitlab.gitlab-shell.resources.limits.memory is set too low.

Webservice

Webservice resources were analyzed during testing with the
10k reference architecture.
Notes can be found in the Webservice resources documentation.

Sidekiq

Sidekiq resources were analyzed during testing with the
10k reference architecture.
Notes can be found in the Sidekiq resources documentation.

KAS

Until we learn more about our users need, we expect that our users will be using KAS the following way.

Idle values
- 0 agents connected, 2 pods
  - cpu: 10m
  - memory: 55M
Minimal Load :
- 1 agents connected, 2 pods
  - cpu: 10m
  - memory: 55M
Average Load : 1 agent is connected to the cluster.
- 5 agents connected, 2 pods
  - cpu: 10m
  - memory: 65M
Stressful Task :
- 20 agents connected, 2 pods
  - cpu: 30m
  - memory: 95M
Heavy Load :
- 50 agents connected, 2 pods
  - cpu: 40m
  - memory: 150M
Extra Heavy Load :
- 200 agents connected, 2 pods
  - cpu: 50m
  - memory: 315M

The KAS resources defaults set by this chart are more than enough to handle even the 50 agents scenario.
If you are planning to reach what we consider an Extra Heavy Load , then you should consider tweaking the
default to scale up.

Defaults : 2 pods, each with
- cpu: 100m
- memory: 100M

For more information on how these numbers were calculated, see the
issue discussion.

Read article

Architecture | GitLab

Docker Container Images
- GitLab Docker Images
- Official Docker Images
The GitLab chart
- Structure of these charts
- Components list
Design Decisions

Architecture

We plan to support three tiers of components:

Docker Containers
Scheduler (Kubernetes)
Higher level configuration tool (Helm)

The main method customers would use to install would be the Helm chart in this repository.
At some point in the future, we may also offer other deployment methods like
Amazon CloudFormation or Docker Swarm.

Docker Container Images

As a foundation, we will be creating a Docker container for each service.
This will allow easier horizontal scaling with reduced image size and complexity.
Configuration should be passed in a standard way for Docker, perhaps environment
variables or a mounted file. This provides a clean common interface with the
scheduler software.

GitLab Docker Images

The GitLab application is built using Docker images that contain GitLab
specific services. The build environments for these images can be found in
the CNG repository.

The following GitLab components have images in the CNG repository.

Gitaly
GitLab Elasticsearch Indexer
mail_room
GitLab Exporter
GitLab Shell
Sidekiq
GitLab Toolbox
Webservice
Workhorse

The following are forked charts which also use GitLab specific Docker images.

Docker images that are used for initContainers and various Job s.

alpine-certificates
kubectl

Official Docker Images

We leverage the following existing official containers for
underlying services:

Docker Distribution (Docker Registry 2.0)
Prometheus
NGINX Ingress
cert-manager
Redis
PostgreSQL
Grafana

The GitLab chart

This is the top level GitLab chart ( gitlab ), which configures all necessary resources
for a complete configuration of GitLab. This includes GitLab, PostgreSQL, Redis,
Ingress, and certificate management charts.

At this high level, a customer can make decisions like:

Whether they want to use the embedded PostgreSQL chart, or to use an external
database like Amazon RDS for PostgreSQL.
To bring their own SSL certificates, or leverage Let’s Encrypt.
To use a load balancer, or a dedicated Ingress.

Customers who would like to get started quickly and easily should begin with this chart.

Structure of these charts

The main GitLab chart is an umbrella chart, made up of many other charts. Each sub-chart is
documented individually, and laid in a structure that matches the
charts directory structure.

Non-GitLab components are packaged and documented on the top level. GitLab
component services are documented under the GitLab chart:

NGINX
MinIO
Registry
GitLab/Gitaly
GitLab/GitLab Exporter
GitLab/GitLab Grafana
GitLab/GitLab Shell
GitLab/Migrations
GitLab/Sidekiq
GitLab/Webservice

Components list

A list of which components are deployed when using the chart, and configuration instructions if needed,
is available on the architecture components list page.

Design Decisions

Documentation of the decisions made regarding the architecture of these charts can
be found in Design Decisions documentation

Read article

Backup and restore | GitLab

Toolbox pod
Backup utility
- Backups
  - Sequence of execution
  - Command line arguments
  - GitLab backup bucket
  - Backing up to Google Cloud Storage
- Restore

Backup and restore

This document explains the technical implementation of the backup and restore into/from CNG.

Toolbox pod

The toolbox chart deploys a pod into the cluster. This pod will act as an entry point for interaction with other containers in the cluster.

Using this pod user can run commands using kubectl exec -it <pod name> -- <arbitrary command>

The Toolbox runs a container from the Toolbox image.

The image contains some custom scripts that are to be called as commands by the user. Those scripts are for running Rake tasks, backup, restore, and some helper scripts for interacting with object storage.

Backup utility

Backup utility is one of the scripts
in the task runner container and as the name suggests it is a script used for doing backups but also handles restoring of an existing backup.

Backups

The backup utility script when run without any arguments creates a backup tar and uploads it to object storage.

Sequence of execution

Backups are made using the following steps, in order:

Backup the database (if not skipped) using the GitLab backup Rake task
Backup the repositories (if not skipped) using the GitLab backup Rake task
For each of the object storage backends
1. If the object storage backend is marked for skipping, skip this storage backend.
2. Tar the existing data in the corresponding object storage bucket naming it <bucket-name>.tar
3. Move the tar to the backup location on disk
Write a backup_information.yml file which contains some metadata identifying the version of GitLab, the time of the backup and the skipped items.
Create a tar file containing individual tar files along with backup_information.yml
Upload the resulting tar file to object storage gitlab-backups bucket.

Command line arguments

--skip <component>

You can skip parts of the backup process by using --skip <component> for every component that you want to skip in the backup process. Skippable components are the database ( db ), repositories ( repositories ), and any of the object storages ( registry , uploads , artifacts , lfs , packages , external_diffs , terraform_state , or ci_secure_files ).
-t <timestamp-override-value>

This gives you partial control over the name of the backup: when you specify this flag the created backup will be named <timestamp-override-value>_gitlab_backup.tar . The default value is the current UNIX timestamp, postfixed with the current date formatted to YYYY_mm_dd .
--backend <backend>

Configures the object storage backend to use for backups. Can be either s3 or gcs . Default is s3 .
--storage-class <storage-class-name>

It is also possible to specify the storage class in which the backup is stored using --storage-class <storage-class-name> , allowing you to save on backup storage costs. If unspecified, this will use the default of the storage backend.

This storage class name is passed through as-is to the storage class argument of your specified backend.

GitLab backup bucket

The default name of the bucket that will be used to store backups is gitlab-backups . This is configurable
using the BACKUP_BUCKET_NAME environment variable.

Backing up to Google Cloud Storage

By default, the backup utility uses s3cmd to upload and download artifacts from object storage. While this can work with Google Cloud Storage (GCS),
it requires using the Interoperability API which makes undesirable compromises to authentication and authorization. When using Google Cloud Storage
for backups you can configure the backup utility script to use the Cloud Storage native CLI, gsutil , to do the upload and download
of your artifacts by setting the BACKUP_BACKEND environment variable to gcs .

Restore

The backup utility when given an argument --restore attempts to restore from an existing backup to the running instance. This
backup can be from either an Omnibus GitLab or a CNG Helm chart installation given that both the instance that was
backed up and the running instance runs the same version of GitLab. The restore expects a file in backup bucket using -t <backup-name> or a remote URL using -f <url> .

When given a -t parameter it looks into backup bucket in object storage for a backup tar with such name. When
given a -f parameter it expects that the given URL is a valid URI of a backup tar in a location accessible from the container.

After fetching the backup tar the sequence of execution is:

For repositories and database run the GitLab backup Rake task
For each of object storage backends:
- tar the existing data in the corresponding object storage bucket naming it <backup-name>.tar
- upload it to tmp bucket in object storage
- clean up the corresponding bucket
- restore the backup content into the corresponding bucket

If the restore fails, the user will need to revert to previous backup using data in

tmp

directory of the backup bucket. This is currently a manual process.

Read article

Welcome to Knowledge Base!

KB at your finger tips

GitLab