Using Docker in GitLab CI/CD Pipelines

In the modern software development landscape, Continuous Integration and Continuous Deployment (CI/CD) have become essential for maintaining code quality and ensuring rapid delivery cycles. GitLab, as a leading DevOps platform, offers robust CI/CD capabilities that can be significantly enhanced by leveraging Docker. This article explores how Docker can be integrated into GitLab CI/CD pipelines to streamline workflows, improve build consistency, and facilitate deployment across various environments.

Understanding the Basics

What is Docker?

Docker is an open-source platform that automates the deployment, scalingScaling refers to the process of adjusting the capacity of a system to accommodate varying loads. It can be achieved through vertical scaling, which enhances existing resources, or horizontal scaling, which adds additional resources. More », and management of applications using containerization. Containers encapsulate an application and its dependencies, allowing it to run"RUN" refers to a command in various programming languages and operating systems to execute a specified program or script. It initiates processes, providing a controlled environment for task execution. More » consistently across different computing environments. This eliminates the "it works on my machine" problem frequently encountered in software development.

What is GitLab CI/CD?

GitLab CI/CD is a built-in feature of GitLab that helps automate the software development process. It enables developers to build, test, and deploy their code automatically when changes are made. GitLab CI/CD uses a .gitlab-ci.yml file, which contains the configuration for the pipeline, defining the various stages, jobs, and scripts necessary for the CI/CD process.

Benefits of Using Docker with GitLab CI/CD

1. Consistent Environments: Docker ensures that the application runs in the same environment during development, testing, and production. This reduces the chances of discrepancies caused by different configurations.

2. Isolation: Containers provide a level of isolation between different applications and their dependencies, preventing conflicts and ensuring stable builds.

3. Scalability: Using Docker in CI/CD pipelines allows for easy scalingScaling refers to the process of adjusting the capacity of a system to accommodate varying loads. It can be achieved through vertical scaling, which enhances existing resources, or horizontal scaling, which adds additional resources. More » of applications. Containers can be spun up or down quickly, depending on the demand.

4. Simplified Dependency Management: Docker images bundle all dependencies required for an application, simplifying the management of libraries and tools.

5. Faster Build Times: Docker images can be cached, significantly speeding up the build process in CI/CD pipelines.

Setting Up Docker with GitLab CI/CD

Prerequisites

Before diving into the implementation, ensure you have the following:

• A GitLab account and a project where you can set up CI/CD pipelines.
• Docker installed on your local machine for building images.
• Basic knowledge of YAMLYAML (YAML Ain't Markup Language) is a human-readable data serialization format commonly used for configuration files. It emphasizes simplicity and clarity, making it suitable for both developers and non-developers. More » syntax, as the .gitlab-ci.yml file is written in YAMLYAML (YAML Ain't Markup Language) is a human-readable data serialization format commonly used for configuration files. It emphasizes simplicity and clarity, making it suitable for both developers and non-developers. More ».

Step 1: Create a .gitlab-ci.yml File

The first step in setting up a GitLab CI/CD pipeline with Docker is to create a .gitlab-ci.yml file at the root of your repositoryA repository is a centralized location where data, code, or documents are stored, managed, and maintained. It facilitates version control, collaboration, and efficient resource sharing among users. More ». This file dictates how the CI/CD processes will run"RUN" refers to a command in various programming languages and operating systems to execute a specified program or script. It initiates processes, providing a controlled environment for task execution. More ».

Here is a basic example:

imageAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More »: docker:latest

services:
  - docker:dind

stages:
  - build
  - test
  - deploy

variables:
  DOCKER_DRIVER: overlay2

build:
  stage: build
  script:
    - docker build -t my-app:latest .

test:
  stage: test
  script:
    - docker run"RUN" refers to a command in various programming languages and operating systems to execute a specified program or script. It initiates processes, providing a controlled environment for task execution. More » --rm my-app:latest ./run_tests.sh

deploy:
  stage: deploy
  script:
    - docker run"RUN" refers to a command in various programming languages and operating systems to execute a specified program or script. It initiates processes, providing a controlled environment for task execution. More » -d -p 8080:80 my-app:latest

Breakdown of the .gitlab-ci.yml File

• imageAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More »: This specifies the Docker imageAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More » to use for the CI/CD pipeline. Here, we are using the latest Docker imageAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More ».

• services: docker:dind (Docker-in-Docker) allows Docker commands to be executed within the CI/CD environment, enabling you to build and run"RUN" refers to a command in various programming languages and operating systems to execute a specified program or script. It initiates processes, providing a controlled environment for task execution. More » containers.

• stages: Defines the stages of the pipeline: build, test, and deploy.

• variables: Here, we set the DOCKER_DRIVER variable to overlay2, which is the preferred storage driver for Docker.

• Jobs:

  • build: In this job, we build a Docker imageAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More » named my-app using the DockerfileA Dockerfile is a script containing a series of instructions to automate the creation of Docker images. It specifies the base image, application dependencies, and configuration, facilitating consistent deployment across environments. More » in the root of the repositoryA repository is a centralized location where data, code, or documents are stored, managed, and maintained. It facilitates version control, collaboration, and efficient resource sharing among users. More ».
  • test: This job runs tests inside the containerContainers are lightweight, portable units that encapsulate software and its dependencies, enabling consistent execution across different environments. They leverage OS-level virtualization for efficiency. More » created by the previous job using the run_tests.sh script.
  • deploy: Finally, we deploy the application by running the Docker containerContainers are lightweight, portable units that encapsulate software and its dependencies, enabling consistent execution across different environments. They leverage OS-level virtualization for efficiency. More » in detached mode and mapping portA PORT is a communication endpoint in a computer network, defined by a numerical identifier. It facilitates the routing of data to specific applications, enhancing system functionality and security. More » 8080 on the host to portA PORT is a communication endpoint in a computer network, defined by a numerical identifier. It facilitates the routing of data to specific applications, enhancing system functionality and security. More » 80 on the containerContainers are lightweight, portable units that encapsulate software and its dependencies, enabling consistent execution across different environments. They leverage OS-level virtualization for efficiency. More ».

Step 2: Build and Push Docker Images

In many scenarios, you may want to push Docker images to a containerContainers are lightweight, portable units that encapsulate software and its dependencies, enabling consistent execution across different environments. They leverage OS-level virtualization for efficiency. More » registryA registry is a centralized database that stores information about various entities, such as software installations, system configurations, or user data. It serves as a crucial component for system management and configuration. More » after building them. GitLab provides its own containerContainers are lightweight, portable units that encapsulate software and its dependencies, enabling consistent execution across different environments. They leverage OS-level virtualization for efficiency. More » registryA registry is a centralized database that stores information about various entities, such as software installations, system configurations, or user data. It serves as a crucial component for system management and configuration. More », which can be leveraged for this purpose.

To push images, the .gitlab-ci.yml file can be extended as follows:

variables:
  DOCKER_DRIVER: overlay2
  IMAGEAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More »: $CI_REGISTRY/my-app

build:
  stage: build
  script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker build -t $IMAGEAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More »:latest .
    - docker push $IMAGEAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More »:latest

Explanation

• IMAGEAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More »: This variable contains the name of the Docker imageAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More », including the GitLab registryA registry is a centralized database that stores information about various entities, such as software installations, system configurations, or user data. It serves as a crucial component for system management and configuration. More » URL.

• docker login: This command logs into the GitLab containerContainers are lightweight, portable units that encapsulate software and its dependencies, enabling consistent execution across different environments. They leverage OS-level virtualization for efficiency. More » registryA registry is a centralized database that stores information about various entities, such as software installations, system configurations, or user data. It serves as a crucial component for system management and configuration. More » using the CI job token, which allows you to push images to the registryA registry is a centralized database that stores information about various entities, such as software installations, system configurations, or user data. It serves as a crucial component for system management and configuration. More » securely.

• docker push: After building the imageAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More », we push it to the GitLab containerContainers are lightweight, portable units that encapsulate software and its dependencies, enabling consistent execution across different environments. They leverage OS-level virtualization for efficiency. More » registryA registry is a centralized database that stores information about various entities, such as software installations, system configurations, or user data. It serves as a crucial component for system management and configuration. More ».

Step 3: Using Docker Compose

For applications that require multiple services (such as databases, caches, etc.), using Docker ComposeDocker Compose is a tool for defining and running multi-container Docker applications using a YAML file. It simplifies deployment, configuration, and orchestration of services, enhancing development efficiency. More » can simplify orchestrationOrchestration refers to the automated management and coordination of complex systems and services. It optimizes processes by integrating various components, ensuring efficient operation and resource utilization. More ». You can integrate Docker ComposeDocker Compose is a tool for defining and running multi-container Docker applications using a YAML file. It simplifies deployment, configuration, and orchestration of services, enhancing development efficiency. More » in your GitLab CI/CD pipeline as follows:

1. Create a docker-compose.yml file in your project’s root directory.

version: '3'
services:
  web:
    build: .
    ports:
      - "8080:80"
  db:
    imageAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More »: postgres:latest
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password

2. Update your .gitlab-ci.yml file to use Docker ComposeDocker Compose is a tool for defining and running multi-container Docker applications using a YAML file. It simplifies deployment, configuration, and orchestration of services, enhancing development efficiency. More »:

build:
  stage: build
  script:
    - docker-compose build

test:
  stage: test
  script:
    - docker-compose up -d
    - docker-compose exec web ./run_tests.sh
    - docker-compose down

Explanation

• docker-compose build: This command builds all services defined in the docker-compose.yml file.

• docker-compose up -d: Starts the services defined in docker-compose.yml in detached mode.

• docker-compose exec: Runs commands within the running serviceService refers to the act of providing assistance or support to fulfill specific needs or requirements. In various domains, it encompasses customer service, technical support, and professional services, emphasizing efficiency and user satisfaction. More » containerContainers are lightweight, portable units that encapsulate software and its dependencies, enabling consistent execution across different environments. They leverage OS-level virtualization for efficiency. More » (in this case, executing tests).

• docker-compose down: Stops and removes the containers defined in the docker-compose.yml.

Best Practices for Using Docker in GitLab CI/CD

1. Use Multi-Stage Builds: Multi-stage builds can help reduce the size of your Docker images by allowing you to separate the build environment from the runtime environment. This can significantly decrease deployment times and improve security:

# First stage: build
FROM nodeNode, or Node.js, is a JavaScript runtime built on Chrome's V8 engine, enabling server-side scripting. It allows developers to build scalable network applications using asynchronous, event-driven architecture. More »:16 AS build
WORKDIRThe `WORKDIR` instruction in Dockerfile sets the working directory for subsequent instructions. It simplifies path management, as all relative paths will be resolved from this directory, enhancing build clarity. More » /app
COPYCOPY is a command in computer programming and data management that facilitates the duplication of files or data from one location to another, ensuring data integrity and accessibility. More » . .
RUN"RUN" refers to a command in various programming languages and operating systems to execute a specified program or script. It initiates processes, providing a controlled environment for task execution. More » npm install && npm run"RUN" refers to a command in various programming languages and operating systems to execute a specified program or script. It initiates processes, providing a controlled environment for task execution. More » build

# Second stage: production
FROM nginx:alpine
COPYCOPY is a command in computer programming and data management that facilitates the duplication of files or data from one location to another, ensuring data integrity and accessibility. More » --from=build /app/dist /usr/share/nginx/html

2. Use Caching Wisely: To speed up the build process, leverage Docker’s caching mechanisms. For example, ordering your DockerfileA Dockerfile is a script containing a series of instructions to automate the creation of Docker images. It specifies the base image, application dependencies, and configuration, facilitating consistent deployment across environments. More » instructions properly can allow Docker to cache layers effectively.

3. Limit Resource Usage: In CI pipelines, especially when running multiple jobs in parallel, it’s essential to limit resource usage. You can specify resource limits in your jobs:

build:
  stage: build
  script:
    - docker build -t my-app:latest .
  resource_requests:
    memory: 512Mi
    cpu: "1"

4. Cleanup Resources: To avoid using up all available storage and memory on the CI runners, ensure you clean up unused images and containers regularly. You can addThe ADD instruction in Docker is a command used in Dockerfiles to copy files and directories from a host machine into a Docker image during the build process. It not only facilitates the transfer of local files but also provides additional functionality, such as automatically extracting compressed files and fetching remote files via HTTP or HTTPS. More » a job to your .gitlab-ci.yml to remove dangling images:

cleanup:
  stage: cleanup
  script:
    - docker rmi $(docker images -f "dangling=true" -q) || true

5. Use Tags for Versioning: Implement tagging for your Docker images in the CI/CD process. This practice helps maintain version control and makes it easier to roll back to previous versions.

build:
  stage: build
  script:
    - docker build -t $IMAGEAn image is a visual representation of an object or scene, typically composed of pixels in digital formats. It can convey information, evoke emotions, and facilitate communication across various media. More »:$CI_COMMIT_TAG .

Conclusion

Integrating Docker into GitLab CI/CD pipelines offers significant advantages in terms of consistency, speed, and scalability. By following best practices and leveraging Docker’s capabilities, teams can improve their development workflows, enhance testing processes, and streamline deployments. As the software landscape continues to evolve, mastering Docker in conjunction with GitLab CI/CD will remain crucial for organizations aiming for agility and reliability in their development processes.

With these insights and configurations in place, you should be well-prepared to implement Docker within your GitLab CI/CD pipelines effectively, ultimately leading to a more efficient and robust software delivery lifecycle.