GitLab

GitLab Pipelines

Pipelines serve as the top-level component of CI/CD in GitLab. They consist of two primary elements:

CI vs CD

1. Continuous Integration

CI is the practice of automating the integration of code changes into a shared repository.

Every change triggers automated builds and tests to ensure the new code integrates with the existing codebase.

Key Characteristics:

Developers commit code changes frequently (often daily).

Automated builds and tests verify the code's integrity.

Focuses on detecting and fixing bugs early.

Example:

A team is building a Node.js application. Developers push code changes to the repository multiple times a day.

Each commit triggers a CI pipeline:

Linting: Checks code formatting and style using a tool like ESLint.

Unit Tests: Runs unit tests using Jest.

SonarQube

Check Code Quality : Identify bugs, code smells, and technical debt in the codebase.

Security Vulnerability Detection: Identify security vulnerabilities and weaknesses in the codebase (e.g., SQL injection, hardcoded credentials).

Enforcing Coding Standards: Ensure that developers adhere to organization-specific coding conventions and best practices.

Historical Trend Analysis: Track code quality trends over time.

Build: Packages the app into a Docker image.

Integration Tests: Verifies that modules interact correctly.

2. Continuous Delivery (CD)

Continuous Delivery ensures the application is always in a deployable state.

Deployment to a staging or production environment is manual but easy and reliable due to automation in testing and packaging.

Key Characteristics:

Automates deployments up to a staging or testing environment.

Requires a manual approval process before deploying to production.

Ensures confidence in the stability of each release.

Example:

The Node.js application from the CI pipeline is now ready for deployment to staging:

Run additional end-to-end tests on a staging environment.
Deploy only when a manager or QA approves.

The GitLab pipeline might include:

Deployment to staging:


helm upgrade --install my-app ./helm-chart --values values-staging.yaml

3. Continuous Deployment (CD)

Continuous Deployment goes one step further than Continuous Delivery by automating deployments to production.

Every successful pipeline run results in an update to the production environment, provided all automated tests pass.

Key Characteristics:

Fully automated deployment, without manual approvals.

Ideal for teams practicing rapid iterations and small, frequent releases.

Requires robust testing and monitoring systems.

Example:

After a successful CI pipeline, the Node.js application automatically deploys to production:

Deployment to production happens immediately after tests pass.
No manual intervention is required.

A GitLab pipeline for Continuous Deployment:

Automatically deploys the app using Helm and Kubernetes:


helm upgrade --install my-app ./helm-chart --values values-prod.yaml

Comparison Between CI, Continuous Delivery, and Continuous Deployment

Aspect	Continuous Integration (CI)	Continuous Delivery (CD)	Continuous Deployment (CD)
Focus	Automating testing and integration	Automating deployment to staging	Automating deployment to production
Manual Approval	Not required	Required before production deployment	Not required
Production Deployment	Not included	Manual	Fully automated
Goal	Ensure code changes integrate seamlessly	Ensure deployability at all times	Rapid delivery of new features/bug fixes

When to Use Each Approach?

CI Only:

Small teams or projects where deployments are infrequent.

Focus on code quality and early detection of bugs.

CI + Continuous Delivery:

Teams that want to ensure a reliable release process but require manual validation before production.

Suitable for systems requiring regulatory approval or strict QA.

CI + Continuous Deployment:

Teams that deploy multiple times a day.

Requires robust testing and monitoring systems to prevent issues in production.

Jobs

Define the tasks to be performed. Jobs are executed by GitLab runners. Multiple jobs in the same stage are executed in parallel (depending on the concurrent runners). If all jobs in any particular stage succeed, the pipeline will move ahead to the next stage. If any job in any particular stage fails, the next stage will not be executed and the pipeline will be stopped at that same point.

Stages

Determine when to run the jobs and how to execute them.

Stages, define when to run the jobs and how to run the jobs. We can have different stages for different sections like building, testing, and deploying. A stage can have zero, one, or multiple jobs inside to execute. All jobs in a particular stage run in parallel. The next stage in a pipeline will be executed only if all jobs from the previous stage are completed successfully.

Default Stages: In GitLab, the Default pipeline stages are:

.pre

build

test

deploy

.post

Note:

.pre will always be the first stage, and we cannot change it.

.post will always be the last stage, and we cannot change it as well.

build test deploy > these stages sequence we can change.


build-job:
  stage: build
  script:
    - echo "Hello, $GITLAB_USER_LOGIN!"

test-job1:
  stage: test
  script:
    - echo "This job tests something"

test-job2:
  stage: test
  script:
    - echo "This job tests something, but takes more time than test-job1."
    - echo "After the echo commands complete, it runs the sleep command for 20 seconds"
    - echo "which simulates a test that runs 20 seconds longer than test-job1"
    - sleep 20

deploy-prod:
  stage: deploy
  script:
    - echo "This job deploys something from the $CI_COMMIT_BRANCH branch."
  environment: production

Script

We use script to specify the commands that we want to execute. We can give single-line or multiple-line commands to execute. All the jobs except trigger jobs require a script keyword. CI/CD variables are supported within the script section.

before_script

We can use before_script to add commands that must be executed before the job's script section commands.

We can give single-line or multiple-line commands to execute.

CI/CD variables are supported within the before_script section.

Uses:

Environment Setup: Configure the job's environment before running the main script.

Preparation Tasks: Perform any necessary setup tasks such as downloading dependencies, installing packages, or setting up configurations.

Security Checks: Execute security scans, static code analysis, or any other pre-deployment checks to ensure the code is safe and ready for deployment.

after_script

We can use after_script to add commands that will be executed after each job, including the failed jobs.

We can give single-line or multiple-line commands to execute.

CI/CD variables are supported within the after_script section.

The commands we specify in after_script will be executed in a new shell, which will be separate from any before_script or script commands.

Uses:

Cleanup Tasks: Perform cleanup operations such as removing temporary files, cleaning up resources, or deallocating resources used during the job.

Reporting and Logging: Generate reports, collect logs, or send notifications about the job's execution status.

Post-Processing: Execute tasks like archiving artifacts, generating code coverage reports, or updating deployment statuses after the main script has completed.

Variables

Predefined CI/CD Variables (Automatically provided by GitLab (e.g., CI_COMMIT_SHA, CI_JOB_NAME).

Variables in .gitlab-ci.yml File (Defined directly in your pipeline configuration file under the variables section.)

Project-Level Variables (Configured at the project settings level, accessible to all pipelines within the project.)

Group-Level Variables (Configured at the group settings level, accessible to all projects within the group.)

Variables Set Through the API (Dynamically created or updated via GitLab's API for programmatic workflows.)

Note on Storing Variables in GitLab CI/CD

Variables in .gitlab-ci.yml File:

Use for non-sensitive project configurations like:

Database URLs
Usernames

Avoid storing sensitive data directly in the pipeline configuration file to ensure security.

Sensitive Variables:

Store sensitive data such as:

API tokens
Secrets
Database credentials
SSH keys

These should be saved in Project Settings under CI/CD variables.

Access Control:

Only Project Members with the Maintainer role can add or update CI/CD variables in the project's settings.

This ensures controlled access to sensitive information.

Timeout

We can use a timeout value to configure a timeout for a specific job.

If our job is running for longer than the timeout, the job will fail.

We can configure the job-level timeout longer than the project-level timeout, but it cannot be longer than the runner's timeout.


build:
  script: build.sh
  timeout: 3 hours 30 minutes
test:
  script: rspec
  timeout: 3h 30m

Only and Except

only: Defines the conditions under which a job will run. except: Defines the conditions under which a job will not run. These keywords help control when a job should or should not run based on specified conditions.

Security

Static Application Security Testing (SAST)

Examines source code, bytecode, or binary code without executing the program.

Identifies vulnerabilities and insecure coding practices by analyzing the code structure.

Helps catch issues early in the development process.

Requirements:

SAST runs in the test stage.

To enable SAST jobs, we need to have GitLab Runner with the docker or Kubernetes executor.

Dynamic Application Security Testing (DAST)

Simulates real-world attacks by testing running applications from the outside.

Identifies vulnerabilities that may not be apparent in the source code.

Provides insights into runtime behavior and potential security weaknesses.

Requirements:

GitLab Runner must be available with the docker executor.

Application must be deployed, which needs to be scanned.

dast stage should be added after the deploy job.

Container Security Scans

Focuses on scanning container images for known vulnerabilities.

Analyzes dependencies and libraries within containerized applications.

Ensures that deployed containers do not contain outdated or vulnerable components. Trivy Grype

Requirements:

GitLab CI/CD pipeline must have a test stage.

GitLab Runner required with the docker or Kubernetes executor.

Build and Publish your Docker image to your container registry.

Infrastructure as Code (IaC) Scanning

Evaluates security configurations in infrastructure code.

Ensures that cloud infrastructure and deployment scripts adhere to security best practices.

Mitigates misconfigurations that could lead to security vulnerabilities.

Runners

GitLab Runner is an application that works with GitLab to run jobs in a pipeline. Similar to an agent in Jenkins.

Open-source and written in Go. Can run in your local machine, VM, Container or Pod.

Should be registered with the server.

A GitLab runner clones the project, read the gitlab-ci.yaml file and do what he is instructed to do and finally sends the result back to GitLab Server.

Registering runners with authentication token.


sudo gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.com/" \
  --token "$RUNNER_TOKEN" \
  --executor "docker" \
  --docker-image alpine:latest \
  --description "docker-runner"

Example Pipeline

Pipeline overview

Our CI/CD pipeline, which is currently in the development stage is designed to automate the entire process from code integration to deployment on AWS EKS.

It begins when developers push code to GitLab, which triggers the pipeline defined in the .gitlab-ci.yml file.

The first stage is the build stage, where we use Maven to compile the code and generate a WAR file as the build artifact. (This ensures that the code can compile without errors.)

The next step is code quality analysis, where the pipeline integrates with SonarQube to scan for bugs, vulnerabilities, and code smells. The Sonar server analyzes this and returns a quality gate result. If the quality gate is passed, the pipeline continues; otherwise, it stops (to maintain high code standards.)

Once the quality gate is cleared, the pipeline stores the build artifact in Nexus,( a hosted Maven repository.)

This artifact is versioned for traceability and reusability.

After that, we containerize the application using Kaniko, which is an ideal tool for Kubernetes environments since it doesn’t rely on the Docker daemon. The Docker image is tagged with the Git commit SHA for traceability and pushed to Docker Hub.

For deployment, we use Helm charts to simplify the process. (Helm packages all the Kubernetes manifests into reusable and parameterized templates, allowing us to deploy consistently across environments.)

Depending on whether we’re deploying to staging, testing, or production, we adjust configurations using environment-specific values.yaml files. The application is deployed to our Kubernetes cluster running on AWS EKS.

Finally, once the application is deployed, integration tests are executed to validate the deployment and ensure everything works as expected. This pipeline covers building, testing, artifact creation, containerization, and deployment, making it robust and efficient for delivering high-quality applications.

Maven

We use Maven as our build tool because it efficiently manages Java-based dependencies and automates the build process. It helps in compiling the code and packaging it into a WAR file, ensuring the application is ready for deployment.

SonarQube

SonarQube is essential for maintaining code quality. It scans the source code for bugs, vulnerabilities, and code smells. This ensures that our code adheres to high standards and meets predefined quality gates before moving further in the pipeline.

Nexus

Nexus acts as our artifact repository. It allows us to store, version, and manage build artifacts, such as the WAR file. By storing artifacts centrally, we ensure they are easily retrievable and can be reused in subsequent stages.

Kaniko

Kaniko is used for containerizing the application. Unlike traditional Docker build processes, Kaniko works without requiring access to the Docker daemon, making it secure and well-suited for Kubernetes-native environments. It creates Docker images and pushes them to Docker Hub.

kaniko solves two problems with using the Docker-in-Docker build method:

Docker-in-Docker requires privileged mode to function, which is a significant security concern.

Docker-in-Docker generally incurs a performance penalty and can be quite slow.

Docker Hub

Docker Hub serves as the container image repository. By tagging images with the Git commit SHA, we maintain version control and ensure traceability of every deployment.

Helm

Helm simplifies Kubernetes deployments by packaging Kubernetes manifests into reusable and parameterized templates. It allows us to manage complex Kubernetes applications and adapt configurations for different environments, like staging or production, using values.yaml files.

AWS EKS

We deploy the application on AWS EKS because it is a managed Kubernetes service that ensures scalability, reliability, and integration with other AWS services. EKS offloads cluster management, letting us focus on application deployment.

Integration Tests

Post-deployment, integration tests validate that the deployed application works as expected. These tests ensure that our pipeline delivers a robust, fully functional application to users.

Code


default:
  image: maven:3.8.2-openjdk-11
  tags:
    - k8s

variables:
  M2_EXTRA_OPTIONS: "-s .m2/settings.xml"
  IMAGE_NAME: spring-petclinic
  TAG: $CI_COMMIT_SHA
  
stages:
  - check
  - build
  - sonarscan
  - push
  - dockerize
  - deploy

check-version:
  stage: check
  script:
    - mvn --version

build-job:
  stage: build
  script:
    - echo "Building the WAR file"
    - mvn package
    - ls -l target/*.war
  artifacts:
    paths:
      - target/
    expire_in: 1 week

sonarscan:
  stage: sonarscan
  variables:
    SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" 
    SONAR_HOST_URL: http://174.138.104.191:9000
    SONAR_TOKEN: sqp_f24fe3dc6557611d13db76f464be6b517c5eb5b4
  image:
    name: sonarsource/sonar-scanner-cli:latest
  script:
   - sonar-scanner -Dsonar.qualitygate.wait=true

push-to-nexus:
  stage: push
  script:
    - mvn $M2_EXTRA_OPTIONS deploy

dockerize:
  stage: dockerize
  image:
    name: gcr.io/kaniko-project/executor:v1.9.0-debug
    entrypoint: [""]
  script:
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${USERNAME}" "${PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor --context "${CI_PROJECT_DIR}" --destination "$USERNAME/$IMAGE_NAME:$TAG"

deploy-to-kubernetes:
  stage: deploy
  image: 
    name: kunchalavikram/kubectl_helm_cli:latest  # Helm
    entrypoint: [""]
  before_script:
    - mkdir -p ~/.kube
    - cat "$KUBECONFIG" > ~/.kube/config 
  script:
    - helm upgrade --install petclinic petclinic-chart/