In today’s fast-paced development environment, integrating security and providing a streamlined infrastructure automation is very important. In the last blog post, we covered a set of tools that can be used to achieve these goalds.
In this post, we’ll explore how to implement such a DevSecOps approach using, detailing both the GitLab structure and the GitLab Runner setup required to bring this concept to life.
Setting Up Your GitLab Structure
Before diving into an Infrastructure-as-Code (IaC) pipeline, it’s essential to establish a robust GitLab structure. If you use GitLab’s SaaS offering, you don’t need to manage your own GitLab instance—the SaaS model provides a ready-to-use platform. However, on a hosted environment, everything will work the same
Organizational Hierarchy
Imagine a root group (for example, named „GitOps-Shared“) acting as the central hub of your DevOps actvities. This root group is the foundation on which you build the rest of your structure:
- Root Group (GitOps Shared):
This group acts as the container for all your subgroups and projects. It’s here that you register a Group Runner, ensuring that every job triggered in any of the subgroups can be efficiently managed.
- Subgroups and Projects:
Under the root group, you might create an **terraform-modules** sub group to house all Terraform modules and a Container subgroup for all container images. In parallel, a **Pipeline-Collection project** can manage your CI/CD and IaC pipelines. This clear segregation allows teams to easily locate and manage resources while keeping secrets secure through integrated vault solutions.
The following picture shows the structure of such a group:
Centralized Resource Management
Within the root group, you can store essential credentials like certificates, tokens, and service access details as CI/CD variables. Although these variables are accessible to jobs in any subgroup, sensitive information is kept secure by storing only the necessary data and managing actual secrets with tools like Vault.
Configuring the GitLab Runner
The GitLab Runner is a critical component that executes jobs defined in your CI/CD pipelines. Here’s how to set it up:
Deploying the Runner with Kubernetes Executor
1. Choosing a Runner Type:
You’ll deploy a Group Runner at the root group level, which leverages a Kubernetes Executor. This method takes advantage of container orchestration for scalability and efficiency.
2. Using the Helm Chart:
GitLab provides a Helm Chart that simplifies the Runner’s deployment on a Kubernetes cluster. The key values you need to configure in the `values.yaml` file are:
- gitlabUrl: Set this to the full URL of your GitLab server (for GitLab SaaS, the official GitLab URL).
- runnerRegistrationToken: A valid registration token, obtainable from your group settings, is required to link the Runner to your GitLab instance.
3. Customizing Performance Settings:
The Helm chart allows further configuration:
- concurrent: This value determines the maximum number of jobs the Runner can execute simultaneously.
- checkInterval: Defines how frequently (in seconds) the Runner polls GitLab for new jobs.
Adjusting these parameters can help optimize the Runner’s performance, especially if you notice job processing delays.
4. Deploying on Kubernetes:
Once your values are set, you deploy the GitLab Runner using the Helm command:
bash
helm install gitlab-runner gitlab/gitlab-runner –namespace your-namespace –name your-runner –values values.yaml
Verify that the Runner is running by checking the Kubernetes pods and ensuring it appears in your GitLab group settings.
Ensuring Proper Job Handling
By default, GitLab activates shared runners. To ensure your newly deployed Group Runner handles the jobs, disable the shared runners via GitLab’s API or graphical interface. This guarantees that your configuration-specific Runner picks up and executes all relevant jobs.
How the GitLab Runner Works: A Quick Walkthrough
Once your Runner is registered and active, it follows a streamlined process to execute CI/CD jobs:
1. Registration:
The Runner registers with GitLab by sending a POST request containing the registration token. This one-time registration links the Runner to your specific group or project.
2. Job Request Cycle:
The Runner continuously polls GitLab by sending POST requests to `/api/v4/jobs/request`. When a job becomes available, GitLab responds with the job details, including a unique `job_token`.
3. Job Execution:
Using the `job_token`, the Runner instructs its Kubernetes Executor to clone the project repository, download any artifacts from previous jobs, and then execute the job. Throughout the process, the Runner relays status updates and logs back to GitLab.
4. Continuous Operation:
This cycle of job requests, execution, and reporting continues seamlessly until the Runner is manually deactivated, fails, or is removed.
The following picture shows how a GitLab Runner works:
Bringing It All Together
By structuring your GitLab environment with a clear hierarchy and leveraging GitLab SaaS, you set the stage for a smooth DevSecOps implementation in your IaC pipeline. The integration of a Group Runner with Kubernetes Executor ensures that your CI/CD jobs are executed efficiently, while centralized resource management and tailored performance settings keep your pipeline secure and agile.
This approach not only simplifies infrastructure management but also enhances collaboration and security across your development teams. As you adopt and expand on these practices, you’ll likely find that the seamless blend of development, security, and operations truly empowers your software delivery process.
Embracing these detailed steps in your GitLab setup will pave the way for a robust, secure, and efficient DevSecOps pipeline, ready to meet the demands of modern infrastructure management. Happy coding!
