In today’s fast-paced IT world, managing infrastructure as code (IaC) is a game changer. Once you have set up your GitLab runner and the overall GitLab structure, you can start writing code for your IT infrastructure using tools like Terraform.
Terraform vs. Ansible
While both Terraform and Ansible help you manage infrastructure as code, they work in fundamentally different ways. With Ansible, you need to define every step of what should be provisioned. In contrast, Terraform lets you declare the desired state of your infrastructure and then automatically provisions it. This approach ensures that your infrastructure is always reproducible, traceable, transparent, and automated.
The Power of the Terraform State
A key advantage of Terraform is its management of the Terraform state. Every time Terraform creates or modifies resources, it records the details in a state file (usually named terraform.tfstate). This file acts as a ledger, keeping track of all your resources so that Terraform can:
- Create resources with terraform apply
- Update resources when configurations change
- Delete resources with terraform destroy (or when they’re removed from the configuration)
- Rebuild resources if certain attributes require a full replacement due to provider constraints
Without the state file, Terraform wouldn’t know which objects have been provisioned, which could lead to serious configuration issues.
When working with GitLab, GitLab handles the state management for you. If you click on Operate > Terraform states within your project, you can see the current state of your Terraform deployment.
Structuring Your Terraform Modules
A typical Terraform module follows a clear structure, usually comprising:
- Root Directory
- main.tf: The heart of your configuration where provider settings and resource blocks (e.g., for groups, projects) are defined.
- variables.tf: Here, you declare all the variables that let developers customize the modul
- outputs.tf: This file is used to output key information about your infrastructure (similar to return values in programming), with the option to mark sensitive data.
- Test Directory: A dedicated folder that mimics the root directory’s structure. Instead of defining resources, this directory references the module, allowing you to run terraform init, terraform validate, and terraform plan to ensure everything works as expected.
Handling State in CI/CD Environments
Running Terraform locally stores the state file in the local directory. However, in CI/CD pipelines—especially with Kubernetes executors—cached files (including the state file) may be lost between stages (Plan, Apply, Destroy). To overcome this challenge, you can create a backend.tf file that configures a backend (like GitLab-managed storage using an HTTP backend) to securely store the state file across pipeline stages.
Integrating Terraform with GitLab
Terraform isn’t limited to public cloud providers like AWS, Azure, or Google Cloud Platform—it also supports virtualization platforms like VMware, container orchestrators, databases, networks, and storage systems. An excellent example is using Terraform with GitLab to build an infrastructure composed of groups and projects. GitLab even provides its own provider, allowing you to create resources such as:
- gitlab_group:Create groups, including subgroups by referencing a parent group ID.
- gitlab_project: Create projects within a specific group using the namespace_id attribute.
Authenticating with GitLab Tokens
Before you can manage GitLab resources, you need to authenticate using a GitLab token. There are several types of tokens available, each serving a different purpose:
- Personal Access Tokens: Authenticate via GitLab API, repository, and registry, with scoped permissions.
- Impersonation Tokens: Created by an admin to act on behalf of a specific user.
- Project Access Tokens: Limited to a specific project.
- Group Access Tokens: Limited to a specific group.
- Deploy Tokens: Allow access to specific packages and cloning repositories without exposing sensitive data.
- CI/CD Job Tokens: Generated during CI/CD jobs with a temporary lifespan.
These tokens are embedded in the Terraform provider block to authenticate requests to GitLab. For example, the GitLab provider uses the xanzy/go-gitlab library to interact with the GitLab REST API. Other providers, like Vault, offer similar mechanisms, often allowing token configuration via environment variables to keep sensitive data out of the source code.
Automating the Process
Once your module is structured and tested, you can automate the entire lifecycle—from provisioning to eventual destruction—using GitLab’s CI/CD pipelines. This not only minimizes manual intervention but also ensures that your infrastructure remains consistent across different environments and over time.
Conclusion
By integrating Terraform with GitLab CI/CD, you can achieve a robust, automated, and reproducible IT infrastructure. From managing state files and configuring modules to securely handling tokens, this approach provides a scalable solution that meets the modern demands of IT infrastructure management. Whether you’re working in the cloud or on-premises, Terraform’s flexibility and power make it a go-to tool for infrastructure as code.
Happy coding and provisioning!
