Terraform Modules Monorepo On GitLab

| Tags: GitLab, CICD, Terraform, Monorepo
Terraform Modules Monorepo On GitLab

Introduction

After several years of working with GitHub and Azure DevOps on a daily basis, using different tools feels counterintuitive to me. However, one of my clients is deeply integrated with GitLab. Since I was hired to resolve some issues, I saw this as the perfect opportunity to dive deep into GitLab CI and implement a robust, version-controlled approach that supports collaboration while maintaining security and documentation standards.

This guide presents an advanced implementation of a Terraform modules monorepo using GitLab, featuring automated versioning, security scanning, and documentation generation.

The Solution

The solution implements a CI/CD pipeline built on GitLab’s native capabilities. Instead of using GitLab’s built-in “Terraform Modules” functionality, I opted for git tags to avoid additional authentication complexity. This approach eliminates the need for additional tokens, as standard git credentials suffice for module downloads during terraform init.

It consists of a main .gitlab-ci.yml file, a pull request template, and PowerShell scripts within the .gitlab directory. This modular design simplifies pipeline maintenance and allows customization with preferred tools.

You can find the repository with the pipelines code here: Terraform Modules Monorepo on GitLab

Implementation Prerequisites

Before uploading your first module to the monorepo, complete the following steps to ensure a smooth configuration.

  1. Repository Import You can import the repository using GitLab’s import feature, but note that this may be disabled in your organization.. - documentatation

Alternatively, if import is disabled, you can manually clone and push the repository.

  1. Access Token Configuration

Create a project access token with the following settings:

  • Scope: api and write_repository
  • Role Maintainer

Ensure you set nice token name - it will be displayed as comments author name in merge requests later! Setting token permissions

  1. Create a new CI/CD variable

Create a new CI/CD variable containing the access token. Use the key name TerraformModulesAccessToken, which is preconfigured in the .gitlab-ci.yml file. If you choose a different key, update the pipeline variables section accordingly.

variables:
  REPOSITORY_ACCESS_TOKEN: $TerraformModulesAccessToken

Add CICD variable

  1. Adjust merge request settings

Enable fast-forward merging with required squashing and merge checks. These settings ensure the pipeline can identify the related merge request af Add CICD variable

  1. Adjust Terraform Version

Modify the Terraform version in pipeline variables section which is used for init, fmt, and validate checks as needed.

variables:
  TERRAFORM_VERSION: '1.9.4'
  1. (Optional) Configure Labels

Define necessary labels: major, minor, patch, and no-release. If missing, the detection step will create them on the first run.

  1. (Optional) Configure Approver Requirements

For free-tier GitLab users who want to enforce reviews, set a minimum number of required approvals using a pipeline variable. If using a paid version, set this to 0 and use GitLab’s native approval feature.

variables:
  MINIMUM_NUMBER_OF_APPROVALS: 1 # This should be set to zero if using Premium or Ultimate

The Monorepo Flow

Files structure

The logic is configured to search for the changes on second directory level. This means that you have to organize your modules in parent folder. It helps with clarity when using multiple providers:

Terraform Monorepo
├── azure
│   └── vpn
│       └── main.tf
└── aws
    └── vpn
        └── main.tf

repo structure

Commit and Create a Pull Request

Add a changelog entry, which appears in the merge request description and the autogenerated documentation.

Assign a label to classify the change:

  • major - Breaking changes (e.g., modifying default values or adding required variables).
  • minor - Non-breaking new features (e.g., adding a variable with a default value).
  • patch - Bug fixes.
  • no-release - Non-code updates (e.g., documentation or pipeline changes).

Create MR with label

If you forget to assign label during the merge request creation, you can do it anytime and trigger merge request pipeline manually from this view:

pipelines view

Several checks are performed. If you forget the changelog entry or label, the workflow blocks the merge request from merging.

Failure because of label missing

Following the initial check and module detection, the workflow performs the following actions on each modified module:

pipeline run

Linting

Uses the tflint tool and configuration from .tflint.hcl. You may need to add additional plugins to the configuration. Default are terraform and azurerm

Security Analysis

Runs checkov and tfsec to detect security vulnerabilities.

Module Validation

Checks for formatting issues, runs terraform init and terraform validate, and terraform fmt. Workflow fails if any checks fail.

The pipeline labels your merge request with the changed modules.

PR labeled

It provides a release plan comment, detailing the new version numbers for each modified module and changelog information for post-merge autogenerated documentation.

Release plan comment

(optional) Approvals check

For users enforcing approvals via the pipeline, the pipeline or job must be manually rerun after approval to proceed with merging.

You can do it for example from this view: Rerun pipeline

Merge

After merging changes, the pipeline generates and publishes documentation to the repository wiki.

wiki entry

It also publishes a new version of the module by adding a tag to the repository.

Tagged module files

Using the Module from monorepo

To use a module in your Terraform template, reference it like this:

module "module_reference_name" {
  source = "git::https://{server_url}/{group_name}/{project_name}.git?ref={module_name}/v{version}"
}

For example:

module "regions" {
  source = "git::https://gitlab.cloudchronicles.blog/delivery/terraformmodules.git?ref=azure/regions/v1.0.0""
}

For testing changes from a branch during development:

module "regions" {
  source = "git::https://gitlab.cloudchronicles.blog/delivery/terraformmodules.git?//module_name?ref=branch_name"
}

If you intend to keep your modules in a private repository, ensure that your app, identity, or user has the proper permissions to access the repo. Terraform uses default git permissions when checking out modules. In local development, your git credentials will suffice, but in order to allow CI/CD from other projects access your repository. To do so you need to set job token permissins in Settings > CI/CD > Job token permissions and either select All groups and projects or specify repositories that would be granted access with their CI_JOB_TOKEN.

Using CI_JOB_TOKEN to Download Module

If you create a repository in a separate project and use CI_JOB_TOKEN, set up additional permissions for each project needing access.

In your .gitlab-ci-yml file place this line in your “before_script” or “script” section:

git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}".insteadOf "https://${CI_SERVER_HOST}"

Example pipeline:

terraform:
  stage: plan
  image: 
    name: hashicorp/terraform:latest
    entrypoint: [""]
  script:
    - git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}".insteadOf "https://${CI_SERVER_HOST}"
    - terraform init
    - terraform plan