Terraform Setup

The terraform-setup job is used to prepare the Terraform infrastructure code for deployment. It sets up SSH authentication, clones the Terraform module repository, and prepares the working directory.

Overview

This job:

• Sets up SSH agent and authentication keys
• Clones the Terraform module repository from GitLab
• Prepares the Terraform working directory
• Enables secure access to infrastructure repositories

Variables

The following variables can be configured:

Variable	Description	Default	Required
TERRAFORM_MODULE_VERSION	The branch or tag of the Terraform module repository to clone	v0.0.1	Yes
TERRAFORM_MODULE_PATH	The GitLab project path for the Terraform module repository	welance/platform/infrastructure/modules/terraform.git	Yes
TERRAFORM_DIR	Directory where Terraform modules are moved	infrastructure_code/infrastructure/	No

Variable Details

• TERRAFORM_MODULE_VERSION: The specific version (branch or tag) of the Terraform module to use (e.g., v0.0.1, main, release/1.0.0)
• TERRAFORM_MODULE_PATH: The GitLab project path where the Terraform module is stored (format: group/project.git)
• TERRAFORM_DIR: The target directory where Terraform modules will be placed after cloning

GitLab CI/CD Variables

The job uses the following built-in GitLab CI/CD variables:

• ACCESS_TOKEN: A GitLab Personal Access Token or Project/Group Access Token with read_repository permissions to clone the Terraform module
• SSH_PRIVATE_KEY: The SSH private key (base64 encoded or plain text) used to authenticate with target hosts
• SSH_PUBLIC_KEY: The SSH public key corresponding to SSH_PRIVATE_KEY

Usage

Basic Usage

variables:
  TERRAFORM_MODULE_VERSION: v0.0.1
  TERRAFORM_MODULE_PATH: welance/platform/infrastructure/modules/terraform.git

.terraform-setup:
  variables:
    TERRAFORM_DIR: infrastructure_code/infrastructure/
  script: 
    - 'which ssh-agent || ( apt-get update -qy && apt-get install openssh-client -qqy )'
    - eval `ssh-agent -s`
    - echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "$SSH_PUBLIC_KEY" >> ~/.ssh/id_rsa.pub
    - echo "clone ansible module version "
    - git clone --depth 1 --branch $TERRAFORM_MODULE_VERSION https://oauth2:[email protected]/$TERRAFORM_MODULE_PATH
    - mv ansible/* $ANSIBLE_DIR
    - cd $ANSIBLE_DIR

Job Details

Script

The job performs the following steps:

1. Install SSH client (if needed):

   which ssh-agent || ( apt-get update -qy && apt-get install openssh-client -qqy )
   • Checks if ssh-agent is available
   • Installs openssh-client if not present

2. Start SSH agent:

   eval `ssh-agent -s`
   • Starts the SSH agent in the current shell session

3. Add SSH private key:

   echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
   • Adds the SSH private key to the SSH agent
   • Removes carriage returns (\r) to handle Windows line endings
   • Redirects output to /dev/null for security

4. Configure SSH directory:

   mkdir -p ~/.ssh
   chmod 700 ~/.ssh
   echo "$SSH_PUBLIC_KEY" >> ~/.ssh/id_rsa.pub
   • Creates ~/.ssh directory if it doesn't exist
   • Sets proper permissions (700) for the SSH directory
   • Adds the public key to ~/.ssh/id_rsa.pub

5. Clone Terraform module:

   git clone --depth 1 --branch $TERRAFORM_MODULE_VERSION \
     https://oauth2:[email protected]/$TERRAFORM_MODULE_PATH
   • Clones the Terraform module repository
   • Uses --depth 1 for shallow clone (faster, only latest commit)
   • Checks out the specified TERRAFORM_MODULE_VERSION branch or tag
   • Uses OAuth2 token authentication via ACCESS_TOKEN

6. Move Terraform modules:

   mv ansible/* $ANSIBLE_DIR
   • Moves the cloned Terraform content to the target directory
   • Note: The variable name $ANSIBLE_DIR may need to be $TERRAFORM_DIR depending on your setup

7. Change to Terraform directory:

   cd $ANSIBLE_DIR
   • Changes the working directory to the Terraform module location
   • Note: The variable name $ANSIBLE_DIR may need to be $TERRAFORM_DIR depending on your setup

Configuration Examples

Basic Terraform Setup

variables:
  TERRAFORM_MODULE_VERSION: v0.0.1
  TERRAFORM_MODULE_PATH: welance/platform/infrastructure/modules/terraform.git
  ACCESS_TOKEN: '$GITLAB_ACCESS_TOKEN' # CI/CD variable
  SSH_PRIVATE_KEY: '$CI_SSH_PRIVATE_KEY' # CI/CD variable
  SSH_PUBLIC_KEY: '$CI_SSH_PUBLIC_KEY' # CI/CD variable

stages:
  - setup
  - deploy

.terraform-setup:
  variables:
    TERRAFORM_DIR: infrastructure_code/infrastructure/
  script: 
    - 'which ssh-agent || ( apt-get update -qy && apt-get install openssh-client -qqy )'
    - eval `ssh-agent -s`
    - echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "$SSH_PUBLIC_KEY" >> ~/.ssh/id_rsa.pub
    - echo "clone terraform module version $TERRAFORM_MODULE_VERSION"
    - git clone --depth 1 --branch $TERRAFORM_MODULE_VERSION https://oauth2:[email protected]/$TERRAFORM_MODULE_PATH
    - mv terraform/* $TERRAFORM_DIR
    - cd $TERRAFORM_DIR

setup-terraform:
  extends: .terraform-setup
  stage: setup

Using Specific Terraform Version

setup-terraform-prod:
  extends: .terraform-setup
  stage: setup
  variables:
    TERRAFORM_MODULE_VERSION: 'release/1.0.0' # Use specific release
    TERRAFORM_DIR: 'infrastructure_code/infrastructure/production/'

Complete Pipeline Example

variables:
  TERRAFORM_MODULE_VERSION: v0.0.1
  TERRAFORM_MODULE_PATH: welance/platform/infrastructure/modules/terraform.git
  ACCESS_TOKEN: '$GITLAB_ACCESS_TOKEN'
  SSH_PRIVATE_KEY: '$CI_SSH_PRIVATE_KEY'
  SSH_PUBLIC_KEY: '$CI_SSH_PUBLIC_KEY'

stages:
  - setup
  - plan
  - apply

.terraform-setup:
  image: hashicorp/terraform:latest
  variables:
    TERRAFORM_DIR: infrastructure_code/infrastructure/
  script: 
    - 'which ssh-agent || ( apt-get update -qy && apt-get install openssh-client -qqy )'
    - eval `ssh-agent -s`
    - echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "$SSH_PUBLIC_KEY" >> ~/.ssh/id_rsa.pub
    - echo "clone terraform module version $TERRAFORM_MODULE_VERSION"
    - git clone --depth 1 --branch $TERRAFORM_MODULE_VERSION https://oauth2:[email protected]/$TERRAFORM_MODULE_PATH
    - mv terraform/* $TERRAFORM_DIR
    - cd $TERRAFORM_DIR

setup-terraform:
  extends: .terraform-setup
  stage: setup

terraform-plan:
  image: hashicorp/terraform:latest
  stage: plan
  needs:
    - setup-terraform
  script:
    - cd infrastructure_code/infrastructure/
    - terraform init
    - terraform plan
  dependencies:
    - setup-terraform

terraform-apply:
  image: hashicorp/terraform:latest
  stage: apply
  needs:
    - setup-terraform
    - terraform-plan
  script:
    - cd infrastructure_code/infrastructure/
    - terraform init
    - terraform apply -auto-approve
  when: manual
  dependencies:
    - setup-terraform

Prerequisites

• Terraform Module Repository: The repository specified by TERRAFORM_MODULE_PATH must exist and be accessible
• Access Token: A GitLab ACCESS_TOKEN with read_repository scope is required to clone the Terraform module
• SSH Keys: SSH_PRIVATE_KEY and SSH_PUBLIC_KEY must be configured as CI/CD variables and have appropriate permissions on target hosts
• Docker Image: The job should use a Docker image that supports SSH and Git (e.g., hashicorp/terraform, alpine/git, or a custom image with these tools)

Security Considerations

• SSH Keys: Store SSH_PRIVATE_KEY as a masked and protected CI/CD variable. Ensure it's only accessible to authorized branches/environments
• Access Token: Store ACCESS_TOKEN as a masked and protected CI/CD variable. Limit its scope to read_repository
• SSH Agent: The SSH agent runs in the current shell session and keys are not persisted
• Private Key Handling: The tr -d '\r' command ensures Windows line endings don't break SSH key parsing

Notes

• The job uses shallow clone (--depth 1) for faster execution, fetching only the latest commit from the specified branch/tag
• The mv ansible/* $ANSIBLE_DIR command suggests the original job may have been copied from an Ansible setup job. You may need to adjust this to mv terraform/* $TERRAFORM_DIR or ensure ANSIBLE_DIR is defined as an alias for TERRAFORM_DIR
• The SSH setup is necessary if the Terraform module needs to access private repositories or infrastructure components via SSH
• The job prepares the Terraform working directory but does not run Terraform commands (those should be in subsequent jobs)
• Consider using a Terraform-specific Docker image (e.g., hashicorp/terraform:latest) for the job if you plan to run Terraform commands in the same job
• The cd $ANSIBLE_DIR at the end sets the working directory for subsequent commands, but ensure the variable name matches your setup