Skip to main content

Craft VM Deploy

The ansible-craft-vm-deploy job is used to deploy Craft CMS applications to virtual machines using Ansible. It deploys application files extracted from Docker containers to VM-based hosting.

Overview​

This job:

  • Uses Ansible to deploy Craft CMS applications to virtual machines
  • Deploys application files extracted from Docker containers (from .docker-build-vm-setup job)
  • Clones a specific version of the Ansible module from GitLab
  • Configures SSH access for remote server management
  • Manages environment-specific configuration files
  • Uses rsync for efficient file transfers

Variables​

The following variables can be configured:

VariableDescriptionDefaultRequired
ANSIBLE_MODULE_VERSIONVersion/tag of the Ansible module to userelease/0.0.119No
ANSIBLE_MODULE_PATHGitLab path to the Ansible module repositorywelance/platform/infrastructure/modules/ansible.gitNo
DOCKER_BUILD_IMAGEDocker image for Docker-in-Docker service'docker:24.0.5'No
DOCKER_BUILD_SERVICEDocker-in-Docker service version'24.0.5-dind'No
DOCKER_COMPOSE_IMAGEDocker Compose image'docker/compose:latest'No
DOCKER_COMPOSE_SERVICEDocker Compose service'dind'No
ANSIBLE_DIRDirectory where Ansible code will be placedinfrastructure_code/configuration_management/No
HOSTTarget host address or hostname''Yes
VERSIONApplication version/tag to deploy''Yes
VAR_FILESPath to Ansible variable files''No
APP_ENVIRONMENTTarget environment''Yes
ANSIBLE_PORTSSH port for Ansible connection''Yes
ANSIBLE_USERSSH user for Ansible connection''Yes

Variable Details​

  • ANSIBLE_MODULE_VERSION: The Git tag or branch of the Ansible module repository to use (e.g., release/0.0.119)
  • ANSIBLE_MODULE_PATH: The GitLab project path to the Ansible module repository
  • DOCKER_BUILD_IMAGE: Docker image version for Docker-in-Docker service (used if Docker operations are needed)
  • DOCKER_BUILD_SERVICE: Docker-in-Docker service version
  • DOCKER_COMPOSE_IMAGE: Docker Compose image (if compose operations are needed)
  • DOCKER_COMPOSE_SERVICE: Docker Compose service
  • ANSIBLE_DIR: Local directory where the Ansible module will be cloned and executed from
  • HOST: The target server hostname or IP address where the deployment will occur
  • VERSION: The application version/tag to deploy. Typically sourced from .env.version file created by build jobs
  • VAR_FILES: Path to Ansible variable files (JSON or YAML) to pass additional configuration
  • APP_ENVIRONMENT: Target environment for deployment. Options:
    • 'develop' - Development environment
    • 'staging' - Staging environment
    • 'production' - Production environment
  • ANSIBLE_PORT: SSH port number for connecting to the target host (e.g., 22, 2222)
  • ANSIBLE_USER: SSH username for authenticating to the target host

GitLab CI/CD Variables​

The job uses the following built-in GitLab CI/CD variables:

  • DOCKER_HOST: Docker daemon connection (set to tcp://docker:2376)
  • DOCKER_TLS_CERTDIR: Docker TLS certificate directory (set to "/certs")

Required Secrets​

The following secrets must be configured in GitLab CI/CD variables:

  • ACCESS_TOKEN: GitLab OAuth token with access to clone the Ansible module repository
  • ANSIBLE_SSH_ID_RSA: SSH private key content for authenticating to target servers

Usage​

Basic Usage​

variables:
  ANSIBLE_MODULE_VERSION: release/0.0.119
  ANSIBLE_MODULE_PATH: welance/platform/infrastructure/modules/ansible.git
  HOST: 'example.com'
  VERSION: ''
  VAR_FILES: ''
  APP_ENVIRONMENT: 'develop'
  ANSIBLE_PORT: '22'
  ANSIBLE_USER: 'deploy'

.ansible-craft-vm-deploy:
  image: willhallonline/ansible:2.16.4-ubuntu-22.04
  services:
    - docker:$DOCKER_BUILD_SERVICE
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/certs"
    ANSIBLE_DIR: infrastructure_code/configuration_management/
    HOST: ''
    VERSION: ''
    APP_ENVIRONMENT: ''
    VAR_FILES: ''
    ANSIBLE_PORT: ''
    ANSIBLE_USER: ''
  before_script:
    - rm -rf dockerimage/app/helm-chart
    - rm -rf dockerimage/app/infrastructure_code   
    - rm -rf dockerimage/app/.env*
    - mv dockerimage/app .
    - apt update && apt install -y rsync
    - echo "clone ansible module version $ANSIBLE_MODULE_VERSION"
    - git clone --depth 1 --branch $ANSIBLE_MODULE_VERSION https://oauth2:[email protected]/$ANSIBLE_MODULE_PATH
    - mv ansible/* $ANSIBLE_DIR
  script:
    - cp .env.${APP_ENVIRONMENT} app/
    - mv app/.env.${APP_ENVIRONMENT} app/.env
    - ls -al ./app/
    - source .env.version
    - echo "Deploy version=$VERSION"
    - mkdir -p ~/.ssh/ && touch ~/.ssh/id_rsa && chmod 600 ~/.ssh/id_rsa
    - cat $ANSIBLE_SSH_ID_RSA > ~/.ssh/id_rsa
    - echo "ANSIBLE_HOST=${HOST}"
    - echo "ANSIBLE_USER=${ANSIBLE_USER}"
    - echo "ANSIBLE_PORT=${ANSIBLE_PORT}"
    - sed -i "s/HOST_TO_BE_REPLACED/${HOST}/g" $ANSIBLE_DIR/hosts-vm
    - sed -i "s/USER_TO_BE_REPLACED/${ANSIBLE_USER}/g" $ANSIBLE_DIR/hosts-vm
    - sed -i "s/PORT_TO_BE_REPLACED/${ANSIBLE_PORT}/g" $ANSIBLE_DIR/hosts-vm
    - cat $ANSIBLE_DIR/hosts-vm    
    - ansible-playbook $ANSIBLE_DIR/playbook-craft-vm.yml -b -i $ANSIBLE_DIR/hosts-vm --extra-vars "@${ANSIBLE_DIR}/${VAR_FILES}" --extra-vars "env=$APP_ENVIRONMENT" --extra-vars "tag=$VERSION" --extra-vars "ansible_ssh_extra_args='-o StrictHostKeyChecking=no'" --private-key="~/.ssh/id_rsa"

Job Details​

Before Script​

The before_script section performs the following setup:

  1. Clean up extracted application files:

    rm -rf dockerimage/app/helm-chart
    rm -rf dockerimage/app/infrastructure_code   
    rm -rf dockerimage/app/.env*
    • Removes Helm charts (not needed for VM deployment)
    • Removes infrastructure code
    • Removes environment files (will be replaced with environment-specific ones)

  2. Move application files:

    mv dockerimage/app .
    • Moves the extracted application files from dockerimage/app to the current directory
    • These files should have been extracted by a previous .docker-build-vm-setup job

  3. Install rsync:

    apt update && apt install -y rsync
    • Installs rsync for efficient file synchronization

  4. Clone Ansible module:

    git clone --depth 1 --branch $ANSIBLE_MODULE_VERSION https://oauth2:[email protected]/$ANSIBLE_MODULE_PATH
    mv ansible/* $ANSIBLE_DIR
    • Clones only the specified version (shallow clone)
    • Moves Ansible files to the configured directory

Script​

The main script performs the following steps:

  1. Copy environment-specific configuration:

    cp .env.${APP_ENVIRONMENT} app/
    mv app/.env.${APP_ENVIRONMENT} app/.env
    • Copies the environment-specific .env file (e.g., .env.develop, .env.staging, .env.production)
    • Renames it to .env in the application directory

  2. List application files (for debugging):

    ls -al ./app/

  3. Source version file:

    source .env.version
    • Loads the VERSION variable from the .env.version file created by build jobs

  4. Display deployment version:

    echo "Deploy version=$VERSION"

  5. Configure SSH key:

    mkdir -p ~/.ssh/ && touch ~/.ssh/id_rsa && chmod 600 ~/.ssh/id_rsa
    cat $ANSIBLE_SSH_ID_RSA > ~/.ssh/id_rsa
    • Creates SSH directory and key file
    • Sets proper permissions (600)
    • Writes the SSH private key from GitLab variable

  6. Display connection information:

    echo "ANSIBLE_HOST=${HOST}"
    echo "ANSIBLE_USER=${ANSIBLE_USER}"
    echo "ANSIBLE_PORT=${ANSIBLE_PORT}"

  7. Configure Ansible hosts file:

    sed -i "s/HOST_TO_BE_REPLACED/${HOST}/g" $ANSIBLE_DIR/hosts-vm
    sed -i "s/USER_TO_BE_REPLACED/${ANSIBLE_USER}/g" $ANSIBLE_DIR/hosts-vm
    sed -i "s/PORT_TO_BE_REPLACED/${ANSIBLE_PORT}/g" $ANSIBLE_DIR/hosts-vm
    cat $ANSIBLE_DIR/hosts-vm
    • Replaces placeholders in the hosts-vm file with actual values
    • Displays the configured hosts file

  8. Run Ansible playbook:

    ansible-playbook $ANSIBLE_DIR/playbook-craft-vm.yml \
      -b \
      -i $ANSIBLE_DIR/hosts-vm \
      --extra-vars "@${ANSIBLE_DIR}/${VAR_FILES}" \
      --extra-vars "env=$APP_ENVIRONMENT" \
      --extra-vars "tag=$VERSION" \
      --extra-vars "ansible_ssh_extra_args='-o StrictHostKeyChecking=no'" \
      --private-key="~/.ssh/id_rsa"

    Playbook Parameters:

    • playbook-craft-vm.yml: The Ansible playbook for Craft VM deployment
    • -b: Use become (sudo) for privilege escalation
    • -i $ANSIBLE_DIR/hosts-vm: Inventory file with target hosts
    • --extra-vars "@${ANSIBLE_DIR}/${VAR_FILES}": Loads additional variables from file (if provided)
    • --extra-vars "env=$APP_ENVIRONMENT": Sets the target environment
    • --extra-vars "tag=$VERSION": Sets the application version/tag
    • --private-key="~/.ssh/id_rsa": SSH private key for authentication

Configuration Examples​

Development Environment​

variables:
  ANSIBLE_MODULE_VERSION: release/0.0.119
  ANSIBLE_MODULE_PATH: welance/platform/infrastructure/modules/ansible.git
  HOST: 'dev.example.com'
  APP_ENVIRONMENT: 'develop'
  ANSIBLE_PORT: '22'
  ANSIBLE_USER: 'deploy'

deploy-craft-vm-dev:
  extends: .ansible-craft-vm-deploy
  stage: deploy
  dependencies:
    - build-vm-setup
  only:
    - develop

Production Environment with Custom Variables​

variables:
  ANSIBLE_MODULE_VERSION: release/0.0.119
  ANSIBLE_MODULE_PATH: welance/platform/infrastructure/modules/ansible.git
  HOST: 'prod.example.com'
  APP_ENVIRONMENT: 'production'
  ANSIBLE_PORT: '2222'
  ANSIBLE_USER: 'deploy'
  VAR_FILES: 'vars/production.yml'

deploy-craft-vm-prod:
  extends: .ansible-craft-vm-deploy
  stage: deploy
  dependencies:
    - build-vm-setup
  only:
    - main
    - tags

Complete Pipeline Example​

variables:
  ANSIBLE_MODULE_VERSION: release/0.0.119
  ANSIBLE_MODULE_PATH: welance/platform/infrastructure/modules/ansible.git

stages:
  - build
  - deploy

# Build job that extracts files from container
build-vm-setup:
  extends: .docker-build-vm-setup
  stage: build
  artifacts:
    paths:
      - dockerimage/
      - .env.version
      - .env.develop
      - .env.staging
      - .env.production

.ansible-craft-vm-deploy:
  image: willhallonline/ansible:2.16.4-ubuntu-22.04
  services:
    - docker:$DOCKER_BUILD_SERVICE
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/certs"
    ANSIBLE_DIR: infrastructure_code/configuration_management/
    HOST: ''
    VERSION: ''
    APP_ENVIRONMENT: ''
    VAR_FILES: ''
    ANSIBLE_PORT: ''
    ANSIBLE_USER: ''
  before_script:
    - rm -rf dockerimage/app/helm-chart
    - rm -rf dockerimage/app/infrastructure_code   
    - rm -rf dockerimage/app/.env*
    - mv dockerimage/app .
    - apt update && apt install -y rsync
    - echo "clone ansible module version $ANSIBLE_MODULE_VERSION"
    - git clone --depth 1 --branch $ANSIBLE_MODULE_VERSION https://oauth2:[email protected]/$ANSIBLE_MODULE_PATH
    - mv ansible/* $ANSIBLE_DIR
  script:
    - cp .env.${APP_ENVIRONMENT} app/
    - mv app/.env.${APP_ENVIRONMENT} app/.env
    - ls -al ./app/
    - source .env.version
    - echo "Deploy version=$VERSION"
    - mkdir -p ~/.ssh/ && touch ~/.ssh/id_rsa && chmod 600 ~/.ssh/id_rsa
    - cat $ANSIBLE_SSH_ID_RSA > ~/.ssh/id_rsa
    - echo "ANSIBLE_HOST=${HOST}"
    - echo "ANSIBLE_USER=${ANSIBLE_USER}"
    - echo "ANSIBLE_PORT=${ANSIBLE_PORT}"
    - sed -i "s/HOST_TO_BE_REPLACED/${HOST}/g" $ANSIBLE_DIR/hosts-vm
    - sed -i "s/USER_TO_BE_REPLACED/${ANSIBLE_USER}/g" $ANSIBLE_DIR/hosts-vm
    - sed -i "s/PORT_TO_BE_REPLACED/${ANSIBLE_PORT}/g" $ANSIBLE_DIR/hosts-vm
    - cat $ANSIBLE_DIR/hosts-vm    
    - ansible-playbook $ANSIBLE_DIR/playbook-craft-vm.yml -b -i $ANSIBLE_DIR/hosts-vm --extra-vars "@${ANSIBLE_DIR}/${VAR_FILES}" --extra-vars "env=$APP_ENVIRONMENT" --extra-vars "tag=$VERSION" --extra-vars "ansible_ssh_extra_args='-o StrictHostKeyChecking=no'" --private-key="~/.ssh/id_rsa"

deploy-craft-vm-develop:
  extends: .ansible-craft-vm-deploy
  variables:
    HOST: 'dev.example.com'
    APP_ENVIRONMENT: 'develop'
    ANSIBLE_PORT: '22'
    ANSIBLE_USER: 'deploy'
  stage: deploy
  dependencies:
    - build-vm-setup
  only:
    - develop

Prerequisites​

  • Docker build with VM setup: A previous job (.docker-build-vm-setup) must extract application files to dockerimage/app/
  • Environment files: Environment-specific .env files (.env.develop, .env.staging, .env.production) must exist
  • Version file: A .env.version file must be created by a previous build job
  • Ansible module repository: The Ansible module must exist at the specified GitLab path
  • SSH access: SSH keys must be configured and have access to target servers
  • GitLab access token: OAuth token with permissions to clone the Ansible module repository
  • Ansible playbook: The repository must contain a playbook-craft-vm.yml file
  • Ansible hosts template: A hosts-vm file with placeholders (HOST_TO_BE_REPLACED, USER_TO_BE_REPLACED, PORT_TO_BE_REPLACED) must exist
  • Target servers: Servers must be accessible via SSH and have rsync available
  • Sudo access: The Ansible user must have sudo privileges on target servers (uses -b flag)

Differences from Docker Deploy​

FeatureCraft Docker DeployCraft VM Deploy
TargetDocker containersVirtual machines
Application SourceDocker images from registryFiles extracted from containers
Deployment MethodDocker pull and runFile synchronization (rsync)
PrerequisitesDocker build jobDocker build + VM setup job
Environment FilesNot copiedCopied and renamed to .env
Privilege EscalationNot requiredUses -b (become/sudo)
CleanupN/ARemoves helm-chart, infrastructure_code, .env* files

Security Considerations​

  • SSH Keys: Store SSH private key (ANSIBLE_SSH_ID_RSA) as a GitLab CI/CD masked variable
  • Access Token: Store the GitLab access token as a masked and protected variable
  • Key Permissions: SSH keys should have minimal required permissions
  • Sudo Access: Ensure the Ansible user has only necessary sudo permissions
  • StrictHostKeyChecking: The job disables strict host key checking for convenience, but this should be reviewed for production use
  • Environment Files: Ensure environment files don't contain sensitive data or are properly secured

Notes​

  • The job expects application files to be extracted by a previous .docker-build-vm-setup job
  • Files are cleaned up before deployment (removes helm-chart, infrastructure_code, .env*)
  • Environment-specific .env files are copied and renamed to .env in the application directory
  • The job uses rsync for efficient file synchronization to target servers
  • The Ansible playbook runs with privilege escalation (-b flag) for system-level operations
  • The hosts-vm file is dynamically configured with target host information
  • Docker-in-Docker service is available but primarily used if Docker operations are needed during deployment
  • The Ansible image willhallonline/ansible:2.16.4-ubuntu-22.04 includes Ansible 2.16.4