Skip to main content

Kubernetes Deploy

The Kubernetes deployment jobs are used to deploy applications to Kubernetes clusters using Helm charts from the Welance charts repository. These jobs support multiple application types with environment-specific configurations.

Overview​

The k8s-deploy jobs include:

  • .deploy-craft-k8s: Deploy Craft CMS applications
  • .deploy-wordpress-k8s: Deploy WordPress applications
  • .deploy-directus-k8s: Deploy Directus CMS applications
  • .deploy-node-k8s: Deploy Node.js applications
  • .deploy-s3-k8s: Deploy S3 static website proxies
  • .deploy-operator-k8s: Deploy Welance Operator

All jobs:

  • Install required tools (Helm, kubectl, yq, AWS CLI)
  • Configure Kubernetes cluster access
  • Deploy using Helm charts from the Welance charts repository
  • Support environment-specific configurations (develop, staging, production)
  • Use different versioning strategies per environment
  • Configure ingress, storage, and security contexts

Variables​

The following variables can be configured:

VariableDescriptionDefaultRequired
PROJECT_IDProject identifier for namespace and hostnames''Yes
APP_ENVIRONMENTTarget environment''Yes
STORAGE_USER_IDUser/Group ID for storage permissions-Yes

Variable Details​

  • PROJECT_ID: Used to:
    • Create Kubernetes namespace: {PROJECT_ID}-{APP_ENVIRONMENT}
    • Generate hostnames: {PROJECT_ID}.{environment}.welance.com
  • APP_ENVIRONMENT: Target environment. Options:
    • 'develop' - Development environment
    • 'staging' - Staging environment
    • 'production' - Production environment
  • STORAGE_USER_ID: User ID, Group ID, and FSGroup for security contexts and storage permissions

GitLab CI/CD Variables​

The jobs use the following built-in GitLab CI/CD variables:

  • CI_COMMIT_REF_NAME: Branch or tag name
  • CI_PIPELINE_ID: Pipeline ID (for develop versioning)
  • CI_PIPELINE_IID: Pipeline internal ID (for staging versioning)
  • CI_COMMIT_SHA: Commit SHA
  • CI_PROJECT_URL: Project URL
  • CI_REGISTRY_IMAGE: Container registry image path

Required Secrets​

The following secrets must be configured in GitLab CI/CD variables:

  • NOPROD_KUBECONFIG: Base64-encoded kubeconfig for non-production clusters
  • PROD_KUBECONFIG: Base64-encoded kubeconfig for production clusters

Common Job Structure​

All Kubernetes deployment jobs follow this structure:

Before Script​

  1. Install Helm:

    curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
    chmod 700 get_helm.sh && ./get_helm.sh

  2. Install AWS CLI:

    pip install awscli

  3. Install kubectl:

    curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
    chmod +x ./kubectl && mv ./kubectl /usr/local/bin

  4. Install yq:

    wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq && chmod +x /usr/bin/yq

  5. Configure Kubernetes access:

    mkdir -p ~/.kube
    helm repo add welance-charts https://welance.gitlab.io/platform/pipelines/helm/welance-charts/
    helm repo update
    if [ ${APP_ENVIRONMENT} = 'production' ]; then 
      echo "$PROD_KUBECONFIG" | base64 -d > ~/.kube/config
    else 
      echo "$NOPROD_KUBECONFIG" | base64 -d > ~/.kube/config
    fi
    chmod 600 ~/.kube/config && export KUBECONFIG=~/.kube/config

Version Tagging Strategy​

Different environments use different versioning strategies:

  • develop: {CI_COMMIT_REF_NAME}-{CI_PIPELINE_ID} (e.g., main-12345)
  • staging: {RELEASE}.rc{CI_PIPELINE_IID} (e.g., 1.0.0.rc42) where RELEASE is extracted from branch name
  • production: {RELEASE} (e.g., 1.0.0) where RELEASE is extracted from branch name

Storage File System IDs​

Different environments use different EFS file system IDs:

  • develop: fs-0e7fcf2d3c61da878
  • staging: fs-0f2d81976ef2bf737
  • production: fs-0c98ff7929df55f36

Job Types​

.deploy-craft-k8s​

Deploys Craft CMS applications to Kubernetes.

Usage​

variables:
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'
  STORAGE_USER_ID: '1000'

deploy-craft-dev:
  extends: .deploy-craft-k8s
  stage: deploy
  only:
    - develop

Helm Chart Configuration​

  • Chart: welance-charts/craft
  • Namespace: {PROJECT_ID}-{APP_ENVIRONMENT}
  • Environment file: .env.{APP_ENVIRONMENT} (e.g., .env.dev, .env.staging, .env.production)
  • Values file: ./helm-chart/values-{APP_ENVIRONMENT}.yaml

Common Helm Values Set​

All environments set:

  • Image tag, commit SHA, branch, and URL
  • Security context (runAsUser, runAsGroup, fsGroup)
  • Storage configuration (gid, uid, fid)
  • Ingress configuration with Cloudflare TLS
  • Service configuration

.deploy-wordpress-k8s​

Deploys WordPress applications to Kubernetes.

Usage​

variables:
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'
  STORAGE_USER_ID: '1000'

deploy-wordpress-dev:
  extends: .deploy-wordpress-k8s
  stage: deploy
  only:
    - develop

Differences from Craft​

  • Uses welance-charts/wordpress chart
  • Same configuration structure as Craft

.deploy-directus-k8s​

Deploys Directus CMS applications to Kubernetes.

Usage​

variables:
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'
  STORAGE_USER_ID: '1000'

deploy-directus-dev:
  extends: .deploy-directus-k8s
  stage: deploy
  only:
    - develop

Differences from Craft​

  • Uses welance-charts/directus chart
  • Same configuration structure as Craft

.deploy-node-k8s​

Deploys Node.js applications to Kubernetes.

Usage​

variables:
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'

deploy-node-dev:
  extends: .deploy-node-k8s
  stage: deploy
  only:
    - develop

Differences from Craft​

  • Uses welance-charts/node chart
  • Does not set security context or storage configuration (simpler configuration)
  • Same ingress and service configuration

.deploy-s3-k8s​

Deploys S3 static website proxies to Kubernetes.

Usage​

variables:
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'

deploy-s3-dev:
  extends: .deploy-s3-k8s
  stage: deploy
  only:
    - develop

Differences from Craft​

  • Uses welance-charts/s3 chart
  • Does not set image tag, commit info, or security context
  • Service port is 80 (instead of 8080)
  • Service name is web (instead of service-web)
  • Simpler configuration focused on ingress proxy

.deploy-operator-k8s​

Deploys Welance Operator to Kubernetes.

Usage​

variables:
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'

deploy-operator-dev:
  extends: .deploy-operator-k8s
  stage: deploy
  only:
    - develop

Differences from Craft​

  • Uses welance-charts/welance-operator chart
  • Minimal configuration (only image tag and repository)
  • No ingress, storage, or security context configuration
  • Only sets environment file for develop environment

Environment-Specific Configuration​

Develop Environment​

Version: {CI_COMMIT_REF_NAME}-{CI_PIPELINE_ID}

Hostname: {PROJECT_ID}.dev.welance.com

Storage FID: fs-0e7fcf2d3c61da878

Example:

APP_ENVIRONMENT: 'develop'
# Results in:
# - Namespace: p222-01-develop
# - Hostname: p222-01.dev.welance.com
# - Version: main-12345

Staging Environment​

Version: {RELEASE}.rc{CI_PIPELINE_IID} (e.g., 1.0.0.rc42)

Hostname: {PROJECT_ID}.staging.welance.com

Storage FID: fs-0f2d81976ef2bf737

Example:

APP_ENVIRONMENT: 'staging'
CI_COMMIT_REF_NAME: 'release/1.0.0'
CI_PIPELINE_IID: '42'
# Results in:
# - Namespace: p222-01-staging
# - Hostname: p222-01.staging.welance.com
# - Version: 1.0.0.rc42

Production Environment​

Version: {RELEASE} (e.g., 1.0.0)

Hostname: {PROJECT_ID}.production.welance.com

Storage FID: fs-0c98ff7929df55f36

Example:

APP_ENVIRONMENT: 'production'
CI_COMMIT_REF_NAME: 'release/1.0.0'
# Results in:
# - Namespace: p222-01-production
# - Hostname: p222-01.production.welance.com
# - Version: 1.0.0

Common Helm Values​

All application deployments (except S3 and Operator) set:

Image Configuration​

  • image.tag: Version tag
  • image.repository: $CI_REGISTRY_IMAGE
  • image.commit.sha: $CI_COMMIT_SHA
  • image.commit.branch: $CI_COMMIT_REF_NAME
  • image.commit.url: $CI_PROJECT_URL/-/commit/$CI_COMMIT_SHA

Security Context​

  • deployment.securityContext.runAsUser: $STORAGE_USER_ID
  • deployment.securityContext.runAsGroup: $STORAGE_USER_ID
  • deployment.securityContext.fsGroup: $STORAGE_USER_ID

Storage Configuration​

  • storage.gid: $STORAGE_USER_ID
  • storage.uid: $STORAGE_USER_ID
  • storage.fid: Environment-specific file system ID

Ingress Configuration​

  • ingresses.web.hosts[0].host: {PROJECT_ID}.{environment}.welance.com
  • ingresses.web.tls[0].hosts[0]: Same as host
  • ingresses.web.tls[0].secretName: cloudflare-tls
  • ingresses.web.hosts[0].paths[0].path: /
  • ingresses.web.hosts[0].paths[0].pathType: Prefix
  • ingresses.web.hosts[0].paths[0].servicePort: 8080 (or 80 for S3)
  • ingresses.web.hosts[0].paths[0].serviceName: service-web (or web for S3)

Environment File​

  • --set-file envFileContent=.env.{APP_ENVIRONMENT}: Loads environment-specific .env file

Complete Pipeline Example​

variables:
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'
  STORAGE_USER_ID: '1000'

stages:
  - build
  - deploy

# Build job creates .env files
build:
  stage: build
  script:
    - echo "DB_HOST=db.example.com" > .env.dev
    - echo "DB_HOST=db.staging.example.com" > .env.staging
    - echo "DB_HOST=db.prod.example.com" > .env.production
  artifacts:
    paths:
      - .env.dev
      - .env.staging
      - .env.production
      - helm-chart/

.deploy-craft-k8s:
  image: python:latest
  before_script:
    - curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && chmod 700 get_helm.sh && ./get_helm.sh
    - pip install awscli
    - curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
    - chmod +x ./kubectl && mv ./kubectl /usr/local/bin
    - wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq && chmod +x /usr/bin/yq
    - mkdir -p ~/.kube
    - helm repo add welance-charts https://welance.gitlab.io/platform/pipelines/helm/welance-charts/
    - helm repo update
    - if [ ${APP_ENVIRONMENT} = 'production' ]; then echo "$PROD_KUBECONFIG" | base64 -d > ~/.kube/config; else echo "$NOPROD_KUBECONFIG" | base64 -d > ~/.kube/config; fi
    - chmod 600 ~/.kube/config && export KUBECONFIG=~/.kube/config
  script:
    - |-
      if [[ $APP_ENVIRONMENT = 'develop' ]]; then
        export VERSION=$CI_COMMIT_REF_NAME-$CI_PIPELINE_ID
        helm upgrade --install --create-namespace -n $PROJECT_ID-$APP_ENVIRONMENT \
          --set image.tag=$VERSION \
          --set image.commit.sha="$CI_COMMIT_SHA" \
          --set image.commit.branch="$CI_COMMIT_REF_NAME" \
          --set-string image.commit.url="$CI_PROJECT_URL/-/commit/$CI_COMMIT_SHA" \
          --set deployment.securityContext.runAsUser=$STORAGE_USER_ID \
          --set deployment.securityContext.runAsGroup=$STORAGE_USER_ID \
          --set deployment.securityContext.fsGroup=$STORAGE_USER_ID \
          --set storage.gid=$STORAGE_USER_ID \
          --set storage.uid=$STORAGE_USER_ID \
          --set storage.fid=fs-0e7fcf2d3c61da878 \
          --set ingresses.web.hosts[0].host="${PROJECT_ID}.dev.welance.com" \
          --set ingresses.web.tls[0].hosts[0]="${PROJECT_ID}.dev.welance.com" \
          --set ingresses.web.tls[0].secretName=cloudflare-tls \
          --set image.repository=$CI_REGISTRY_IMAGE \
          --set ingresses.web.hosts[0].paths[0].path=/ \
          --set ingresses.web.hosts[0].paths[0].pathType=Prefix \
          --set ingresses.web.hosts[0].paths[0].servicePort=8080 \
          --set ingresses.web.hosts[0].paths[0].serviceName=service-web \
          $PROJECT_ID-$APP_ENVIRONMENT welance-charts/craft \
          --set-file envFileContent=.env.dev \
          -f ./helm-chart/values-$APP_ENVIRONMENT.yaml
      fi
  dependencies:
    - build

deploy-craft-dev:
  extends: .deploy-craft-k8s
  stage: deploy
  only:
    - develop

Prerequisites​

  • Kubernetes cluster: Accessible Kubernetes cluster (production or non-production)
  • Kubeconfig: Base64-encoded kubeconfig files stored as CI/CD variables
  • Helm charts repository: Welance charts repository must be accessible
  • Environment files: .env.{environment} files must exist for the target environment
  • Values files: helm-chart/values-{environment}.yaml files must exist
  • Container images: Images must be built and pushed to the container registry
  • Cloudflare TLS: TLS secret cloudflare-tls must exist in the cluster
  • EFS file systems: EFS file systems must be configured and accessible

Security Considerations​

  • Kubeconfig: Store NOPROD_KUBECONFIG and PROD_KUBECONFIG as masked CI/CD variables
  • Namespace isolation: Each project and environment gets its own namespace
  • Storage permissions: Storage user IDs should be configured appropriately
  • TLS certificates: Cloudflare TLS certificates are managed separately
  • Image registry: Ensure container registry access is properly configured

Notes​

  • The job uses helm template to validate before actual deployment (echoed command)
  • Namespaces are created automatically with --create-namespace
  • Different storage file system IDs are used per environment
  • Version tagging differs by environment (develop uses branch-pipeline, staging uses release-rc, production uses release)
  • Ingress hostnames follow the pattern: {PROJECT_ID}.{environment}.welance.com
  • All jobs use Cloudflare TLS certificates via the cloudflare-tls secret
  • The Python image is used as a base for tool installation
  • Helm charts are pulled from the centralized Welance charts repository
  • Environment-specific values files allow customization per environment