Skip to main content

S3 Pipeline Template

The s3.yml pipeline template provides a complete CI/CD workflow for S3 static website deployments with optional Kubernetes/GitOps support.

Overview​

This template orchestrates the deployment lifecycle for static websites and S3-based applications, including:

  • Docker image building (for containerized static site generators)
  • Version validation
  • GitOps-based deployment to Kubernetes (optional)
  • S3 deployment support
  • Release management

Usage​

include:
  - project: 'welance/platform/pipelines/templates/pipeline/git-flow'
    ref: release/1.0.0
    file: 's3.yml'
variables:
  PROJECT_ID: 'p222-01'
  DEV_TARGET: 'welance'
  STAGING_TARGET: 'welance'
  PRODUCTION_TARGET: 'welance'

Pipeline Stages​

The S3 pipeline template includes the following stages:

  1. init - Version validation and preparation
  2. build - Docker image building
  3. deploy - GitOps-based deployment to Kubernetes (or S3)
  4. release - Release tagging and GitLab release creation

Included Job Templates​

The template includes the following job templates from the ci-jobs repository:

Build Jobs​

  • build/docker-build.yml - Docker image building

Deploy Jobs​

  • deploy/k8s-deploy.yml - Kubernetes deployments
  • deploy/s3-deploy.yml - S3 deployments
  • deploy/gitops-and-value-sync.yml - GitOps deployments

Release Jobs​

  • release/merge-and-tag.yml - Merge request handling and tagging

Default Variables​

variables:
  CI_ARTIFACT_TOKEN: $CI_ARTIFACT_PULL_TOKEN
  REGISTRY_ID: 4495833
  DEV_TARGET: 'welance'
  STAGING_TARGET: 'welance'
  PRODUCTION_TARGET: 'welance'

Variable Details​

  • CI_ARTIFACT_TOKEN: Token for pulling artifacts from other jobs
  • REGISTRY_ID: Container registry project ID
  • DEV_TARGET: Deployment target for develop environment (default: welance)
  • STAGING_TARGET: Deployment target for staging environment (default: welance)
  • PRODUCTION_TARGET: Deployment target for production environment (default: welance, must be welance for release jobs)

Continuous Integration Workflow​

Build Jobs​

build_develop_job​

  • Stage: build
  • Extends: .docker-build
  • Environment: develop
  • Runs on: develop branch
  • Artifacts: helm-chart/, .env*, pipeline.env
  • Purpose: Builds Docker image for develop environment

build_staging_job​

  • Stage: build
  • Extends: .docker-build
  • Needs: check_test
  • Environment: staging
  • Runs on: release/* and hotfix/* branches (when STAGING_TARGET != "mittwald")
  • Artifacts: helm-chart/, infrastructure_code/, .env*
  • Purpose: Builds Docker image for staging environment

build_production_job​

  • Stage: build
  • Extends: .docker-build
  • Needs: check_prod
  • Environment: production
  • Runs on: MRs targeting master branch (when PRODUCTION_TARGET != "mittwald")
  • Artifacts: helm-chart/, infrastructure_code/, .env*, pipeline.env
  • Purpose: Builds Docker image for production environment

Continuous Delivery Workflow​

Version Validation​

check_test​

  • Stage: init
  • Purpose: Validates semantic versioning for staging deployments
  • Runs on: release/* and hotfix/* branches
  • Output: Sets VERSION={RELEASE}.rc{CI_PIPELINE_IID} (e.g., 1.0.0.rc42)

check_prod​

  • Stage: init
  • Purpose: Validates semantic versioning for production deployments
  • Runs on: MRs targeting master branch
  • Output: Sets VERSION={RELEASE}, TAG=v{VERSION}, EXTRA_DESCRIPTION

Deployment (Welance/Kubernetes)​

The S3 pipeline supports GitOps-based Kubernetes deployments:

deploy_develop_job​

  • Stage: deploy
  • Extends: .gitops-update-values-and-sync
  • Target: Argo CD application
  • Runs on: develop branch (when DEV_TARGET == "welance")
  • Updates: GitOps repository with new image tag and environment variables

deploy_staging_job​

  • Stage: deploy
  • Extends: .gitops-update-values-and-sync
  • Target: Argo CD application
  • Runs on: release/* and hotfix/* branches (when STAGING_TARGET == "welance")
  • Updates: GitOps repository with new image tag and environment variables

deploy_production_job​

  • Stage: deploy
  • Extends: .gitops-update-values-and-sync
  • Target: Argo CD application
  • Runs on: MRs targeting master (when PRODUCTION_TARGET == "welance")
  • When: manual
  • Updates: GitOps repository with new image tag and environment variables

S3 Deployment​

The pipeline includes the .deploy-s3 job template, which can be used for direct S3 deployments. To use S3 deployment, you would need to add custom jobs that extend .deploy-s3. The default pipeline focuses on GitOps/Kubernetes deployments.

Release Jobs​

merge_and_tag​

  • Stage: release
  • Extends: .merge_and_tag_job
  • Needs: deploy_production_job
  • When: manual (for production)
  • Merges MR, creates realign branch, creates alignment MR

release_job​

  • Stage: release
  • Image: registry.gitlab.com/gitlab-org/release-cli:latest
  • Needs: merge_and_tag
  • Creates GitLab release with tag and description

Required Variables​

  • PROJECT_ID - Your project identifier (e.g., p222-01)
  • STORAGE_USER_ID - User ID for storage permissions (for staging/production builds)

Environment-Specific Variables​

For Welance (Kubernetes) Deployments​

  • NOPROD_TARGET_REPO - GitOps repository for non-production
  • PROD_TARGET_REPO - GitOps repository for production
  • NOPROD_ARGOCD_SERVER - Argo CD server for non-production
  • PROD_ARGOCD_SERVER - Argo CD server for production
  • NOPROD_ARGOCD_USER / NOPROD_ARGOCD_PASSWORD - Argo CD credentials
  • PROD_ARGOCD_USER / PROD_ARGOCD_PASSWORD - Argo CD credentials
  • NOPROD_GITLAB_USER / NOPROD_GITLAB_PASSWORD - GitLab credentials for GitOps

For S3 Deployments (if using)​

  • AWS_REGION - AWS region for S3 bucket (default: eu-central-1)
  • BUILD_FOLDER - Directory containing built static files (default: build)
  • AWS credentials (via CI/CD variables or IAM roles)

Example Configuration​

GitOps/Kubernetes Deployment​

include:
  - project: 'welance/platform/pipelines/templates/pipeline/git-flow'
    ref: release/1.0.0
    file: 's3.yml'
variables:
  PROJECT_ID: 'p222-01'
  DEV_TARGET: 'welance'
  STAGING_TARGET: 'welance'
  PRODUCTION_TARGET: 'welance'
  STORAGE_USER_ID: '1000'
  NOPROD_TARGET_REPO: 'https://gitlab.com/welance/platform/gitops/noprod.git'
  PROD_TARGET_REPO: 'https://gitlab.com/welance/platform/gitops/prod.git'
  NOPROD_ARGOCD_SERVER: 'argocd-noprod.welance.com'
  PROD_ARGOCD_SERVER: 'argocd-prod.welance.com'

Adding S3 Deployment​

To add S3 deployment alongside GitOps, you can extend the pipeline:

include:
  - project: 'welance/platform/pipelines/templates/pipeline/git-flow'
    ref: release/1.0.0
    file: 's3.yml'
variables:
  PROJECT_ID: 'p222-01'
  STAGING_TARGET: 's3'  # Use S3 for staging
  PRODUCTION_TARGET: 'welance'  # Use GitOps for production

deploy_staging_s3:
  stage: deploy
  extends: .deploy-s3
  needs:
    - build_staging_job
  variables:
    APP_ENVIRONMENT: 'staging'
    BUILD_FOLDER: 'dist'
    AWS_REGION: 'eu-central-1'
  rules:
    - if: '$CI_COMMIT_BRANCH && $STAGING_TARGET == "s3" && $CI_COMMIT_BRANCH =~ "/^release/"'
      when: on_success

Branch Strategy​

  • develop - Continuous integration and deployment to develop environment
  • release/{version} - Build, test, and deploy to staging
  • hotfix/{version} - Build, test, and deploy to staging
  • master (via MR) - Build, test, and deploy to production (manual)

Version Tagging​

  • Develop: Uses pipeline ID for unique versions
  • Staging: `{RELEASE}.rc{CI_PIPELINE_IID}` (e.g., 1.0.0.rc42)
  • Production: `{RELEASE}` (e.g., 1.0.0)

Caching​

The pipeline caches Node.js modules:

cache:
  key: "$CI_PROJECT_NAME"
  paths:
    - './.node_modules/'

This speeds up subsequent pipeline runs by reusing installed dependencies.

GitOps Workflow​

The S3 pipeline uses GitOps for deployments when *_TARGET == "welance":

  1. Build: Docker image is built and pushed to registry
  2. GitOps Update: Pipeline updates Helm values in GitOps repository:
    • Updates image.tag with new version
    • Updates image.commit.* with commit information
    • Updates envFileContent with environment variables from .env.dev, .env.staging, or .env.production
  3. Argo CD Sync: Pipeline triggers Argo CD to sync the application
  4. Deployment: Argo CD deploys the new version to Kubernetes

S3 Bucket Naming (if using S3 deployment)​

When using S3 deployment, buckets are automatically named using the pattern:

`` `{PROJECT_ID}-welance-{APP_ENVIRONMENT}` ``

Examples:

  • Development: p222-01-welance-develop
  • Staging: p222-01-welance-staging
  • Production: p222-01-welance-production

Differences from Other Pipelines​

  • Hybrid Deployment: Supports both GitOps/Kubernetes and S3 deployments
  • Docker Build: Uses .docker-build for building containerized static site generators
  • Targets master Branch: Uses master instead of main for production
  • Node.js Caching: Includes specific caching for Node.js modules
  • S3 Support: Includes S3 deploy job template (can be extended for direct S3 deployments)

Prerequisites​

  • Dockerfile: Project must have a Dockerfile for building the application (if using containerized builds)
  • GitLab Container Registry: Must be enabled for storing images
  • GitOps Repository: Must be configured and accessible (for GitOps deployments)
  • Argo CD: Must be configured and accessible (for GitOps deployments)
  • AWS Credentials: Must be configured for S3 deployments (if using)
  • Semantic Versioning: Branch names must follow release/{version} or hotfix/{version} format

Notes​

  • Production deployments require manual approval
  • The default pipeline focuses on GitOps/Kubernetes deployments
  • S3 deployment job template is included but not used by default - extend the pipeline to add S3 deployment jobs
  • The pipeline validates semantic versioning before staging/production deployments
  • Node.js modules are cached to speed up pipeline execution
  • Release jobs create GitLab releases with container image links
  • The pipeline targets master branch (not main) for production
  • Environment-specific .env files (.env.dev, .env.staging, .env.production) are used for GitOps deployments
  • The pipeline excludes Mittwald deployments (STAGING_TARGET != "mittwald", PRODUCTION_TARGET != "mittwald")