S3 Deploy

The deploy-s3 job is used to deploy static files to Amazon S3 or S3-compatible storage. It synchronizes files from a build directory to an S3 bucket.

Overview

This job:

• Deploys static files to S3 buckets using AWS CLI
• Synchronizes files with delete option (removes files not in source)
• Supports environment-specific bucket naming
• Excludes Git and CI/CD files from deployment

Variables

The following variables can be configured:

Variable	Description	Default	Required
AWS_REGION	AWS region for S3 bucket	"eu-central-1"	No
PROJECT_ID	Project identifier for bucket naming	''	Yes
APP_ENVIRONMENT	Target environment	''	Yes
BUILD_FOLDER	Source directory to deploy	'build'	No

Variable Details

• AWS_REGION: AWS region where the S3 bucket is located (default: eu-central-1)
• PROJECT_ID: Used to construct the bucket name: {PROJECT_ID}-welance-{APP_ENVIRONMENT}
• APP_ENVIRONMENT: Target environment. Options:
  • 'develop' - Development environment
  • 'staging' - Staging environment
  • 'production' - Production environment
• BUILD_FOLDER: Directory containing files to deploy (default: build)

GitLab CI/CD Variables

The job uses the following built-in GitLab CI/CD variables:

• AWS credentials should be configured via GitLab CI/CD variables or IAM roles

Required Secrets

The following secrets must be configured in GitLab CI/CD variables or via IAM roles:

• AWS_ACCESS_KEY_ID: AWS access key ID
• AWS_SECRET_ACCESS_KEY: AWS secret access key
• Or configure the runner with IAM role for S3 access

Usage

Basic Usage

variables:
  AWS_REGION: "eu-central-1"
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'
  BUILD_FOLDER: 'build'

.deploy-s3:
  image: python:latest
  before_script:
    - pip install awscli
  script:
    - export BUCKET_NAME=$PROJECT_ID-welance-$APP_ENVIRONMENT
    - aws s3 sync $BUILD_FOLDER/ s3://$BUCKET_NAME/ --delete --exclude ".git/*" --exclude ".gitlab-ci.yml"

Job Details

Before Script

The before_script section installs the AWS CLI:

pip install awscli

This installs the AWS Command Line Interface, which is used to interact with S3.

Script

The main script performs the following steps:

1. Set bucket name:

   export BUCKET_NAME=$PROJECT_ID-welance-$APP_ENVIRONMENT
   • Constructs bucket name: {PROJECT_ID}-welance-{APP_ENVIRONMENT}
   • Example: p222-01-welance-develop

2. Sync files to S3:

   aws s3 sync $BUILD_FOLDER/ s3://$BUCKET_NAME/ \
     --delete \
     --exclude ".git/*" \
     --exclude ".gitlab-ci.yml"

   Sync Options:

   • $BUILD_FOLDER/: Source directory (e.g., build/)
   • s3://$BUCKET_NAME/: Destination S3 bucket
   • --delete: Removes files in S3 that don't exist in the source directory
   • --exclude ".git/*": Excludes Git directory from deployment
   • --exclude ".gitlab-ci.yml": Excludes CI/CD configuration file

Bucket Naming

The bucket name follows this pattern:

{PROJECT_ID}-welance-{APP_ENVIRONMENT}

Examples:

• Development: p222-01-welance-develop
• Staging: p222-01-welance-staging
• Production: p222-01-welance-production

Configuration Examples

Development Environment

variables:
  AWS_REGION: "eu-central-1"
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'develop'
  BUILD_FOLDER: 'build'

deploy-s3-dev:
  extends: .deploy-s3
  stage: deploy
  dependencies:
    - build
  only:
    - develop

Production Environment

variables:
  AWS_REGION: "eu-central-1"
  PROJECT_ID: 'p222-01'
  APP_ENVIRONMENT: 'production'
  BUILD_FOLDER: 'dist'

deploy-s3-prod:
  extends: .deploy-s3
  stage: deploy
  dependencies:
    - build
  only:
    - main
    - tags

Complete Pipeline Example

variables:
  AWS_REGION: "eu-central-1"
  PROJECT_ID: 'p222-01'
  BUILD_FOLDER: 'build'

stages:
  - build
  - deploy

# Build static files
build:
  stage: build
  script:
    - npm install
    - npm run build
  artifacts:
    paths:
      - build/

.deploy-s3:
  image: python:latest
  before_script:
    - pip install awscli
  script:
    - export BUCKET_NAME=$PROJECT_ID-welance-$APP_ENVIRONMENT
    - aws s3 sync $BUILD_FOLDER/ s3://$BUCKET_NAME/ --delete --exclude ".git/*" --exclude ".gitlab-ci.yml"

deploy-s3-develop:
  extends: .deploy-s3
  variables:
    APP_ENVIRONMENT: 'develop'
  stage: deploy
  dependencies:
    - build
  only:
    - develop

deploy-s3-production:
  extends: .deploy-s3
  variables:
    APP_ENVIRONMENT: 'production'
  stage: deploy
  dependencies:
    - build
  only:
    - main
    - tags

AWS Configuration

Using AWS Credentials

Configure AWS credentials as GitLab CI/CD variables:

variables:
  AWS_ACCESS_KEY_ID: "${AWS_ACCESS_KEY_ID}"
  AWS_SECRET_ACCESS_KEY: "${AWS_SECRET_ACCESS_KEY}"
  AWS_DEFAULT_REGION: "${AWS_REGION}"

Using IAM Roles

If running on AWS infrastructure (e.g., EC2, ECS), you can use IAM roles instead of credentials:

1. Attach an IAM role to the runner instance with S3 permissions
2. The AWS CLI will automatically use the instance role
3. No need to set AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY

Required IAM Permissions

The IAM user or role needs the following S3 permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::*-welance-*",
        "arn:aws:s3:::*-welance-*/*"
      ]
    }
  ]
}

Prerequisites

• S3 Bucket: The bucket must exist and be accessible
• AWS Credentials: AWS access key and secret key, or IAM role with S3 permissions
• Build Artifacts: The BUILD_FOLDER directory must exist and contain files to deploy
• AWS CLI: Installed via pip install awscli in the before_script

File Exclusions

The job automatically excludes:

• .git/* - Git repository files
• .gitlab-ci.yml - CI/CD configuration file

Additional Exclusions

You can add more exclusions by modifying the sync command:

script:
  - export BUCKET_NAME=$PROJECT_ID-welance-$APP_ENVIRONMENT
  - aws s3 sync $BUILD_FOLDER/ s3://$BUCKET_NAME/ \
      --delete \
      --exclude ".git/*" \
      --exclude ".gitlab-ci.yml" \
      --exclude "*.map" \
      --exclude ".env*"

Notes

• The --delete flag ensures the S3 bucket matches the source directory exactly
• Files in S3 that don't exist in the source will be removed
• The bucket name is automatically constructed from PROJECT_ID and APP_ENVIRONMENT
• The job uses the Python image which includes pip for installing AWS CLI
• AWS CLI credentials can be provided via environment variables or IAM roles
• The sync operation is idempotent - running it multiple times produces the same result
• Only files that have changed are transferred, making subsequent deployments faster
• The job excludes Git and CI/CD files to avoid deploying unnecessary files