Run Tasks in AWS
Background
There are a number of tasks that exist in AWS as part of each environment. These vary depending on the environment, and what features it has enabled (there won’t be data-sync tasks if data-sync isn’t enabled).
These tasks fall into a few categories:
- Scheduled business focused tasks.
- Scheduled utility tasks.
- Environment setup/testing tasks.
Scheduled business focused tasks can be triggered against the environment of your choice using the Run Nightly Task Jenkins Job. Environment setup/testing tasks are usually executed via the Create Dev Environment or Behat Tests Jobs.
However you may need to run a utility task, you might be working on a new task which isn’t included in the job, or you may just want quicker feedback than the Jenkins jobs are able to give you. This guide will show you how to run tasks from your local machine.
Prerequisites
You need to have the following tools installed and configured.
- aws-vault - make sure this is configured with the identity profile referenced in the new starter guide.
- direnv - make sure you hook it into your shell correctly following these instructions.
- ecs-runner - This is an in house built utility.
- terraform - make sure you install the version currently being used by #opg-starfox.
You can install all of these tools via homebrew, so it’s useful to have that installed.
brew install aws-vault
brew install direnv
brew install ministryofjustice/opg/ecs-runner
brew install terraform
Make sure to also pull down the opg-sirius-infrastructure repository.
Running a Task
We’ll be working in the context of the opg-sirius-infrastructure repository, within the environment directory. All your commands that interact with AWS will be prefaced with aws-vault and the identity profile.
Update the .envrc file to set the TF_WORKSPACE variable to the environment you want to execute the task against, this will be
what you entered as your ticket number in the Create Development Environment Jenkins job.
Run direnv allow to point at the environment you want to execute the task against.
Initialise terraform: aws-vault exec identity -- terraform init
Create the terraform.outputs.json file which the ecs-runner uses:
aws-vault exec identity -- make outputs
The role block at the top of the terraform.outputs.json file might reference sirius-ci, you can’t assume that role as it’s too powerful,
so amend it to reference operator.
Before executing the task, check the terraform.outputs.json file to ensure that the task definitions are pointing at the right
environment, using the correct account_id. You can find the environment references such as the account_id within the
terraform.tfvars.json file.
You may wish to run a mock task (such as the status-api task below) to ensure that everything is set up correctly before running your
task.
Execute your task:
aws-vault exec identity -- ecs-runner -task {your task name} -timeout {seconds}
eg.
aws-vault exec identity -- ecs-runner -task status-api -timeout 300
That will start the task and stream a copy of the logs out to your terminal.
Gotchas
If you’re deploying a new version of the application into the environment the you need to refresh the outputs, and ensure that
the role is upto date with operator