Nov 06, 2015
D2iQ
D2iQ
3 min read
This post was written by Abhishek Amralkar of Talentica Software.
In this post, we explain how to use the CI/CD environment CircleCI with Mesosphere's Datacenter Operating System (DCOS). We are using DCOS for our testing environment and it works seamlessly. Our aim is to deploy our containers as Marathon apps in DCOS, and we use CircleCI for continuous integration. CircleCI creates the Docker images and pushes them into a private Docker registry.
One of the manual tasks that we were doing was to update Marathon apps manually after code changes. However, it turns out that we can automate this process via CircleCI as well.
So, to start off, we assume that you are running your DCOS on Amazon Web Services, as we will use Amazon S3 to store our JSON and shell script. In turn, CircleCI will download those at runtime and execute it.
Setup
The first step is to append the following lines in the dependencies section of your circleci.yml file:
Dependencies: Pre: - pip install awscli - aws s3 cp s3://aaa-tools/scripts/marathon.json /home/ubuntu/dcos/; - aws s3 cp s3://aaa-tools/scripts/put.sh /home/ubuntu/dcos/; chmod a+x /home/ubuntu/dcos/put.sh - mkdir -p /home/ubuntu/dcos
Make sure to replace the S3 bucket path with your own S3 bucket name and file name. We use the following conventions:
- aaa-tools is the S3 bucket
- marathon.json is the Marathon app spec file
- put.sh is the shell script that updates Marathon via its HTTP API
Our marathon.json file looks as follows:
{ "id": "aaa-notifier", "cpus": 0.4, "mem": 256, "uris": ["hdfs://namenode1.hdfs.mesos:50071/artifact_store/.dockercfg"], "ports": [], "instances": 1, "container": { "type": "DOCKER", "docker": { "image": "docker-registry-000-staging:443/aaa/aaa-notifier:latest", "network": "BRIDGE", "portMappings": [{ "containerPort": 0, "hostPort": 0, "servicePort": 0, "protocol": "tcp" }], "forcePullImage": true } }}… and put.sh would contain:
#!/usr/bin/env shMARATHON=http://master-001-dcos4.aaa.net:8080MYAPP=aaa-notifier
cd /home/ubuntu/dcosfor file in *.jsondo echo "Installing $file..." curl -X PUT "$MARATHON/v2/apps/$MYAPP" -d @"$file" -H "Content-type: application/json" echo ""
Now, copy the above two files (marathon.json and put.sh) to the S3 bucket. Then, append the below code snippet at the very end of circle.yml file in the docker section:
- bash /home/ubuntu/dcos/post.sh
After CircleCI triggers the build, it automatically updates the Marathon app and you're done!
How to verify?
We can verify by using the version number; after the build succeeds, we will get the following message in CircleCI:
Installing marathon.json.{"version":"2015-11-04T05:56:03.996Z","deploymentId":"9772b7a4-8870-479d-bfac-a25c93da90a5"}Then, in the Marathon UI check for the app version:
2015-11-04T05:56:03.996Z
This should correspond with what you get from CircleCI.
In this post I walked you through setting up CircleCI with the DCOS and automating the entire CI/CD process. I hope it is useful for you and if you have any suggestions or questions, please leave a comment, below.
