Use Cases

How to Run Elastically Scalable CI/CD on DC/OS

Jan 31, 2019

Alex Hisaka

D2iQ

5 min read

We have chosen to sunset DC/OS, with an end-of-life date of October 31, 2021. With D2iQ Kubernetes Platform (DKP), our customers get the same benefits provided by DC/OS and more.

Learn more about D2iQ Kubernetes Platform here.
There can be significant challenges in implementing a full CI/CD toolchain. However, ensuring that your organization adopts this process is a business imperative for scaling digital business transformation. So, how can an organization quickly implement and deploy CI/CD with minimal friction? Luckily, there are additional platforms that you can leverage in order to significantly simplify deployment and time-to-value. Mesosphere DC/OS is the premier platform for building, deploying, and elastically scaling modern applications and big data. DC/OS makes running containers, data services, and microservices easy across your own hardware and cloud instances.
 
What is DC/OS and How Does it Work?
DC/OS provides enterprises with a powerful platform to meet the challenges that enterprises face when implementing a complete DevOps strategy. It's a flexible platform built on top of Apache Mesos, an open source resource manager. Mesos is fault-tolerant and scalable to 10,000 nodes or more, which is why it's used in production by many large-scale enterprise companies. At its lowest level, Mesos enables more efficient resource utilization for CI/CD workloads.
 
Mesosphere DC/OS relies on Mesos' two-level scheduler to make high-availability and elastically scalable CI/CD infrastructure straightforward to implement. Resources are shared across DevOps teams, dramatically increasing utilization, while strong isolation and role-based access control regulates their usage. Health checks ensure that developer tools are always up and running so if an instance fails, it's automatically restarted without data loss—which is business-critical.
 
Because DC/OS pools resources, teams can set up and tear down environments for development, test, staging, and production with just a few clicks. The CI/CD process includes Source Code Control, Build and Test, Release, Deploy, Monitor, and Log. Once a service is installed, DC/OS can be run across the entire CI/CD pipeline, datacenter, and any combination of public and private clouds, elastically sharing resources. Development teams face no restrictions when running application code, containers, and distributed applications.
 
The benefits of DC/OS encompass the entire application lifecycle. The ease with which new infrastructure can be deployed makes it possible to leverage any infrastructure configuration across isolated development, test, staging, and production environments. Additionally, DC/OS shares resources across DevOps teams to dramatically increase utilization. Developers enjoy deploying their favorite CI/CD tools from a self-service portal. DC/OS scales CI/ CD tools as needed to provide a high-performance experience for developers. This in turn increases developer productivity.
 
Scaling and Securing Jenkins for Improved Developer Productivity
DC/OS offers many scalability and security benefits for Jenkins, a highly popular build automation tool. DC/OS automates infrastructure functions so CI/CD services, like Jenkins, can scale up and down elastically based on the number of developers and build jobs. Instead of the static partitions that are common in the Jenkins world, Jenkins for DC/OS can create and destroy agents as demand fluctuates. Multiple Jenkins masters can share a single pool of compute resources for increased efficiency and resiliency. The Mesos Jenkins plug-in launches agents on demand (from resources that were previously allocated) and Jenkins scales itself up and down using the Mesos resource scheduler.
 
Automating Deployments for Improved Operator Productivity
DC/OS provides much needed automation, reliability, and flexibility to the CD process. The combination of Jenkins, Marathon (or Kubernetes) orchestration, and Mesos resource scheduling allows DevOps teams to rapidly construct CD pipelines that deploy code changes to production within minutes. The Mesosphere-developed Marathon Plugin for Jenkins communicates directly with the Marathon API, enabling seamless deployments without complex scripting.
 
Flexibility in Different Methods of Deployment
The combination of DC/OS and CI/CD provides support for many different methods to get new software from development to production. This is the final section of the CI/CD pipeline that enables teams to rapidly deploy new features that create transformative customer experiences.
The deployment methods supported by DC/OS for use in the CI/CD process include:
 
  • In-Place: Rapidly replaces all code on existing servers at once. In-place deployment can cause downtime because code needs to be replaced and servers restarted. A failed deployment results in downtime, while rollback is difficult and time-consuming.
  • Rolling: Servers are updated one at a time so that, for a brief period, old and new software versions are both running in the same environment. Rolling updates are a favorite because they allow new software to be deployed without downtime, and if the deployment fails, only the updated instances are affected.
  • Canary: A variation of Rolling Deployment, Canary deploys the new code to a very small subset of servers (maybe even just one) before deploying to the entire production environment. This method is useful for testing and monitoring how the new software behaves in production. With Canary, it is very easy to roll back and recover from failures.
  • Immutable: Deploys onto a completely new set of servers. Immutable deployment leverages the ability 4 of cloud-native infrastructure to create new server instances programmatically through API calls.
  • Blue/Green: A variation of Immutable Deployment. The existing production environment is considered "blue". A new "green" environment is created, and the complete application is deployed to the new "green" environment. Once it passes all tests, users are sent to the application running in the "green" environment. The old "blue" environment remains idle in case it's necessary to roll back.
 
Enterprises are rapidly realizing the competitive advantages of leveraging DevOps methodologies to accelerate software development and deployment using CI/CD toolsets. However, manually integrating, administering, and scaling CI/CD tools is no easy task. Mesosphere simplifies application development, deployment, and ongoing operations, fulfilling the mission of DevOps to more closely align technical and business teams to provide the best customer experience possible.
 
To learn how you can run elastically scalable CI/CD pipelines in your company, download the ebook, "Implementing Elastic CI/CD."

Ready to get started?