Jul 30, 2018
Laura Kelso
D2iQ
4 min read
There's big, and then there's GE big. GE's 300,000+ employees are in the business of designing, building, operating and servicing industrial assets and infrastructure, and much of that history and knowledge is infused into the software created by GE Digital. GE Digital's Predix portfolio of products enable customers from across the industrial spectrum to drive new levels of productivity by connecting machine data with the powerful Predix platform and industrial applications. A key part of GE Digital's platform success comes from the practices and processes established by the Digital teams. A great example of one such practice is the Continuous Integration & Deployment (CI/CD) platform that enables the thousands of GE developers and staffers to serve more than 1,000 customers, built on the capabilities delivered through DC/OS.
At a meetup held at O'Reilly's June 2018 Velocity Conference, Seshagiri (Seshu) Nuthalapati, one of GE Digital's directors of engineering, took to the stage to describe the challenges he faced in creating a companywide cloud-based and open-source development environment and how he chose to tackle them.
Nuthalapati had four main goals going into the project:
- Increasing developer productivity by creating well integrated self-service tools that let developers focus on writing code rather figure out how to manage complex CI/CD functions;
- Reducing the time to release new applications from one to three months down to two to four weeks;
- Improving software quality with a metrics-driven infrastructure that included automated pipeline states;
- Optimizing IT costs by consolidating many systems from around the company into a single reliable and scalable development platform.
To achieve those goals, Nuthalapati had to manage legacy internal applications and systems, limited capacity, hard-to-manage shared environments, a lack of container capabilities, and a lack of established continuous delivery practices in some of the other GE business units. It was crucial to create a pipeline-driven development platform, one in which developers didn't need to spend significant time setting up and maintaining a CI/CD environment.
Knowing that he wanted to move forward with a cloud-based solution, Nuthalapati chose DC/OS along with Marathon for orchestration of Docker containers. For distributed storage he chose GlusterFS, and when it came to the CI/CD tools themselves, he focused on Jenkins, Sonarqube, Whitesource—which manages open source license compliance and security—and Tinfoil, Enterprise DTR, Artifactory, Github, and, for code analysis, Coverity.
Built on top of the underlying cloud infrastructure, DC/OS provides the platform layers that manage resources and orchestrate containerized applications, bringing together API gateways, microservices, system and user-level metrics with a unified control plane. The result is a single and easy-to-manage pipeline for the entire development process. Even legacy tools can connect to this highly flexible system.
In the new environment, all development, including the build stages, is shifted into Docker containers within DC/OS. "I deploy microservices," said Nuthalapati. "As a developer, do I care how it's all running? No. The system should take care of things. I just want the orchestrations to happen. That's why I chose DC/OS and Mesos."
In a live demo, Nuthalapati showed off the stability and scalability of DC/OS as it sustained a load 507 of containers, 254 Jenkins instances, and 120 Sonarqube instances. After eight months of service, he noted, the system has never been down. He also pointed out that one of the big advantages of DC/OS was its ability to accommodate Docker-in-Docker containers, an innovation so successful that 90 percent of the platform's VMs are currently operating in that fashion.
"DC/OS is the right fit. It has all the components we're looking for," Nuthalapati said. The number of development pipelines per day is up, as is the overall growth rate of new pipelines. And the story is really just starting. "Only five percent of GE traffic is on this cluster so far," he added, "so we are ahead of the game and making progress every day."
To learn more, watch Nuthalapati's entire Velocity presentation.
Seshagiri (Seshu) Nuthalapati, Director of Engineering at GE Digital
Seshu, director of engineering at GE Digital, has been leading the team of full-stack engineers responsible for creating Propel, an internal CI/CD self-service platform built on DC/OS, that provides users with the ability to spin up Continuous Integration tools (Jenkins) and a static code analysis tool (Sonarqube) within minutes – all through a simple configuration dashboard. It also integrates seamlessly with other Build.GE tools such as Github, Artifactory, etc.
