Write a distributed app in days, build a business on DCOS | D2iQ

Since Mesosphere announced the Datacenter Operating System in December, we've been talking about all the services we support, datacenter services that you can install in your datacenter or cloud environment almost as easily as you install an app on your laptop or smartphone. Need a file system? Install HDFS. Need a messaging system? Try Kafka. Need a database? Give Cassandra a go.

Now, we're getting ready to take the wraps off the Mesosphere SDK and developer program.

The developer program and SDK provide tools -- documentation, reference libraries and support -- that will make it simple to develop all types of distributed services or applications on top of the DCOS. The operative words here are simple and distributed. Because it's built with Apache Mesos at its core, the DCOS takes care of common distributed computing headaches such as job scheduling, high availability, resource isolation and networking -- meaning developers don't have to.

The SDK supports development in Java, Scala, Python, Go and C++.

ISVs and developers are already on board

"Who is building new services for the DCOS?" some of you might be asking.

That's a fair question, because Mesosphere's DCOS -- and the concept of a datacenter operating system, in general -- is a relatively new thing. We think the answer is pretty clear, and that it's anyone building anything that could benefit from running in an automated and distributed environment.

For example:

• Developers inside companies that use the DCOS can tackle company-specific pain points by writing new distributed software for the platform that already powers many of the company's other services. We already have large customers in the financial services, media and telecommunications industries, among others, but users of our free cloud-based Community Edition will also have ideas about how they could improve their operations with new distributed services.
• Developers anywhere can write the next big database, file system, stream processing and monitoring tools or pretty much anything they want. They can rest easy knowing they don't have to be distributed-systems experts in order to do it, and that their creations will run the same anywhere that Mesosphere's DCOS is installed. As the community of DCOS users grows large -- and it is, quickly -- there will be a natural market of potential adopters and buyers of custom-built DCOS services.

The latter scenario is already playing out inside some cutting-edge startups and even established vendors:

• Scala creators and platform vendor Typesafe recently released a distribution of the popular Apache Spark data-processing framework that's built specifically to run on the DCOS.
• MemSQL, a red-hot real-time database startup that has already raised more than $45 million from major venture capital firms and strategic investors, has certified its technology to run on the DCOS.
• Smaller startups such ArangoDB and Crate have created versions of their database systems that take advantage of the distributed goodness and easy installation the DCOS provides.

To show off how flexible the Mesosphere SDK is and how easy it is to use, we had two of our summer interns, Lily Chen and Derek Chiang, build their own frameworks almost immediately upon starting their internships. In a few days, Derek built a Bitcoin miner that grabs as many resources as it can from a Mesos cluster. In about a week, Lily had built an iterative version of a divide-and-conquer algorithm for calculating the closest distance between 2-dimensional points in a dataset.

We think developers who approach the Mesosphere SDK with sufficient time, and solid ideas of what they'd like to build, will be able to create some truly powerful products. Maybe it will be the next Cassandra, Spark or Cloud Foundry, or perhaps a new take on an end-user application designed with scalability as a guiding principle.

The power of Mesos, illustrated

A good example of this is a startup called Concord Systems, which has created a real-time stateful stream processing engine that's built on Mesos right from the start. Concord has essentially bet its business on Mesos as an enterprise-grade distributed kernel, and is working with Mesosphere to take full advantage of the DCOS, as well.

"Without Mesos, we would have to -- completely parallel to the development of our stream processor -- develop a resource allocator and scheduler," explained Cole Brown, co-founder and VP of engineering at Concord, during the Mesosphere HackWeek earlier this year. "We'd have to deal with isolating processes, which is incredibly important to us. We'd have to deal with communicating failure and system health."

He continued: "One thing that we're incredibly excited about is the DCOS platform. We see the DCOS platform as the first real step toward developing a strong community around Mesos, where a cohesive set of tooling and a good package manager makes it simple to deploy distributed apps, which previously has been an extremely difficult task requiring a dedicated staff."

 

Now's the time to get on board

As part of opening up our SDK and developer program, Mesosphere is also kicking off a VIP Partner program that should help companies of all stripes reap the same benefits of early partners such as Concord, Typesafe, MemSQL, Crate and ArangoDB. These could be ISVs like those just mentioned, systems integrators such as early partner Container Solutions, or even hardware vendors such as early partner Cisco.

Existing companies with technologies that could benefit from added scalability and resilience the DCOS provides, or that could complement a platform that will power the next generation of enterprise applications, will want to join our fast-growing DCOS ecosystem. We're building the future of enterprise IT, and now is the time to get on board.

If you want to be among the first to test out the Mesosphere SDK and see how powerful it really is, sign up for early access. We expect to open it up for public consumption -- and to see the next generation of distributed applications start emerging -- later this year.