Jun 15, 2015
D2iQ
D2iQ
4 min read
Mesosphere and Typesafe have put a lot of effort into making the popular Apache Spark data-processing framework run on Apache Mesos, and today that work has reached the next level with the announcement of Spark on DCOS -- a specially built distribution of Spark that's optimized for Mesosphere's Datacenter Operating System.
Spark on DCOS has been certified by Databricks, meaning it is interoperable with other Spark distributions and the broader Spark ecosystem.
The technological benefits of such a product should be obvious to anyone familiar with the two projects. Spark is the big data project that's taking the world by storm -- replacing Hadoop MapReduce in many instances, and serving as the default engine for new workloads -- because of its focus on easy programming and high performance. It's the heir apparent to MapReduce, and is the most-active project in the entire Apache Software Foundation.
Mesosphere's DCOS is the platform that lets users run Spark, as well as a slew of other big data frameworks and application types, on a single shared cluster. Apache Mesos serves as the core of the DCOS and handles resource scheduling, orchestration and high availability across the cluster. Mesosphere's commercial DCOS technology provides tools for easily installing, managing, monitoring, and building these types frameworks and services.
But the icing on the cake when it comes to Spark on DCOS is that Typesafe will be providing enterprise-class support for the product. Mesosphere already provides enterprise-class support as part of its DCOS Enterprise Edition announced last week. This means companies can embrace two new technologies at once, and sleep easy knowing the vendors' claims around performance, usability, scalability and availability are backed up by contracts.
If you're asking yourself why Typesafe has partnered with Mesosphere on Spark on DCOS, you needn't look further than Scala for the answer. The founders of Typesafe helped create Scala and the company is still its driving force. Spark -- which is written in Scala -- becomes the natural choice for giving developers a proven, reliable and familiar big data framework.
Mesosphere also shares a lot of values with Typesafe, such as a fondness for "Reactive" applications -- applications that resilient and elastic -- two things that Mesosphere knows something about. The DCOS is built around the idea that it should be easy to give applications the resources they need, and that a hardware failure shouldn't take an application down with it.
There is a new class of applications coming online that take all these ideas -- resiliency, elasticity, usability and even big data processing -- as their baseline. Mesosphere is proud to partner with Typesafe to help make developing them an even better experience.
