Mar 20, 2018
D2iQ
D2iQ
7 min read
One of the goals of DC/OS 1.11 is to improve the security of the container-based applications and data services that run on it. We're excited to deliver the multi-layer security you need to protect your data-intensive applications.
The foundation of DC/OS's security is an encrypted control plane that secures internal cluster communications. On top of that we protect DC/OS itself with sophisticated role-based access controls (RBAC). In addition, applications and data services running on the DC/OS cluster are isolated with strong perimeter security using a public and private agent architecture.
DC/OS 1.11 augments the above security features so you can know that your fast data initiatives run on a platform that protects your valuable data. We've recently strengthened our multi-layered approach to security with the following additions:
- Secure communications within distributed data services
- Control over which applications can access data services
- Improved secrets management
Challenges in Securing Data-Intensive Applications
Most IT shops rely on a series of manual procedures (that vary from package to package) to secure distributed data services. For example, manually configuring and securely deploying the distributed components of the popular SMACK stack, such as Apache Cassandra and Apache Kafka, involves roughly 20+ time-consuming and error-prone manual steps per service (that's 100+ steps across the whole stack). Many of these steps require manually creating directory service entries, distributing certificates, editing configuration files and restarting services to apply security settings.
In general, securing data services involves:
- Satisfying environmental prerequisites such as securely creating, distributing, and storing the proper certificates and, as necessary, reconfiguring host operating systems.
- Configuring the service's internal access control list (ACL) to provide authorization and apply role-based access controls per user, application, and service.
- Integrating the data service with external directory service for Kerberos-based authentication and authorization mechanism, such as LDAP or Microsoft ActiveDirectory, and creating all needed accounts with the right privileges.
- Configuring transport encryption and trust for each instance of the service, between each service node and between clients and the service.
- Restarting related services to apply security configurations.
- Securely distributing credentials and certificates to client applications.
DC/OS 1.11 Automates Security for Data-Intensive Applications
With so many difficulties inherent to securing distributed application and data service environments, it's no wonder the news is full of data breaches and tales of failed digital transformation initiatives.
Fortunately, DC/OS 1.11 includes a combination of features that automate authentication, authorization, and transport encryption for our certified data services. DC/OS standardizes security functionality across data services with a consistent configuration and management experience via the DC/OS GUI and command line. DC/OS transforms the tedious collection of manual tasks described above into a few checkboxes. Within minutes, you can configure and enforce a security policy for the data-intensive applications your business relies on, with consistent results every time on any infrastructure. Combining security automation with existing application-aware scheduling means DC/OS automates and secures the entire application lifecycle enabling a fully elastic infrastructure.
Sophisticated hierarchical secrets management provides the core of security services in DC/OS 1.11. The DC/OS Secret Store secures sensitive information including passwords, API tokens, private keys, and binary files. Secrets are stored and managed hierarchically ensuring only pre-authorized users and services can access them. Services retrieve secrets at deployment and store their values as environmental variables. You can work with secrets through the DC/OS GUI, command line, or via the Secrets API. DC/OS leverages an internal PKI service to generate, sign, and manage X.509 certificates, and provides them to the DC/OS Secret Store for use with infrastructure components, applications, and data services.
Let's take a closer look at security protection in DC/OS 1.11.
DC/OS 1.11 Secures communications within distributed data services
Transport encryption via Transport Layer Security (TLS) and Secure Socket Layer (SSL) is widely used to protect data transmitted across networks. Both SSL and TLS provide certificate-based authentication and rely on asymmetric encryption to privately share a symmetric key, then use symmetric encryption to encrypt data in transit. For the most part, TLS has replaced SSL as the encryption protocol of choice, although some clients still require SSL. Many environments require TLS encrypted communications in order to ensure compliance with regulations such as PCI and HIPAA.
Encrypt data-in-transit between different data service components
Starting with DC/OS 1.11, Apache Cassandra, Apache HDFS, Elastic, Apache Kafka, Datastax Enterprise, Confluent Kafka all support full automated transport encryption on top of DC/OS. While these services have support for transport encryption built-in, enabling and configuration typically requires numerous steps, such as manually creating the dozens of read-only and read-write certificates required for each instance of each distributed service.
Tight integration between DC/OS and these data services reduces this to three steps that are consistent between packages: create service accounts, set service account permissions, and enable SSL/TLS. DC/OS automatically creates the multitude of secrets that each service node requires. After installing the service (or, if the service is already installed, performing a non-disruptive rolling restart), transport encryption is active and communications are secured. As additional data service nodes are added, DC/OS repeats this process automatically, saving time and preventing potential user error.
Once transport encryption is enabled, communication between different components of each service is encrypted, and only trusted data service components can communicate within the cluster. Depending on your requirements, plain text communication can also be allowed.
Encrypt data-in-transit between client applications and data services
DC/OS 1.11 extends the same transport encryption functionality and automation to secure communications between client applications and data services. For example, you can configure your applications to always use encryption when reading and writing data to and from data services like Cassandra, Hadoop, Elastic, Kafka, and more. This is critical when reading and writing data across security domains such as datacenters, internal networks, public cloud providers, the public internet, and partner networks.
DC/OS 1.11 controls which applications can access data services
Transport encryption is only part of the protections DC/OS relies on to secure communications between client applications and data services. First, clients must engage in identification and authentication via an open source Kerberos provider (such as LDAP) or Microsoft ActiveDirectory. Once authenticated, client applications must then be authorized by the data service they are accessing, and then all communications are encrypted using SSL/TLS.
Client application authentication
You can enable client authentication for connections from your application to your data service in order to maintain tight control over which applications can connect to which data services and whether they can read from and write to them. DC/OS provides support for Kerberos authentication for Apache Kafka, Apache Hadoop, Datastax Enterprise, Confluent Platform, and Apache Zookeeper/Confluent Zookeeper via LDAP or Active Directory, and support for SSL certificate based authentication for Kafka.
Client application authorization
You can enable client application authorization of read and write operations to and from data services running on DC/OS. After performing identification and authorization via Kerberos or SSL certificates, client applications are subject to data service-specific access control mechanisms. This allows you to apply granular control over specific services. For example, you can define specific applications that are allowed to read from a specific Kafka topic. You can also restrict write access to Kafka topics to prevent data pollution or fraudulent activities.
DC/OS 1.11 Improves secrets management
IT departments are tasked with managing multitudes of secrets in order to secure application and data infrastructures. There are certificates, encryption keys, connection strings, credentials, binary files, and more. Operators, developers, and security admins all play a role in managing secrets, plus services running in a variety of locations need access to secrets. Secret management becomes exponentially more time-consuming and complex as environments expand.
DC/OS provides a centralized encrypted and access-controlled location for secrets, and a simple and automated way to enroll or generate these secrets. When applications launch, they are automatically provided with the right secrets, decreasing the likelihood of configuration error.
In addition, DC/OS 1.11 adds sophisticated hierarchical management features to the DC/OS secrets store. Secrets can be grouped together in a namespace and user privileges to list, get, modify, and create secrets can be managed via that namespace. In this way, secrets are isolated between users and teams, ensuring that they can only be accessed by the appropriate party and/or service.
DC/OS 1.11 Automates Multi-Layer Security for Data-Intensive Applications
Mesosphere DC/OS streamlines and automates security (identification, authentication, authorization, and transport encryption) for client applications and data services. Please see documentation for additional details. To learn about the full release of DC/OS 1.11, please see our announcement blog. To see a full product demo, RSVP for the DC/OS 1.11 webcast.
