Shaun Abram

Technology and Leadership Blog

Why use containers?

Containers, and the supporting orchestration platforms, are increasingly popular tools for deploying applications. This article focuses on why you would want to use a container ecosystem. While there are many reasons, including portability and reduced boot times (compared to VMs), this article concentrates on security and cost.

We will focus on Docker, since it is by far the most dominant container, and on Kubernetes, since it seems to have “won” against alternatives such as Swarm and Mesos.

Contents

1 Security
2 Costs
3 Summary
4 References

Security

Mean time to update

A big benefit in moving to a containerized environment is to improve your “mean time to update”, which has huge implications from a security perspective. “Update” is meant in a broad sense here, including:

releasing a new version of an app
patching OS and application stack vulnerabilities
updating or rotating credentials
just regularly “repaving” instances from known good images, even if there is no new software to deliver

App instances which are regularly rebuilt, particularly with updated credentials, are a moving and much more challenging target for those trying to compromise our systems. A related and excellent read is The Three Rs of Enterprise Security: Rotate, Repave, and Repair.

Namespaces and cgroups

Regardless of the mean time to update, there are a number of other security enhancements that are available when using containers. At the core of the concept of containers are the Linux kernel features of namespaces and cgroups. When you start a Docker container, you are really creating a set of namespaces and control groups as a “container” for your app to run in, and these can be used to provide both the isolation most users seek in using containers, and the security required for their use.

Namespaces provide isolation in the form of logical partitions; a wrapped a set of resources that appear to be dedicated to your container. A process running in one container cannot see processes in another container. The concept of namespaces in Linux is almost 20 years old and so has been battle tried and tested in many production environments.

Control groups, or cgroups, place limits on the resources each container can use. This means that containers can share common resources (such as CPU and memory) without stepping on each other toes. While control groups are primarily about isolation and stability, they also have an important role to play in security, since the limits they impose can be used to prevent container-based DoS attacks.

Kubernetes security features

In addition to Linux/Docker security features, Kubernetes also provides additional security mechanisms and capabilities. For example, role-based access control (RBAC), Admission control, and the ability to specify a security context in the pod definition.

Building security into your proxy

Edge Proxy

Another way to enhance security when using containers is to utilize a reverse proxy in front of the cluster – a so called “edge proxy”. Such a proxy can handle networking and security features so that each and every individual containered microservice does not need to. Common tools include F5, HAProxy, Nginx and Envoy. For example, rather than having each service handle TLS communication, perhaps using a common library, it may be better to have the proxy handle it for all services in a centralized and consistent manner. TLS can be terminated at the proxy leaving the services to happily communicate in plain text behind the proxy.

Service mesh

This concept of pushing core security features out to a proxy can be taken a step further with the concept of a service mesh, using tools such as Envoy or Linkerd. A service mesh can be broadly considered as the network of microservices that make up your application, but is often more narrowly considered as the infrastructure layer of that network, outside of your apps, for dealing with service-to-service communication. Each service instance now has a “sidecar” proxy running alongside it in the same VM or Kube node. For example, each sidecar can handle mTLS termination, meaning that services continue to communicate using TLS across the data center (sometimes a corporate requirement in public cloud environments such as AWS), while the individual service itself only communicates to the sidecar in plain http. You now have the benefits of mTLS for communication across the data center, but being handled by the sidecar so that individual services don’t need to worry about it.

Such service meshes can be paired with a “control plane” such as Istio.

Costs

Using Docker and Kubernetes, we can get higher app density per ECS instance type, without impacting performance (at least without doing it in unacceptable ways).

There is a limit on how many instances you can run on an EC2 instance type. For example, you are limited to running a maximum of 20 On-Demand instances across the AWS EC2 instance family. These specific limits can be raised by opening a Support case with AWS, but there will always be some limit imposed by AWS.

No such limit exists for containers, or at least, you have more control. In addition, containers typically facilitate higher app density because they do not require a full OS. 1 VM = 1 OS. Multiple containers share a common, underlying OS. Finally, Docker, Kubernetes, and the underlying Linux OS support various methods for maximizing the underlying resources.

For example, one approach to deploying apps in an isolated manner, without using containers, is to deploy 1 app per VM. This does keep things simple and isolated. However, many of your apps may have very low CPU and memory requirements, for example, they may only be using 1% of the available CPU. I’ve heard of apps running in production which are vital on occasion, but for the most part, the health-check is the biggest consumer of CPU! In such cases, we may end up underutilizing the CPU, memory and other resources on the EC2 instance types in use. In essence, you pay for resources that are not being used.

With Docker and Kubernetes, we can deploy more containers per “bare metal” EC2 instance type. You could conceivably double the number of apps running without reaching CPU limits. This approach is not without is issues either though. We need to protect against “noisy neighbors”. If one of the apps becomes resource (e.g. CPU) hungry, we need to ensure that it does not adversely affect the other app. Fortunately both Docker and Kubernetes provide tools to manage this. Docker for example allows you to limit a container’s resources, such as access to memory (–memory) and CPUs (e.g. –cpu-quota) and Kubernetes has the concept of Resource QoS (Quality of Service), which helps manage pods by different priorities.

Utilizing containers for cost cutting reasons is not a new idea. For example, QBox reported saving 50% per Month on their AWS Bills Using Kubernetes. And there are also many other ways to reduce costs, including using on premise data centers (or “colos”), and using reserved instances in the cloud. But containerization with Kubernetes is still a useful option to consider.

Summary

The whole container ecosystem is exploding right now, particularly around Kubernetes because the tools really do deliver real benefits. Those include portability and reduced boot times, but the security and cost considerations covered here are also important and hugely beneficial.

References

Tags: containers, continuousdeployment, docker, k8s, kube, kubernetes

28 Feb 18 | Architecture, OpenSource, Products