Docker Container Monitoring: Protect Your Investment
Table of Contents
Has your boss asked you about Docker containers yet? No? Well, I’ll wait. He’s probably on the way to your desk or office right now.
Why? Because your company is probably already using containers, whether it’s for testing or for production, and Docker is the most common container platform out there. The companies using Docker containers are using more every year. Docker containers offer flexibility, efficiency, and work well with DevOps and agile methodologies. However, they also raise cybersecurity challenges.
If you’ve begun using containers, you’re likely wondering how best to protect your investment. Whether deployed on-premise or in the cloud, you need to know how your containers are performing, especially since containers are becoming the backbone of mission-critical services.
Effective monitoring will allow your system administrators to react to and resolve issues quickly. To set up the right monitoring, you’ll need to know what to monitor and what tools are available to help you. First, we’ll look at the challenges of monitoring Docker containers.
The Unique Challenges of Monitoring Docker Containers
Containers help you to use computing resources efficiently. They’re easy to deploy and are perfect for holding microservices. This separation of small pieces, however, leads to increased complexity.
Your users see one application. To them, microservices are like the ants behind the ant hill. You see the results of the work, but not the tiny little soldiers working together to build it. Your application is not one piece, but multiple separate services that link together to perform the necessary functions.
Now each one of these little services and the containers they occupy needs to be monitored. Monitoring such complex architectures is a challenge. But it can be done effectively.
What Should You Monitor?
Performance metrics are the basic measurements to start with. Containers are processes and need to be allocated memory and CPU cycles. Keep an eye on how much memory and CPU is taken up by your Docker containers. Watch network latency to ensure your services can communicate quickly with each other. Filesystem monitoring ensures your host machines have enough space to hold the data your services require.
Application metrics are key measurements that are application specific and usually tracked using the code. HTTP error messages, container error messages, and application-specific error messages fall into this category. Services that process a queue of jobs should keep track of jobs processed, jobs in error, and other messages. Good application metrics allow you to react to application performance issues before they cause a problem for users.
What About Security?
Docker container Monitoring shouldn’t include only performance or application metrics, but security policies as well. It can be easy to overlook security when dealing with new technology. Here are some tips on keeping your Docker containers secure.
Container images make downloading and using containers easier by using open source libraries for much of the needed functionality. While code scanning is often used to prevent vulnerabilities from creeping into codebases, open source libraries could introduce vulnerabilities without your knowledge. Use container scanning tools to look for unsafe versions of operating systems and open source libraries.
Deploy your containers with known safe configurations. Look out for any containers that aren’t operating according to those configurations. Don’t run containers as root. Only pull container images from a known safe registry. Use secure network configuration, such as network access control lists, to control which containers can speak to each other. Don’t forget to secure your orchestration solution as well, be it Kubernetes or any other you may use.
Basic container security monitoring coupled with good security policies reduce risk and exposure. One compromised container could lead to host takeover or denial of service. Don’t let security monitoring slip through the cracks, or an attacker might slip through too.
Monitoring Kubernetes
Container orchestration tools are quickly becoming necessary, and Kubernetes is the leader in this space. Kubernetes allows you to monitor all of your containers and replace them as needed. It helps run all of your container infrastructures but also introduces some new challenges.
Kubernetes follows a philosophy of treating containers like cattle. Containers are ephemeral: if anything goes wrong they are destroyed and a new one is created. Thus a challenge arises of knowing what containers are currently running, for how long, and what occurred to the ones you had before. Containers must also have good labeling conventions to help keep track of what’s happening in your environment.
Monitoring Kubernetes involves monitoring clusters, which contain pods within them, with pods holding the actual containers. Clusters have to be allocated memory and CPU resources. It’s essential to monitor Kubernetes to know when to scale your containers up or down. When monitoring Kubernetes installations, you need to break each of these larger pieces down to their individual parts and get statistics from the individual containers for the best information.
Several open source tools exist to help monitor your Kubernetes installation and run it efficiently. We’ll look at four tools–two that are built-in pieces of Kubernetes and two that use the built-in pieces to create a comprehensive view of your clusters.
The Kubelet is a bridge between the Kubernetes master and the nodes. It breaks down each pod into the separate containers within and gathers metrics on the individual containers. It uses the containers built-in runtime API to retrieve this information. It then aggregates the information and exposes it through its own API.
Another open source container monitoring agent is cAdvisor. cAdvisor is included in the Kubelet binary and is the mechanism used to gather statistics on resource allocation and performance that is fed back to the Kubelet. cAdvisor includes a rudimentary UI where you can see the live performance metrics of your containers, including CPU, memory, filesystem, and network usage statistics. cAdvisor is a limited tool, however; you shouldn’t depend on cAdvisor alone.
Kubernetes Dashboard is a user-friendly UI for managing Kubernetes. It allows you to manage the applications running in the cluster, as well as the cluster itself. You can use Dashboard to manage load balancing, configuration, and launching or tearing down of containers and clusters. It is helpful for smaller Kubernetes installations but may not fit your needs if you have a large Kubernetes footprint.
Prometheus is an open source tool that gathers statistics using all of the built-in features and adds powerful functionality on top. It features awesome visualizations and alerts based on what you define is important. One of its most powerful features is the PromQL query language used to query the data your Kubernetes clusters is feeding it. You can search for the data that’s important to you and turn it into a graph for easy consumption. If your Kubernetes cluster is large and difficult to monitor, check out Prometheus — it may save you many monitoring headaches.
Protect Your Investment With Docker Container Monitoring
No matter how many Docker containers you’re running, monitoring is the key to protecting your investment.
If you are just starting out with a few containers, begin monitoring them with cAdvisor or similar small tools. You’ll build the skills necessary for monitoring at a larger scale.
As you grow your container footprint, start using orchestration tools like Kubernetes to better manage and monitor your containers. Kubernetes Dashboard is a good start. Prometheus is even better.
And never forget to secure your containers. Don’t download images from random repositories on the Internet. Scan for open source vulnerabilities. Never run containers as root.
Like any technology, containers offer great benefits and a new set of challenges. Navigate these challenges successfully to protect your investment in this new technology.
