Hello there!! How is it going?
Welcome to scaleyourapp.com
This article is a comprehensive write-up on cAdvisor – an open source tool for running performance analysis & collecting usage data from the containers. This write-up will answer all our queries about it in-depth such as What is it? Why use it? How does it work exactly?
I’ll also share a bit of my industry experience with the tool.
So, without any further ado.
Let’s get on with it.
1. What is cAdvisor?
cAdvisor stands for container advisor. As it is evident from the name it provides resource usage, performance characteristics & related information about the containers running on the cloud. It is an open-source tool & runs as a daemon process in the background collecting, processing & aggregating useful DevOps information.
The tool has native support for Docker & enables us to track historical resource usage with histograms & stuff. This helps in understanding the resource consumption, memory footprint of the code running on the servers.
The information extracted helps us figure & weed out the performance bottlenecks if any, track the processes too hungry for memory & make better decisions regarding the scalability of our systems.
The official cAdvisor releases are built on Linux & have a small image size.
In Kubernetes cAdvisor is integrated into the Kubelet binary. It is pretty intelligent to auto-discover all the containers running in the machine & collect CPU, memory, file system & network usage statistics. It also provides a comprehensive overall machine usage by analyzing the root container.
Now, what is a Kubelet? I’ve talked about it in the cAdvisor Kubernetes section up ahead in the article. So, stay tuned.
1.1 How Do I Access the Container Data Collected by cAdvisor?
The data collected by cAdvisor can be viewed with the help of a web-based UI which it exposes at its port.
It also exposes its data via a REST API
Typically, the container information returned is the container name, the list of sub-containers, container-spec, detailed usage statistics of the container for the last N seconds, a histogram of resource usage etc.
The machine information returned is the number of schedulable logical core CPUs, memory capacity in bytes, maximum supported CPU frequency, available file systems, network devices, the machine topology – nodes, cores, threads etc.
1.1 My Industry Experience With cAdvisor
In the recent past, I worked on a massive e-Comm project where we used Grafana, Prometheus & cAdvisor to set up a dashboard monitoring system. The dashboards were used to study the server instances, their uptime, exceptions, errors their contextual scenarios etc.
All the data was displayed on custom Grafana dashboards, queries were fired from the dashboard which hit Prometheus, which was plugged-in to Grafana as a data source.
Container information was streamed into Prometheus from cAdvisor.
The below diagram shows the data flow between these open source tools, kind of a Grafana, Prometheus, cAdvisor based dashboard monitoring architectural flow. For more information on how Grafana was deployed read here.
2. How to Run cAdvisor With Docker?
If you intend to run the tool in a Docker container, there is an image already available which includes everything required to run it. A single cAdvisor can be run to monitor the whole machine.
It can also be run with Canary. Here are the details.
Also, the tool is a static Go binary with no external dependencies. We can run it standalone if we wish to. The runtime behaviour can be controlled via a series of flags.
cAdvisor also performs some housekeeping periodically. With these flags, we can control how & when the tool performs the task.
3. Running cAdvisor With Prometheus
cAdvisor is largely used with Prometheus in the industry. And Prometheus is largely used with Grafana. That is kinda the de facto combination for data analytics & monitoring.
cAdvisor exposes container statistics as Prometheus metrics intrinsically. Jobs are configured in Prometheus to connect with cAdvisor.
The tools web-based UI is pretty equipped to explore different kinds of stats with it Prometheus expression browser is used to get a more comprehensive view of things.
Here is a pretty nice writeup on Monitoring docker services with Prometheus & cAdvisor by CenturyLink.
For more technical information here you go.
4. Running cAdvisor With Kubernetes
In a Kubernetes cluster, the application can be examined at several different levels that are containers, pods, individual services or the entire cluster. If you are interested in knowing how pods, services, zones and clusters are related. I’ve written an article on Apache CloudStack an open source infrastructure as a service platform. I wrote about how on an infrastructural level resources are managed. Have a read.
Alright coming back to the topic.
Kubernetes uses a project known as Heapster, which acts as a base monitoring platform. It collects cluster-wide monitoring & event data. Heapster runs in Kubernetes pods & pulls data from Kubelets.
What is Kubelet?
A Kubelet kind of manages things on the cluster. It manages the pods & the containers on a machine. It is responsible for fetching individual container usage statistics from cAdvisor. The collected data is exposed via REST API
As stated the Kubelets, in turn, pull data from the cAdvisor. The entire data is saved in a data store such as InfluxDB & streamed to Grafana for visualization.
More on the Blog
Well, Guys!! This is pretty much it. I am sure by now we have an idea of what cAdvisor is? How does it work? What is the underlying architecture? & stuff.
I would love to know your thoughts on this. Do let me know in the comments.
If you liked the article. Do share it with your geek friends. I am doing this already…
I’ll see you in the next article.
- Distributed Systems, Scalability & System Design #1 – Heroku Client Rate Throttling
- Zero to Software/Application Architect – Learning Track
- Java Full Stack Developer – The Complete Roadmap – Part 2 – Let’s Talk
- Java Full Stack Developer – The Complete Roadmap – Part 1 – Let’s Talk
- Best Handpicked Resources To Learn Software Architecture, Distributed Systems & System Design