In the dynamic landscape of IT infrastructure, robust monitoring is indispensable for ensuring optimal performance and availability. This guide explores the integration of Prometheus, Grafana, Node Exporter, CAdvisor, and Skedler Reports to create a powerful monitoring solution for servers and Docker containers. We’ll delve into the core components, step-by-step setup, and additional considerations for advanced monitoring scenarios.

Monitoring Servers with Prometheus & Grafana: Building a Robust Infrastructure

Effective infrastructure monitoring forms the bedrock of application performance management. Continual optimization of system health and availability is imperative, necessitating the tracking of key metrics such as CPU, memory, network, and disk usage. This guide explains the process of monitoring servers and Docker containers using an ensemble of cutting-edge tools: 

• Prometheus for event monitoring and alerting.
• Grafana for database analytics and visualization.
• Node Exporter for capturing Linux host metrics.
• CAdvisor for monitoring container metrics.
• Skedler Reports for automating actionable reports.

Grafana – Database Analytics & Monitoring Solution

Grafana empowers users to query, visualize, and monitor metrics irrespective of the underlying data storage location. Beyond analytics, Grafana facilitates the creation of alerts for critical metrics, fostering a collaborative, data-driven culture within teams.

Prometheus – Event Monitoring and Alerting

Originally developed at SoundCloud, Prometheus has evolved into a robust open-source system monitoring and alerting toolkit. Widely adopted, it boasts an active developer and user community, ensuring continuous enhancements and versatility.

Node Exporter – Monitoring Linux Host Metrics

Node Exporter serves as a Prometheus exporter, capturing hardware and OS metrics with pluggable collectors. It enables the measurement of crucial machine resources such as memory, disk, and CPU utilization.

CAdvisor – Monitoring Metrics for Running Containers

Container Advisor aggregates and processes metrics for running containers, providing insights into their performance and resource utilization.

Skedler Reports – Automating Actionable Reports

Skedler Reports offers a potent, flexible, and user-friendly data monitoring solution. It aids companies in surpassing customer SLAs, achieving compliance, and empowering internal IT and business leaders.

Prerequisites

Before diving into the setup process, ensure you have:

1. A Linux machine
2. Docker installed
3. Docker Compose installed

Getting Started

Log in to your Linux machine and ensure the repository is up-to-date.

Install Docker and Docker Compose.

Create a Directory called monitoring 

ubuntu@guidanz:~$ mkdir monitoring

ubuntu@guidanz:~$ cd monitoring/

ubuntu@guidanz:~$ vim docker-compose.yml

Now, create a Docker Compose file for Prometheus, You also need to create a Prometheus configuration file, prometheus.yml Docker Compose file for Prometheus as below. 

Note: We will keep on extending the same docker file as we move forward to install other components.

version: ‘3’

services:

 prometheus:

   image: prom/prometheus:latest

   container_name: prometheus

   volumes:

     – ./prometheus.yml:/etc/prometheus/prometheus.yml

     – ./prometheus_db:/var/lib/prometheus

     – ./prometheus_db:/prometheus

     – ./prometheus_db:/etc/prometheus

     – ./alert.rules:/etc/prometheus/alert.rules

   command:

     – ‘–config.file=/etc/prometheus/prometheus.yml’

     – ‘–web.route-prefix=/’

     – ‘–storage.tsdb.retention.time=200h’

     – ‘–web.enable-lifecycle’

   restart: unless-stopped

   ports:

     – ‘9090:9090’

   networks:

     – monitor-net

Create a Prometheus configuration file (prometheus.yml) and paste the provided configuration.

global:

 scrape_interval: 5s

 external_labels:

   monitor: ‘guidanz-monitor’

scrape_configs:

 – job_name: ‘prometheus’

   static_configs:

     – targets: [‘monitoring.guidanz.com:9090’] ## IP Address of the localhost

The compose file consists of two volume mappings to the container. One is the Prometheus configuration and the other one (prometheus_db) is to store the Prometheus database locally. Now, you can run the docker-compose.

ubuntu@guidanz:~/monitoring$ mkdir prometheus_db

ubuntu@guidanz:~/monitoring$ docker-compose up -d

Access Prometheus UI using the provided IP and Port: http://ip_address:9090

Now we will configure the Node Exporter. It is one of the best components used together with Prometheus to capture metrics from the server where Prometheus is running. Node Exporter captures all hardware and kernel-related metrics such as CPU, Memory, Disk, Disk Read/Write, among others.

To install the Node exporter, simply append the docker-compose.yml file and prometheus.yml file as below.

node-exporter:

 image: prom/node-exporter

 ports:

   – ‘9100:9100’

Append the prometheus.yml as follows:

– job_name: ‘node-exporter’

 static_configs:

   – targets: [‘monitoring.guidanz.com:9100’]

The Composite docker-compose file will look like below:

version: ‘3’

services:

 prometheus:

   image: prom/prometheus:latest

   volumes:

     – ./prometheus.yml:/etc/prometheus/prometheus.yml

     – ./prometheus_db:/var/lib/prometheus

   command:

     – ‘–config.file=/etc/prometheus/prometheus.yml’

   ports:

     – ‘9090:9090’

 node-exporter:

 image: prom/node-exporter

 ports:

   – ‘9100:9100’

And prometheus.yml will look like this:

global:

 scrape_interval: 5s

 external_labels:

   monitor: ‘guidanz-monitor’

scrape_configs:

 – job_name: ‘prometheus’

   static_configs:

     – targets: [‘monitoring.guidanz.com:9090’]

 – job_name: ‘node-exporter’

   static_configs:

     – targets: [‘monitoring.guidanz.com:9100’]

Execute the following commands to start Node Exporter and restart Prometheus.

ubuntu@guidanz:~/monitoring$ docker-compose start node-exporter

ubuntu@guidanz:~/monitoring$ docker-compose restart prometheus

Or you can also just compose up and down: 

ubuntu@guidanz:~/monitoring$ docker-compose down

ubuntu@guidanz:~/monitoring$ docker-compose up -d

Take a look at the targets in Prometheus. You will notice node exporter as well as a target.

Append the Docker Compose file with CAdvisor configuration.

cadvisor:

 image: google/cadvisor:latest

 ports:

   – ‘8080:8080’

 volumes:

   – /:/rootfs:ro

   – /var/run:/var/run:rw

   – /sys:/sys:ro

   – /var/lib/docker/:/var/lib/docker:ro

Also, append the prometheus.yml with a bit code yml code. We are actually adding the CAdvisor service in Prometheus configuration.

– job_name: ‘cAdvisor’

 static_configs:

   – targets: [‘monitoring.guidanz.com:8080’]

Access CAdvisor from the URL: http://IP_Address:8080/docker/

Now we will set up Grafana, where we will be using Prometheus as a data source. We can have a better Dashboard in Grafana for the metrics visualization.

Append the code in the above docker compose and restart.

grafana:

 image: grafana/grafana

 user: “1000”

 environment:

   – GF_SECURITY_ADMIN_PASSWORD=secure_pass

 volumes:

   – ./grafana_db:/var/lib/grafana

 depends_on:

   – prometheus

 ports:

   – ‘3000:3000’

Access Grafana UI from 3000 port, default user will be admin and the password you set in the compose file.

Now we will set up the Skedler Reports, where we will be using Grafana as a data source. Skedler offers a simple and easy-to-add reporting and alerting solution for Kibana and Grafana. Please review the documentation to install Skedler Reports.

Now, you can set up Skedler Reports, for this append the Docker compose with the below code.

reports:

   image: skedler/reports:latest

   container_name: reports

   privileged: true

   cap_add:

     – SYS_ADMIN

   volumes:

     – /sys/fs/cgroup:/sys/fs/cgroup:ro

     – reportdata:/var/lib/skedler

     – ./reporting.yml:/opt/skedler/config/reporting.yml

   ports:

     – 3001:3001

This comprehensive guide aims to equip you with the knowledge and tools needed to establish a robust infrastructure monitoring system. Whether you are monitoring Linux servers, Docker containers, or exploring advanced configurations, the integration of Prometheus, Grafana, and associated components provides a versatile and powerful solution for your monitoring needs. If you found this article helpful in establishing a robust monitoring system with Prometheus, Grafana, and Docker, we recommend exploring related articles to enhance your understanding and skills further. Dive into topics like Automating Reports for Business Success for insights into optimizing reporting processes, or delve into the comparative analysis of Prometheus vs Grafana to make informed decisions about your monitoring stack. We hope you found this article about monitoring servers with Prometheus interesting. Expanding your knowledge base will empower you to navigate the complexities of infrastructure monitoring effectively. Happy learning!