Installing, configuring Skedler Reports as Kibana Plugin with Elasticsearch and Kibana Environment using Docker Compose

Introduction

If you are using ELK stack, you can now install Skedler as a Kibana plugin. Skedler Reports plugin is available for Kibana versions from 6.5.x to 7.6.x.

Let’s take a look at the steps to Install Skedler Reports as a Kibana plugin.

Prerequisites:

  1. A Linux machine
  2. Docker Installed
  3. Docker Compose Installed

Let’s get started!

Login to your Linux machine and update the repository and install Docker and Docker Compose. Then follow the below steps to update the Repository:

Setting Up Skedler Reports

Create a Directory, say skedlerplugin

Now, create a Docker Compose file for Skedler Reports. You also need to create a Skedler Reports configuration file, reporting.yml, and a Docker Compose file for Skedler as below,

Create an Elasticsearch configuration file – reporting.yml and paste the config as below.

Download the reporting.yml file found here

Setting Up Elasticsearch

You also need to create an Elasticsearch configuration file, elasticsearch.yml. Docker Compose file for Elasticsearch is below,

Create an Elasticsearch configuration file elasticsearch.yml and paste the config as below.

Setting Up Skedler Reports as Kibana Plugin

Create a Directory inside skedlerplugin, say kibanaconfig

Now, create a Docker file for Kibana and check the Docker file for Kibana as below,

Then, copy the URL of the Skedler Reports plugin matching your exact Kibana version from here.

You also need to create a Docker Compose file for Kibana is below,

Create a Kibana configuration file kibana.yml inside the kibanaconfig folder and paste the config as below.

Create a Skedler Reports as Kibana Plugin configuration file skedler_reports.yml inside the kibanaconfig folder and paste the config as below.

Configure the Skedler Reports server URL in the skedler_reports_url variable. By default, the variable is set as shown below,

If the Skedler Reports server URL requires basic authentication, for example, Nginx, uncomment and configure the skedler_username and skedler_password with the basic authentication credentials as shown below: Now run the docker-compose.

Access Skedler Reports the IP and Port and you will see the Skedler Reports UI.

| http://ip_address:3000

Access Elasticsearch the IP and Port and you will see the Elasticsearch UI.

| http://ip_address:9200

Access Kibana using the IP and Port and you will see the Kibana UI.

| http://ip_address:5601

So now the Composite docker-compose file will look like below,

You can Simply do compose up and down.

Summary

Docker compose is a useful tool to manage container stacks for your client. And manage all related containers with one single command.

Monitoring Servers and Docker Containers using Elasticsearch with Grafana

Introduction

Infrastructure monitoring is the basis for application performance management. The underlying system’s availability and health must be maximized continually. To achieve this, one has to monitor the system metrics like CPU, memory, network, and disk. Response time lag, if any must be addressed swiftly. Here we’ll take a look at how to Monitor servers (and even Docker Containers running inside the Server) using Grafana, Elasticsearch, Metricbeat, and Skedler Reports.

Core Components

Grafana-Analytics & monitoring solution for database

Elasticsearch-Ingest and index logs

Metricbeat-Lightweight shipper for metrics

Skedler Reports –Automate actionable reports

Grafana — Analytics & monitoring solution for database

Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored. Create, explore, and share dashboards with your team and foster a data-driven culture.

Elasticsearch-Ingest and index logs

Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected.

Metricbeat — Lightweight shipper for metrics

Collect metrics from your systems and services. From CPU to memory, Redis to NGINX, and much more, Metricbeat is a lightweight way to send system and service statistics.

Skedler Reports — Automate actionable reports

Skedler offers the most powerful, flexible and easy-to-use data monitoring solution that companies use to exceed customer SLAs, achieve compliance, and empower internal IT and business leaders.

Export your Grafana Dashboard to PDF Report in Minutes with Skedler. Fully featured free trial.

Prerequisite,

  1. A Linux machine
  2. Docker Installed
  3. Docker Compose Installed

Now, Create a Docker Compose file for Elasticsearch, You also need to Create an Elasticsearch configuration file, elasticsearch.yml Docker Compose file for Elasticsearch is below,

Note: We will keep on extending the same docker file as we will move ahead to install other components.

Create an Elasticsearch configuration file elasticsearch.yml and paste the config as below.

Now run the docker-compose.

Access Elasticsearchusing the IP and Port and you will see the Elasticsearch UI.

http://ip_address:9200

Now We will setup the Metricbeat. It is one of the best components used along with the Elasticsearch to capture metrics from the server where the Elasticsearch is running. It Captures all hardware and kernel-related metrics like system-level CPU usage, memory, file system, disk IO, and network IO statistics, as well as top-like statistics for every process running on your systems.

To Install the Metricbeat, simply append the docker-compose.yml file, metricbeat.yml, and modules.d file as below.

Append the metricbeat.yml as below,

The compose file consists of the volume mapping to the container, one is the metricbeat configuration and the second one (modules.d) is to Mount the modules.d directory into the container. This allows users to potentially make changes to the modules and they will be dynamically loaded. Now run the docker-compose.

Append the system.yml as below inside the module.d folder,

So now the Composite docker-compose file will look like below,

You can Simply do compose up and down.

Now See the Targets in Elasticsearch, you will see metricbeat as well as a target.

Now eventually we will set up the grafana, where we will be using Elasticsearch as a data source. We can have a better Dashboard in grafana for the metrics visualization.

Append the code in the above docker compose and restart.

Access grafana UI from 3000 port, default user will be admin and the password you set in the compose file.

Now eventually we will set up the Skedler Reports, where we will be using Grafana as a data source. Skedler offers a simple and easy to add reporting and alerting solution for Elastic Stack and Grafana. Please review the documentation to install Skedler.

Now, Setup Skedler Reports, for this append the docker compose with the below code.

Generate Report from Grafana in Minutes with Skedler. Fully featured free trial.

Monitoring Servers and Docker Containers using Prometheus with Grafana

Introduction

Infrastructure monitoring is the basis for application performance management. The underlying system’s availability and health must be maximised continually. To achieve this, one has to monitor the system metrics like CPU, memory, network, and disk. Response time lag, if any must be addressed swiftly. Here we’lltake a look at how to Monitor servers (and even Docker Containers running inside the Server) using Grafana, Prometheus, Node Exporter, CAdvisor and Skedler Reports.

Core Components

Grafana- Database analytics & monitoring solution

Prometheus- Event monitoring and alerting

Node-Exporter- Monitoring Linux host metrics

Wmi-Exporter- Monitoring Windows host metrics

CAdvisor- Monitoring metrics for the running Containers.

Skedler Reports –Automating actionable reports

Grafana - Database Analytics & monitoring solution 

Grafana equips users to query, visualize,  and monitor metrics, no matter where the underlying data is stored. With Grafana, one can also set alerts for metrics that require attention, apart from creating, exploring, and sharing dashboards with their team and fostering a data-driven culture.

Prometheus -Event monitoring and alerting

Prometheus is an open-source system monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open source project and maintained independently of any company.

Node Exporter - Monitoring Linux host metrics

Node Exporter is a Prometheus exporter for hardware and OS metrics with pluggable metric collectors. It allows measuring various machine resources such as memory, disk, and CPU utilization

WMI Exporter -Monitoring Windows host metrics

Prometheus exporter for Windows machines, using the WMI (Windows Management Instrumentation).

CAdvisor - Monitoring metrics for the running Containers.

It Stands for Container Advisor and is used to aggregate and process all the metrics for the running Containers.

Skedler Reports — Automating actionable reports

Skedler offers the most powerful, flexible and easy-to-use data monitoring solution that companies use to exceed customer SLAs, achieve compliance, and empower internal IT and business leaders.

Export your Grafana Dashboard to PDF Report in Minutes with Skedler. Fully featured free trial.

Prerequisite,

  1. A Linux machine
  2. Docker Installed
  3. Docker Compose Installed

So Let’s get started.

Login to your Linux machine and update the repository and Install Docker and Docker Compose. To Update the Repository,

Create a Directory, say monitoring

Now, create a Docker Compose file for Prometheus, You also need to create a Prometheus configuration file, prometheus.yml Docker Compose file for Prometheus as below,

Note: We will keep on extending the same docker file as we move forward to install other components.

Create a Prometheus configuration file and paste the config as below.

The compose file consists of two volume mappings to the container. One is the Prometheus configuration and the other one (prometheus_db) is to store the Prometheus database locally. Now run the docker-compose.

Access Prometheus using the IP and Port and you will see the Prometheus UI, http://ip_address:9090

Now we will setup the Node Exporter. It is one of the best components used along with the Prometheus to capture metrics from the server where the Prometheus is running. It Captures all hardware and kernel-related metrics like CPU, Memory, Disk, Disk Read/Write, etc.

To Install the Node exporter, simply append the docker-compose.ymlfile and prometheous.yml file as below.

Append the prometheus.yml as below,

So now the Composite docker-compose file will looks like below,

And prometheus.yml will look like below,

Now Create the Node Exporter Container and Restart the Prometheus container using the below commands

Or, One can simply do compose up and down.

Now take a look at the targets in Prometheus. You will notice node exporter as well as a target.

Now, setup CAdvisor, for this append the docker compose with below code.

Also, append the prometheus.yml with a bit code yml code. We are actually adding the CAdvisor service in Prometheus configuration.

Access CAdvisor from the URL, http://IP_Address:8080/docker/

Now eventually we will set up the grafana, where we will be using Prometheus as a data source. We can have a better Dashboard in grafana for the metrics visualization.

Append the code in the above docker compose and restart.

Access grafana UI from 3000 port, default user will be admin and the password you set in the compose file.

Now eventually we will set up the Skedler Reports, where we will be using Grafana as a data source. Skedler offers a simple and easy to add reporting and alerting solution for Elastic Stack and Grafana. Please review the documentation to install Skedler Reports.

Now, Setup Skedler Reports, for this append the docker compose with the below code.

Schedule and Automate Your Grafana Reports Free with Skedler. Fully featured free trial.

Simplifying Skedler Reports with Elasticsearch and Kibana Environment using Docker Compose

Docker compose is a tool for defining and running multi-container (Skedler Reports, Elasticsearch and Kibana) Docker applications.  With Compose, you use a YAML file to configure your application’s services. Then with a single command, you create and start all the services from your configuration.

In this section, I will describe how to create containerized installation for Skedler Reports, Elasticsearch and Kibana.

Benefits:

  • You describe the multi-container setup in a clear way and bring up the containers in a single command.
  • You can define the priority and dependency of the container to other containers.

Step-by-Step Instruction:

Step 1: Define services in a Compose file:

Create a file called docker-compose.yml in your project directory and paste the following

docker-compose.yml:

This Compose file defines three services, Skedler Reports, Elasticsearch and Kibana.

Step 2: Basic configurations using reporting.yml and kibana.yml

Create a files called reporting.yml in your project directory.

Getting the reporting.yml file found here

Note: For more configuration options kindly refer the article reporting.yml and ReportEngineOptions Configuration

Create a files called kibana.yml in your project directory.

Note: For more configuration options kindly refer the article kibana.yml

Step 3: Build and run your app with docker-compose

From your project directory, start up your application by running

Compose pulls a Skedler Reports, Elasticsearch and Kibana images, builds an image for your code, and starts the services you defined

Skedler Reports is available at http://<hostIP>:3000,  Elasticsearch is available at http://<hostIP>:9200 and Kibana is available at http://<hostIP>:5601 .

Summary

Docker compose is a useful tool to manage container stacks for your client. And manage all related containers with one single command.

Alerting for Elasticsearch – How to install with docker

Skedler Alerts provides an easy to use alerting solution for Elasticsearch data.  It is designed for users who find:

  • yaml based rules are difficult to manage and time consuming
  • alternative pack based plugins are cost prohibitive

Powerful, Yet Easy to Use Alerting for Elasticsearch

Skedler Alerts simplifies how you create and manage alert rules for Elasticsearch.  In addition, it provides a flexible approach to notification.  You can send notifications via email or slack.  You can also integrate alert notifications to your application using webhooks.  Last but not the least, with just a few button clicks you can drill down from notifications to root cause documents.

Easy drilldown from notification to root cause event

Alerts Installation

Installation of Skedler Alerts is equally simple.  Of course, the easiest way to is install Skedler Alerts with Docker.  Please note that Skedler Alerts is available for Linux 7 only.

[video_embed video=”EuNKgZPki8c” parameters=”” mp4=”” ogv=”” placeholder=”” width=”700″ height=”400″]

If you would like to install Skedler on a VM, reach out to us via the free trial page.  You will hear back from us within 24 hours to help you get started.

Questions/Comments?

Reach out to us via Skedler Forum  with your questions and comments.   Happy, Easy Alerting!

Translate »