What’s New in Skedler

The release of Skedler in November came with many improvements, such as auto-scaling support for Grafana dashboard layout reports and an updated user interface. In the December release, we came up with more features like Autoscaling support for charts in Kibana and the option to configure proxy URL. We are very proud of these releases, but the team is always looking forward to new ways of making Skedler better for you. We are already improving our product further and wanted you to know about our newly added features and UI.So, before we end the year, we want to update you on the features we released and go through some of the important ones in this blog.

Halt your reporting schedules for Specific Days

Want to make sure you are not sending your reports on a holiday? We got you covered! You can now choose the days you do not wish to schedule reports with our new Weekday feature.

Weekday feature

Autoscaling support for charts in Kibana

Skedler now supports autoscaling of charts in Kibana. You do not have to worry about your reports being messy or missing out on important information when you add more data to your chart because Skedler will automatically take care of that.

Autoscaling in Kibana

Added an auto-scaling support for Grafana dashboard layout reports 

You can now stop worrying about your graphs and modules getting distorted in your reports as Skedler has added auto-scaling support for generating reports from Grafana Dashboard.

Autoscaling in Grafana

 Added a privilege to super admin users to change their email id

Super Admins can now update their email ID in their profile. You can add a new Mail ID instead of the one you used when you opened your account.

Super Admin User

 Generate reports using Grafana dashboard timezone

You can now generate reports in Skedler as per your Grafana time window by selecting “use dashboard time” in Skedler. You do not have to worry about missing or skipping any reports.

Dashboard Timezone

Support for fiscal year time window in Grafana dashboards. 

Grafana 8.2  has the option of the configurable fiscal year in the time picker. This option enables fiscal quarters as time ranges for business-focused and executive dashboards. Skedler now supports this feature too!

Fiscal Time Year Window

Added support for Outlook SMTP

Skedler now supports Outlook. So you can set up Outlook as your notification channel in your Skedler account.

Outlook SMTP

These are just some of the new features of Skedler. For more details on these features, do check out our release notes.

If you would like to stay updated on the latest release news or know about upcoming features, please feel free to reach out to the team and keep an eye out for our monthly newsletters.

Kibana Single Sign-On with OpenId Connect and Azure Active Directory

Introduction

Open distro supports OpenID so you can seamlessly connect your Elasticsearch cluster with Identity Providers like Azure AD, Keycloak, Auth0, or Okta. To set up OpenID support, you just need to point Open distro to the metadata endpoint of your provider, and all relevant configuration information is imported automatically. In this article, we will implement a complete OpenID Connect setup including Open distro for Kibana Single Sign-On.

What is OpenID Connect?

OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol. It allows Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.

OpenID Connect allows clients of all types, including Web-based, mobile, and JavaScript clients, to request and receive information about authenticated sessions and end-users. The specification suite is extensible, allowing participants to use optional features such as encryption of identity data, the discovery of OpenID Providers, and session management, when it makes sense for them.

Configuring OpenID Connect in Azure AD

Next, we will set up an OpenID Connect client application in Azure AD which we will later use for Open Distro for Elasticsearch Kibana Single Sign-On. In this post, we will just describe the basic steps.

Adding an OpenID Connect client application

Our first step is, we need to register an application with the Microsoft identity platform that supports OpenID Connect. Please refer to the official documentation.

Login to azure ad and open the Authentication tab in-app registrations and enter the redirect URL as https://localhost:5601/auth/openid/login and save it.

Besides the client ID, we also need the client secret in our Open Distro for elasticsearch Kibana configuration. This is an extra layer of security. An application can only obtain an id token from the IdP if it provides the client secret. In Azure AD you can find it under the Certificates & secrets tab of the client settings.

Connecting OpenDistro with Azure AD

For connecting Open Distro with Azure AD we need to set up a new authentication domain with type openid in config.yml. The most important information we need to provide is the Metadata Endpoint of the newly created OpenID connect client. This endpoint provides all configuration settings that Open Distro needs. The URL of this endpoint varies from IdP to IdP. In Azure AD the format is:

Since we want to connect Open Distro for Elasticsearch Kibana with Azure AD, we also add a second authentication domain which will use the internal user database. This is required for authenticating the internal Kibana server user. Our config.yml file now looks like:

Adding users and roles to Azure AD

While an IDP can be used as a federation service to pull in user information from different sources such as LDAP, in this example we use the built-in user management. We have two choices when mapping the Azure AD users to Open Distro roles. We can do it by username, or by the roles in Azure AD. While mapping users by name is a bit easier to set up, we will use the Azure AD roles here.

With the default configuration, two appRoles are created, skedler_role and guidanz_role, which can be viewed by choosing the App registrations menu item within the Azure Active Directory blade, selecting the Enterprise application in question, and clicking the Manifest button

A manifest is a JSON object that looks similar to:

There are many different ways we might decide to map how users within AAD will be assigned roles within Elasticsearch, for example, using the tenantid claim to map users in different directories to different roles, using the domain part of the name claim, etc.

With the role OpenID connect token attribute created earlier, however, the appRole to which an AAD user is assigned will be sent as the value of the Role Claim within the OpenID connect token, allowing:

  • Arbitrary appRoles to be defined within the manifest
  • Assigning users within the Enterprise application to these roles
  • Using the Role Claim sent within the SAML token to determine access within Elasticsearch.

For the purposes of this post, let’s define a Superuser role within the appRoles:

And save the changes to the manifest:

Configuring OpenID Connect in Open Distro for Kibana

The last part is to configure OpenID Connect in Open Distro for Kibana. Configuring the Kibana plugin is straight-forward: Choose OpenID as the authentication type, and provide the Azure AD metadata URL, the client name, and the client secret. Please refer to the official documentation.

Activate OpenID Connect by adding the following to kibana.yml:

Done. We can now start Open Distro for Kibana and enjoy Single Sign-On with Azure AD! If we open Kibana, we get redirected to the login page of Azure AD. After providing username and password, Kibana opens, and we’re logged in.

Summary

OpenID Connect is an industry-standard for providing authentication information. Open Distro for Elasticsearch and their Open Distro for Kibana plugin support OpenID Connect out of the box, so you can use any OpenID compliant identity provider to implement Single Sign-On in Kibana. These IdPs include Azure AD, Keycloak, Okta, Auth0, Connect2ID, or Salesforce.

Reference

If you wish to have an automated reporting application, we recommend downloading  Skedler Reports.

Installing, configuring Skedler Reports as Kibana Plugin with Elasticsearch and Kibana Environment using Docker Compose

Introduction

If you are using ELK stack, you can now install Skedler as a Kibana plugin. Skedler Reports plugin is available for Kibana versions from 6.5.x to 7.6.x.

Let’s take a look at the steps to Install Skedler Reports as a Kibana plugin.

Prerequisites:

  1. A Linux machine
  2. Docker Installed
  3. Docker Compose Installed

Let’s get started!

Login to your Linux machine and update the repository and install Docker and Docker Compose. Then follow the below steps to update the Repository:

Setting Up Skedler Reports

Create a Directory, say skedlerplugin

Now, create a Docker Compose file for Skedler Reports. You also need to create a Skedler Reports configuration file, reporting.yml, and a Docker Compose file for Skedler as below,

Create an Elasticsearch configuration file – reporting.yml and paste the config as below.

Download the reporting.yml file found here

Setting Up Elasticsearch

You also need to create an Elasticsearch configuration file, elasticsearch.yml. Docker Compose file for Elasticsearch is below,

Create an Elasticsearch configuration file elasticsearch.yml and paste the config as below.

Setting Up Skedler Reports as Kibana Plugin

Create a Directory inside skedlerplugin, say kibanaconfig

Now, create a Docker file for Kibana and check the Docker file for Kibana as below,

Then, copy the URL of the Skedler Reports plugin matching your exact Kibana version from here.

You also need to create a Docker Compose file for Kibana is below,

Create a Kibana configuration file kibana.yml inside the kibanaconfig folder and paste the config as below.

Create a Skedler Reports as Kibana Plugin configuration file skedler_reports.yml inside the kibanaconfig folder and paste the config as below.

Configure the Skedler Reports server URL in the skedler_reports_url variable. By default, the variable is set as shown below,

If the Skedler Reports server URL requires basic authentication, for example, Nginx, uncomment and configure the skedler_username and skedler_password with the basic authentication credentials as shown below: Now run the docker-compose.

Access Skedler Reports the IP and Port and you will see the Skedler Reports UI.

| http://ip_address:3000

Access Elasticsearch the IP and Port and you will see the Elasticsearch UI.

| http://ip_address:9200

Access Kibana using the IP and Port and you will see the Kibana UI.

| http://ip_address:5601

So now the Composite docker-compose file will look like below,

You can Simply do compose up and down.

Summary

Docker compose is a useful tool to manage container stacks for your client. And manage all related containers with one single command.

The Best Tools for Exporting Elasticsearch Data from Kibana

As a tool for visualizing elasticsearch data, Kibana is a perfect choice. Its UI interface allows creating a dashboard, search, and visualizations in minutes and analyzing the data with its help.

Despite having tons of visualizations, the open source version of Kibana does not have advanced reporting capability. Automating export of data into CSV, Excel, or PDF requires additional plugins.  

We wrote an honest and unbiased review of the following tools that are available for exporting data directly from Elasticsearch.

  1. Flexmonster Pivot plugin for Kibana 
  2. Sentinl (for Kibana)
  3. Skedler Reports

1. Flexmonster Pivot plugin for Kibana

https://github.com/flexmonster/pivot-kibana

Flexmonster Pivot covers the need in summarizing business data and displaying results in a cross-table format interactively & fast. All these Excel-like features, to which so many of you are used to, and its extended API will multiply your analytics results remarkably.

Though initially created as a pivot table component that can be incorporated into any app that uses JavaScript, it can serve as a part of Kibana as well. You can connect it to the Elasticsearch index, fetch the documents from it and start exploring the data.

Pros of Flexmonster Pivot plugin for Kibana

  • Flexmonster is in line with the concept of Kibana
  • Simply embeddable Pivot for Kibana

Cons of Flexmonster Pivot plugin for Kibana

  • To automate the exporting of data on a periodic basis, you need to write your own cron job.
  • Flexmonster Pivot plugin installation is a bit tricky. 

2. Sentinl (for Kibana)

https://github.com/sirensolutions/sentinl

SENTINL extends Kibana with Alerting and Reporting functionality to monitor, notify and report on data series changes using standard queries, programmable validators and a variety of configurable actions – Think of it as a free and independent “Watcher” which also has scheduled “Reporting”.

SENTINL is also designed to simplify the process of creating and managing alerts and reports in Siren Investigate/Kibana 6.x via its native App Interface, or by using native watcher tools in Kibana 6.x+.

Pros of Sentinl

  • It’s simple to install and configure
  • Added as a Kibana plugin.

Cons of Sentinl

  • This tool supports only 6x versions of Elasticsearch.  It does not support 7.x.
  • For non-technical users, it’s difficult to use 
  • Automation requires scripting which makes it laborious

3. Skedler Reports

https://www.skedler.com/

Disclosure: Skedler Reports is one of our products.

Skedler offers a simple and easy to add reporting and alerting solution for Elastic Stack and Grafana.  There is also a plugin for Kibana that is easy to install and use with the Elasticsearch data. It’s called Skedler Reports as Kibana Plugin. 

Pros of Skedler Reports

  • Simple to install, configure, and use
  • Send HTML, PDF, XLS, CSV reports on-demand or periodically via email or #slack
  • Report setup takes less than 5 minute
  • Easy to use, no coding required

Cons of Skedler Reports

  • It requires a paid license which includes software and also enterprise support
  • Installation is difficult for users who are not fully familiar with Elastic Stack or Grafana

What tools do you use?

Do you have to regularly export data from Kibana for external analysis or reporting purposes? Do you use any other third-party plugins?   Email us about the tool at hello at skedler.com.

Tabular Reports from Elastic Stack – New in Skedler Reports v4.4

We are excited to announce the release of Skedler Reports v4.4. As always, it’s packed with capabilities to help you meet compliance, audit, and snapshot reporting requirements.

Tabular PDF, Excel, CSV Reports from Kibana Data Table

If you are a security analyst or network admin looking for the list of unauthorized IP addresses connecting to your machines, Skedler can deliver the data to you in the form of PDF or Excel. With just a couple of clicks, schedule a PDF and/or Excel report that uses the Kibana data table as a source, sit back and have the reports delivered to your stakeholders automatically!

[video_embed video=”l-4JSKe9ee4″ parameters=”” mp4=”” ogv=”” placeholder=”” width=”700″ height=”400″]

Schedule Reports with Custom Time Ranges

If your customer needs a daily report that summarizes the top security events during the work hours of 9 AM – 5 PM, you can send it to them right away. Simply create a custom time range in Kibana and customize your dashboard to use this time range.  In Skedler, schedule a daily report with the dashboard as a data source and you’re all set!

Here is the list of additional features in the new release:

  • You can use the latest features in Elastic Stack 7.3 and Grafana 6.3 and generate reports with Skedler.
  • Users do not need administrator privileges to configure Grafana as a data source in Skedler.

Go Ahead and Try it Out

Test out the data table reports with custom time ranges in ELK 7.3 or Grafana 6.3 environment! Start now below by doing the following:

  1. Download Skedler Reports
  2. Follow the simple steps in our documentation and start generating reports.

An Easy Way to Export / Import Dashboards, Searches and Visualizations from Kibana

Introduction

Manually recreating Kibana dashboards, searches, and visualizations during upgrades, production deployment or recovery is a time-consuming affair. The easiest way to recreate the prebuilt Kibana dashboard and other objects is by exporting and importing dashboards, searches, and visualizations. This can be achieved by using,

  • Kibana API (available since Kibana 7.x) 
  • Kibana UI

If are you looking to export and import the Kibana dashboards and its dependencies automatically, we recommend the Kibana API’s. Also, you can export and import dashboard from Kibana UI.

Note: User should add the dependencies of the dashboards like visualization, index pattern individually while exporting or importing from Kibana UI.

Export Objects From Kibana API

The export API enables you to retrieve a set of saved objects that can later be imported into Kibana.

Request

Request Body

At least type or objects must be passed in within the request body.

type (optional)

(array/string) The saved object type(s) that the export should be limited to.

The following example exports all index pattern saved objects.

Example Curl:

objects (optional)

(array) A list of objects to export

The following example exports specific saved objects.

Example Curl:

Response Body

The response body will have a format of newline delimited JSON and the successful call returns a response code of 200 along with the exported objects as the response body.

Import Objects From Kibana API

The import API enables you to create a set of Kibana saved objects from a file created by the export API.

Request

Request Body

The request body must be of type multipart/form-data.

File

A file exported using the export API.

Example

The following example imports an index pattern and dashboard.

The file.ndjson file would contain the following.

Response Body

A successful call returns a response code of 200 and a response body containing a JSON structure similar to the following example:

Export Objects From Kibana UI:

You can now export your objects from Kibana UI under Management > Saved Objects > Export. Select the checkboxes of the objects you want to export, and click Export. Or to export objects by type:

  • Click Export objects.
  • Select the object types you want to export.
  • Click Export All.

kibana export

Import Objects From Kibana UI:

 You can import your JSON file from Kibana UI under Management > Saved Objects > Import. Follow the below steps to import your 

  • Click Import.
  • Navigate to the JSON file that represents the objects to import.
  • Indicate whether to overwrite objects already in Kibana.
  • Click Import.

kibana

Summary:

Exporting and importing the saved objects from the Kibana is an effective and easiest way to recreate dashboards and other objects in new environments or during migrations.

If you are looking to automate and make the process simpler,  we recommend using the Kibana APIs or else you can use the Kibana UI for granular export and import.

If you are looking for a Kibana reporting solution, be sure to test drive Skedler.

Skedler v4.1: Next Generation Reporting for Elasticsearch Kibana 7.0 and Grafana 6.1 is here

We are excited to announce that we have just released version 4.1 of Skedler Reports!  

[button title=”Download Skedler 4.1 Now” icon=”” icon_position=”” link=”https://www.skedler.com/download/” target=”_blank” color=”#800080″ font_color=”#000″ large=”0″ class=”v4download” download=”” onclick=””]

Self Service Reporting Solution for Elasticsearch Kibana 7.0 and Grafana 6.1

We understand that your stakeholders and customers need intuitive and flexible options to save time in receiving the data that matters to them and we’ve achieved exactly that with the release of Skedler 4.1.  The newly enhanced UI offers a delightful user experience for creating and scheduling reports from your Elasticsearch Kibana 7.0 and Grafana 6.1 .

[video_embed video=”4flSLj5q1yk” parameters=”” mp4=”” ogv=”” placeholder=”” width=”700″ height=”400″]

Multi-Tenancy Capabilities

If you are a service provider, you need a simple and automated way to provide different groups of users (i.e. “tenants”) with access to different sets of data. Skedler 4.1’s powerful and secure multi-tenancy capabilities will now allow you to send reports to your customers from your multi-tenant analytics application within minutes.  Supported with Search Guard, Open Distro & X-Pack.

Intuitive and Mobile Ready Reports

Skedler 4.1 will now allow you to produce high-resolution HTML reports from Elasticsearch Kibana and Grafana that will make it easy and convenient for your end users to access to critical data through their mobile devices and email clients. No more cumbersome and large PDF attachments.

[video_embed video=”soFITSdyDdE” parameters=”” mp4=”” ogv=”” placeholder=”” width=”700″ height=”400″]

The latest release also includes:

  • Support for the latest and greatest version of Elastic Stack and Grafana. Skedler 4.1 supports the following versions:
    • Elastic stack 6.7 and 7.0
    • Grafana 6.1.x
    • Open distro for Elasticsearch 6.7 and 7.0.  

Please continue to send us feedback for what new capabilities you’d like to see in the future by reaching out to us at [email protected]

Simplifying Skedler Reports with Elasticsearch and Kibana Environment using Docker Compose

Docker compose is a tool for defining and running multi-container (Skedler Reports, Elasticsearch and Kibana) Docker applications.  With Compose, you use a YAML file to configure your application’s services. Then with a single command, you create and start all the services from your configuration.

In this section, I will describe how to create containerized installation for Skedler Reports, Elasticsearch and Kibana.

Benefits:

  • You describe the multi-container setup in a clear way and bring up the containers in a single command.
  • You can define the priority and dependency of the container to other containers.

Step-by-Step Instruction:

Step 1: Define services in a Compose file:

Create a file called docker-compose.yml in your project directory and paste the following

docker-compose.yml:

This Compose file defines three services, Skedler Reports, Elasticsearch and Kibana.

Step 2: Basic configurations using reporting.yml and kibana.yml

Create a files called reporting.yml in your project directory.

Getting the reporting.yml file found here

Note: For more configuration options kindly refer the article reporting.yml and ReportEngineOptions Configuration

Create a files called kibana.yml in your project directory.

Note: For more configuration options kindly refer the article kibana.yml

Step 3: Build and run your app with docker-compose

From your project directory, start up your application by running

Compose pulls a Skedler Reports, Elasticsearch and Kibana images, builds an image for your code, and starts the services you defined

Skedler Reports is available at http://<hostIP>:3000,  Elasticsearch is available at http://<hostIP>:9200 and Kibana is available at http://<hostIP>:5601 .

Summary

Docker compose is a useful tool to manage container stacks for your client. And manage all related containers with one single command.

Skedler Update: Version 3.9 Released

Skedler Update: Version 3.9 Released

Here’s everything you need to know about the new Skedler v3.9. Download the update now to take advantage of its new features for both Skedler Reports and Alerts.

What’s New With Skedler Reports v3.9

  • Support for:
    • ReadOnlyRest Elasticsearch/Kibana Security Plugin.
    • Chromium web browser for Skedler report generation.
    • Report bursting in Grafana reports if the Grafana dashboard is set with Template Variables.
    • Elasticsearch version 6.4.0 and Kibana version 6.4.0.
  • Ability to install Skedler Reports through Debian and RPM packages.
  • Simplified installation levels of Skedler Reports here.
  • Upgraded license module
    • NOTE: License reactivation is required when you upgrade Skedler Reports from the older version to the latest v3.8. Refer to this URL to reactivate the Skedler Reports license key.
    • Deactivation of Skedler license key in UI

What’s New With Skedler Alerts v3.9

  • Support for:
    • Installing Skedler Alerts via Debian and RPM packages.
    • GET method type in Webhook.
    • Elasticsearch 6.4.0.
  • Simplified installation levels of Skedler. Refer to this URL for installation guides.
  • Upgraded license module:
    • NOTE: License reactivation is required when you upgrade Skedler Alerts from the older version to the latest v3.8. Refer to this URL to reactivate the Skedler Alerts license key.
  • Deactivation of Skedler Alerts license key in UI

 

Get Skedler Reports

Download Skedler Reports

Get Skedler Alerts

Download Skedler Alerts

 

A Comparison of Reporting Tools for Elastic Stack – Elastic Reporting and Skedler Reports

Elasticsearch is stronger with every new release while the Kibana visualizations are getting more sophisticated thereby helping users explore the Elasticsearch data effortlessly. All the search, analytics and visualization capability lead to one thing: reporting.

We recently published a white paper discussing the reporting options for Elastic Stack.

  • Elastic Reporting, from Elastic as part of Elastic Stack Features (formerly X-Pack)
  • Skedler Reports, a reporting solution provided by Guidanz Inc.

In the white paper, we dive into the details of the two reporting tools, compare their features and discuss their use cases. While both the tools provide excellent reporting features for Elastic stack, they differ in several areas. Below is a brief highlight:  

Customization

Being able to customize reports is very important, it not only allows for flexibility in presenting the information, but it also enables users to personalize the reports while building the feeling of ownership and brand. Elastic Reporting currently offers basic customization features which includes an option to add a logo, two built-in layouts, and two formats (CSV and PDF). Although this may prove to be useful in some scenarios, Elastic Reporting may be too narrow due to the lack of customization.

Skedler Reports, on the other hand, features a long list of customization features from Kibana dashboards, searches, and Grafana dashboards. Skedler Reports offers three report formats (CSV, PDF, and XLS), three layouts including a report designer for custom visualization layout, flexible templates, and report bursting. Report bursting allows users to send multiple personalized reports to groups of recipients based on a single report definition.

Ease of Use

Outstanding ease of use can dramatically decrease the resources and time needed to integrate reporting into your application. Elastic Reporting currently require users to write scripts to schedule reports and send notifications. This may not be an issue for users who are comfortable with scripts, but it may become a maintenance issue for those who aren’t. Elastic Reporting also does have a one minute time limit for generating reports, making it difficult for those who have larger dashboards.

Skedler Reports does not require the user to write scripts at any time making it easy to learn and use regardless of the user’s background. In addition, Skedler Reports can easily generate reports from large dashboards without any time limits. This allows reports to be seamlessly generated from a substantial amount of data without experiencing glitches.

Affordable

Technical abilities are not the only things that differentiate Elastic Reporting and Skedler Reports, their licensing models are also different. Elastic Reporting is part of the licensed Elastic Stack Features (formerly X-Pack) that bundles other capabilities into one package.  To deploy reporting, users must register for a Gold or Platinum license subscription (or the Free license for basic features – like CSV export). The license subscriptions can become expensive and users might end up paying for features that they don’t really need.

Skedler Reports offers a flexible and affordable licensing option.  By paying only for the reporting features that they need, users can use Skedler in conjunction with open source or third-party tools for Elasticsearch.   

Comparison

The following table summarizes the significant differences between Elastic Reporting and Skedler Reports.

Skedler Reports vs. Elastic Reporting Comparison

Conclusion

Reporting has become a critical requirement as organizations use Elastic Stack in a variety of use cases. It is crucial that users adequately evaluate and choose the best option for their organization.  The white paper discusses several scenarios for using Elastic Reporting and Skedler Reports. For more guidance on choosing the best reporting option for your use case, download the full white paper and discover the reporting solution that works best for you.

Download The White Paper

 

Translate »