The release of Skedler in November came with many improvements, such as auto-scaling support for Grafana dashboard layout reports and an updated user interface. In the December release, we came up with more features like Autoscaling support for charts in Kibana and the option to configure proxy URL. We are very proud of these releases, but the team is always looking forward to new ways of making Skedler better for you. We are already improving our product further and wanted you to know about our newly added features and UI.So, before we end the year, we want to update you on the features we released and go through some of the important ones in this blog.
Halt your reporting schedules for Specific Days
Want to make sure you are not sending your reports on a holiday? We got you covered! You can now choose the days you do not wish to schedule reports with our new Weekday feature.
Autoscaling support for charts in Kibana
Skedler now supports autoscaling of charts in Kibana. You do not have to worry about your reports being messy or missing out on important information when you add more data to your chart because Skedler will automatically take care of that.
Added an auto-scaling support for Grafana dashboard layout reports
You can now stop worrying about your graphs and modules getting distorted in your reports as Skedler has added auto-scaling support for generating reports from Grafana Dashboard.
Added a privilege to super admin users to change their email id
Super Admins can now update their email ID in their profile. You can add a new Mail ID instead of the one you used when you opened your account.
Generate reports using Grafana dashboard timezone
You can now generate reports in Skedler as per your Grafana time window by selecting “use dashboard time” in Skedler. You do not have to worry about missing or skipping any reports.
Support for fiscal year time window in Grafana dashboards.
Grafana 8.2 has the option of the configurable fiscal year in the time picker. This option enables fiscal quarters as time ranges for business-focused and executive dashboards. Skedler now supports this feature too!
Added support for Outlook SMTP
Skedler now supports Outlook. So you can set up Outlook as your notification channel in your Skedler account.
These are just some of the new features of Skedler. For more details on these features, do check out our release notes.
If you would like to stay updated on the latest release news or know about upcoming features, please feel free to reach out to the team and keep an eye out for our monthly newsletters.
According to GrafanaLabs, Grafana is an open-source visualization and analytics software. No matter where your data is stored, it can be queried, visualized, and explored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations.
Why do companies use Grafana?
Companies use Grafana to monitor their infrastructure and log analytics, predominantly to improve their operational efficiency. Dashboards make tracking users and events easy as it automates the collection, management, and viewing of data. Product leaders, security analysts, and developers use this data to guide their decisions. Studies show companies that rely on database analytics and visualization tools like Grafana are far more profitable than their peers.
Why is Grafana important?
Grafana shows teams and companies what their users really do, not just what they say they do. These are known as revealed behaviors. Users aren’t very adept at predicting their own futures. Having analytics allows tech teams to dig deeper than human-error-prone surveys and monitoring.
Grafana makes that data useful again by integrating all data sources into one single organized view
What Is a Grafana Dashboard?
A Grafana dashboard supports multiple panels in a single grid. You can visualize results from multiple data sources simultaneously. It is a powerful open-source analytical and visualization tool that consists of multiple individual panels arranged in a grid. The panels interact with configured data sources including (but not limited to) AWS CloudWatch, Microsoft SQL server, Prometheus, MySQL, InfluxDB, and many others.
Grafana supports a huge list of data sources including (but not limited to) AWS CloudWatch, Microsoft SQL server, Prometheus, MySQL, InfluxDB, and many others.
What features does Grafana provide?
The tools that teams actually use to uncover insights vary from organization to organization. The following are the most common (and useful) features they might expect of a data analytics/visualization tool like Grafana.
Common Grafana features:
Visualize: Grafana has a plethora of visualization options to help you understand your data from graphs to histograms, you have it all.
Alerts: Grafana lets you define thresholds visually, and get notified via Slack, PagerDuty, and more
Unify: You can bring your data together to get better context. Grafana supports dozens of databases, natively.
Open-Source: It’s completely open source. You can use Grafana Cloud, or easily install on any platform.
Explore Logs: Using label filters you can quickly filter and search through the laundry list of logs.
Display dashboards: Visualize data with templated or custom reports.Create and Share reports:
Create and Share reports: Create and share reports to your customers and stakeholders. This feature is not available in the open-source version. You can upgrade to avail it.
Check out these 3 best Grafana reporting tools here
How to use Grafana
All data visualization platforms are built around two core functions that help companies answer questions about users and events:
Tracking data: Capturing visits, events, and monitoring actions through logs
Analyzing data: Visualizing data through dashboards and reports.
With data that’s been tracked, captured, and organized, companies are free to analyze:
What actions are users taking on the device, network, etc.?
The typical behavior flow that users take through our network or app
Opportunities to reduce SLA churn
The answers they receive arm them with statistically valid facts upon which to base security and operational decisions. Grafana is also commonly used to monitor synthetic metrics.
What are Synthetic Metrics?
Synthetic metrics are a collection of multi-stage steps required to complete an API call or transaction.
A set of metrics for an API call would contain:
Time to connect to API (connect latency)
Duration of request (response latency)
Size of response payload
Result Code of request (200, 204, 400, 500, etc)
Success/Failure state of the request
From there, teams typically graduate to proving or disproving hypotheses. For instance, a patch management solution provider/user may get the following questions addressed — “When is the best time to patch all the systems? Which are the unpatched systems in the network? What are the most vulnerable devices in a network etc.. Over time, teams build up a repository of data-backed evidence which allows them to create positive feedback loops. That is, the more data teams get back from Grafana, the more they can iterate their operations.
Getting started with Grafana is easy — Install Grafana Locally > Configure your data source > Create your first dashboard
What Are Some of the Real-World Industry Use Cases of Grafana?
As mentioned by 8bitmen.com, Grafana dashboards are deployed all over the industry be it Gaming, IoT, FinTech or E-Comm.
Grafana allows companies to fully understand the Hows and Whats of users/events with respect to their infrastructure or network. It is especially useful for security analytics teams so they can track events and users’ digital footprints to see what they are doing inside their network. Analytics is a critical piece of modern SecOps and DevOps as most apps and websites aren’t designed to run detailed reports or visualizations on themselves. Without proper visualizations, the data they collect is often inconsistent and improperly formatted (known as unstructured data). Grafana makes that data useful again by integrating all data sources into one single organized view.
Well, you might as well ask if Grafana is free to use. Yes, absolutely. But if you want to send reports of your dashboards to your customers and stakeholders you will have to opt for grafana enterprise.
Open distro supports OpenID so you can seamlessly connect your Elasticsearch cluster with Identity Providers like Azure AD, Keycloak, Auth0, or Okta. To set up OpenID support, you just need to point Open distro to the metadata endpoint of your provider, and all relevant configuration information is imported automatically. In this article, we will implement a complete OpenID Connect setup including Open distro for Kibana Single Sign-On.
What is OpenID Connect?
OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol. It allows Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.
Configuring OpenID Connect in Azure AD
Next, we will set up an OpenID Connect client application in Azure AD which we will later use for Open Distro for Elasticsearch Kibana Single Sign-On. In this post, we will just describe the basic steps.
Adding an OpenID Connect client application
Our first step is, we need to register an application with the Microsoft identity platform that supports OpenID Connect. Please refer to the official documentation.
Besides the client ID, we also need the client secret in our Open Distro for elasticsearch Kibana configuration. This is an extra layer of security. An application can only obtain an id token from the IdP if it provides the client secret. In Azure AD you can find it under the Certificates & secrets tab of the client settings.
Connecting OpenDistro with Azure AD
For connecting Open Distro with Azure AD we need to set up a new authentication domain with type openid in config.yml. The most important information we need to provide is the Metadata Endpoint of the newly created OpenID connect client. This endpoint provides all configuration settings that Open Distro needs. The URL of this endpoint varies from IdP to IdP. In Azure AD the format is:
Since we want to connect Open Distro for Elasticsearch Kibana with Azure AD, we also add a second authentication domain which will use the internal user database. This is required for authenticating the internal Kibana server user. Our config.yml file now looks like:
While an IDP can be used as a federation service to pull in user information from different sources such as LDAP, in this example we use the built-in user management. We have two choices when mapping the Azure AD users to Open Distro roles. We can do it by username, or by the roles in Azure AD. While mapping users by name is a bit easier to set up, we will use the Azure AD roles here.
With the default configuration, two appRoles are created, skedler_role and guidanz_role, which can be viewed by choosing the App registrations menu item within the Azure Active Directory blade, selecting the Enterprise application in question, and clicking the Manifest button
A manifest is a JSON object that looks similar to:
"description":"Skedler with administrator access",
"description": "guidanz with readonly access",
There are many different ways we might decide to map how users within AAD will be assigned roles within Elasticsearch, for example, using the tenantid claim to map users in different directories to different roles, using the domain part of the name claim, etc.
With the role OpenID connect token attribute created earlier, however, the appRole to which an AAD user is assigned will be sent as the value of the Role Claim within the OpenID connect token, allowing:
Arbitrary appRoles to be defined within the manifest
Assigning users within the Enterprise application to these roles
Using the Role Claim sent within the SAML token to determine access within Elasticsearch.
For the purposes of this post, let’s define a Superuser role within the appRoles:
"description":"Superuser with administrator access",
And save the changes to the manifest:
Configuring OpenID Connect in Open Distro for Kibana
The last part is to configure OpenID Connect in Open Distro for Kibana. Configuring the Kibana plugin is straight-forward: Choose OpenID as the authentication type, and provide the Azure AD metadata URL, the client name, and the client secret. Please refer to the official documentation.
Activate OpenID Connect by adding the following to kibana.yml:
Done. We can now start Open Distro for Kibana and enjoy Single Sign-On with Azure AD! If we open Kibana, we get redirected to the login page of Azure AD. After providing username and password, Kibana opens, and we’re logged in.
OpenID Connect is an industry-standard for providing authentication information. Open Distro for Elasticsearch and their Open Distro for Kibana plugin support OpenID Connect out of the box, so you can use any OpenID compliant identity provider to implement Single Sign-On in Kibana. These IdPs include Azure AD, Keycloak, Okta, Auth0, Connect2ID, or Salesforce.
If you wish to have an automated reporting application, we recommend downloading Skedler Reports.
We are excited to announce the general availability of Skedler Reports V4.6. Skedler Reports can now be used to generate PDF, CSV, Excel reports from version 7.4.x of Elastic Stack and version 6.4.3 of Grafana.
Skedler Reports version 4.6 is available now on our official download page. Here’s everything you need to know about the new Skedler Reports v4.6.
New With the Skedler Reports v4.6
Support for the latest version of Elastic Stack and Grafana. Skedler Reports v4.6 works with the following versions:
There is a simple and effective way to add reporting for your Elasticsearch Kibana 7.x (including Open Distro for Elasticsearch) or Grafana 6.x applications that are deployed to Kubernetes. In this part of the article, you are going to learn how to deploy Skedler Reports for Elasticsearch Kibana and Grafana applications to Kubernetes with ease.
What is Kubernetes?
For those that haven’t ventured into container orchestration, you’re probably wondering what Kubernetes is. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
Kubernetes (“k8s” for short), was a project originally started at, and designed by Google, and is heavily influenced by Google’s large scale cluster management system, Borg. More simply, k8s gives you a platform for managing and running your applications at scale across multiple physical (or virtual) machines.
Kubernetes ConfigMaps allows containerized application to become portable without worrying about configurations. Users and system components can store configuration data in ConfigMap. In Skedler Reports ConfigMaps can be used to store database connection string information such as datastore settings, port number, server information and files locations, log directory etc.
2. To deploy your configmap, execute the following command,
Creating Deployment and Service
To deploy our Skedler Reports, we’re going to use the “skedler-deployment” pod type. A deployment wraps the functionality of Pods and Replica Sets to allow you to update your application. Now that our Skedler Reports application is deployed, we need a way to expose it to traffic from outside the cluster. To this, we’re going to add a Service inside the skedler-deployment.yaml file. We’re going to open up a NodePort directly to our application on port 30000.
1.Create a file called skedler-deployment.yaml in your project directory and paste the following
2. For deployment, execute the following command,
3. To get your deployment with kubectl, execute the following command,
kubectl get deployments
4. We can get the service details by executing the following command,
kubectl get services
Now, Skedler will be deployed in 30000 port.
Skedler Reports can be accessed from the following URL, http://<hostIP>:30000
This blog was a very quick overview of how to get a Skedler Reports for Elasticsearch Kibana 7.x and Grafana 6.x application up and running on Kubernetes with the least amount of configuration possible. Kubernetes is an incredibly powerful platform that has many more features than we used today. We hope that this article gave a headstart and saved you time.
Self Service Reporting Solution for Elasticsearch Kibana 7.0 and Grafana 6.1
We understand that your stakeholders and customers need intuitive and flexible options to save time in receiving the data that matters to them and we’ve achieved exactly that with the release of Skedler 4.1. The newly enhanced UI offers a delightful user experience for creating and scheduling reports from your Elasticsearch Kibana 7.0 and Grafana 6.1 .
If you are a service provider, you need a simple and automated way to provide different groups of users (i.e. “tenants”) with access to different sets of data. Skedler 4.1’s powerful and secure multi-tenancy capabilities will now allow you to send reports to your customers from your multi-tenant analytics application within minutes. Supported with Search Guard, Open Distro & X-Pack.
Intuitive and Mobile Ready Reports
Skedler 4.1 will now allow you to produce high-resolution HTML reports from Elasticsearch Kibana and Grafana that will make it easy and convenient for your end users to access to critical data through their mobile devices and email clients. No more cumbersome and large PDF attachments.
Matteo Zuccon is a software developer with a passion for web development (RESTFull services, JS Frameworks), Elasticsearch, Spark, MongoDB, and agile processes. He runs whiletrue.run. Follow him on Twitter @matteo_zuccon.
With Kibana you can create intuitive charts and dashboards. Since Aug 2016 you can export your dashboards in a PDF format thanks to Reporting. With Elastic version, 5 Reporting has been integrated into X-Pack for the Premium and Enterprise subscriptions.
Recently I tried Skedler, an easy to use report scheduling and distribution application for Kibana that allows you to centrally schedule and distribute Kibana Dashboards and Saved Searches as hourly/daily/weekly/monthly PDF, XLS or PNG reports to various stakeholders.
Skedler is a standalone app that allows you to utilize a new dashboard where you can manage Kibana reporting tasks (schedule, dashboards and saved search). Right now there are four different price plans (from free to premium edition).
In this post I am going to show you how to install Skedler (on Ubuntu) and how export/schedule a Kibana dashboard.
Skedler Reports has a number of configuring options that can be defined in its reporting.yml file (located in the skedler folder). In the reporting.yml file, you can configure options to run Skedler in an air-gapped environment, change the port number, define the hostname, change the location for the Skedler database, and log files.
If you had made configuration changes in the reporting.yml, then the Skedler URL is of the following format:
http://<hostname or your domainurl>:3005
http://<hostname or your domain url>:<port number>
Login to Skedler Reports
By default, you will see the Create an account UI. Enter your email to create an administrator account in Skedler Reports. Click on Continue.
Note: If you have configured an email address and password in reporting.yml, then you can skip the create account step and proceed to Login.
An account will be created and you will be redirected to the Login page.
Sign in using the following credentials:
Username: <your email address> (or the email address you configured in reporting.yml)Password: admin (or the password you configured in reporting.yml)
Click Sign in.
You will see the Reports Dashboard after logging in to the skedler account.
In this post, I demonstrated how to install and configure Skedler and how to create a simple schedule for our Kibana dashboard. My overall impression of Skedler is that it is a powerful application to use side-by-side with Kibana that allows you to deliver reports directly to your stakeholders.
These are the main benefits that Skedler offers:
It’s easy to install
Linux, Windows and Mac OS support (it runs on Node.js server)
Reports are generated locally (your data isn’t sent to the cloud or Skedler servers)
Competitive price plans
Supports Kibana and Grafana.
Automatically discovers your existing Kibana Dashboards and Saved Searches (so you can easily use Skedler in any environment with no new stack installation needed)
It lets you centrally schedule and manage who gets which reports and when they get them
Allows for hourly, weekly, monthly, and yearly schedules
Generates XLS and PNG reports besides PDF as opposed to Elastic Reporting that only supports PDF.
I strongly recommend that you try Skedler because it can help you to automatically deliver reports to your stakeholders and it integrates within your ELK environment without any modification to your stack.
Every company carries valuable data, whether it’s relevant to a specific client’s private information, statistics, or finances. Alongside the fear that you might lose precious data or incur a security breach with faulty programs, data needs to be filed and exported accurately in order to meet certain deadlines and practical standards; which is why Kibana reporting has proven to be an imperative tool for loss prevention within any given company.
Kane LPI Solutions is a prime example of how Kibana reporting achieved higher marketability within the cognoscente program Skedler. A trusted provider of Third Party Administration services for more than 15 years, Kane LPI has issued over US$11 billion of offshore annuity and investment products for an extensive global client base. They needed a robust solution to help prevent financial and reputation losses as well as boost KPIs. As it stood, competition was fierce — there were many much larger players out there on the field.
The company’s practical processes specifically involved sending sensitive and accurate post-trading files on a timely basis. Kane LPI’s clients had strict operational requirements for control and compliance: if the time window for sending files was missed, financial loss was borne both for them and the company. Delays like this could incur strict penalties by settlement and clearing corporations such as The Depository Trust and Clearing Corporation (DTCC). With so much at stake, our challenge was to fundamentally satisfy operational concerns to prevent those losses, as well as improve pragmatic business flow.
Kibana Reporting: A Means of Automation
Kibana reporting allowed Kane LPI to send out daily scheduled reports to both clients and internal users from thousands of lines of log entries within multiple systems, serving as a key monitoring tool and satisfying auditing requirements in the process. As a result, Skedler became a critical go-to ELK stack tool for this company and its clients, allowing Kane LPI to send out reports to their clients and internal users, which the company couldn’t do before.
Developers were able to receive automated error reports at the beginning of the day, allowing them to ensure information is sent to regulatory and settlement organizations to meet deadlines. Simultaneously, managers received daily and weekly reports on SLA performance and non-compliance, enabling them to take remedial steps to prevent recurrence and to protect time-sensitive transactions. Vendors also received daily and weekly reports on SLA performance and non-compliance of their product in KPI solution, which fundamentally reused the investment of energy and time in an ELK stack based solution.
As a precautionary measure, Skedler introduced a critical line of defence to errors by inaugurating manager and software vendors with automated reporting sent at the beginning of the day, which described potential errors during transfer of files, allowing future errors to be minimized. We then added another layer of security by administering any issues during batch runs from the previous 24 hours.
Statistically, all of these enhancements enabled Kane LPI’s clients to avoid up to $5 million per month in trade and exchange losses, as well as protect their well-nurtured reputation as a high-quality provider.
Ready to start saving time by creating, scheduling and distributing Kibana reports automatically? Try Skedler for free.
Automated processes are, invariably, becoming more and more integral to our everyday lives, both in and out of the office. They’ve replaced much of the manual workforce and have improved systematic procedures, which otherwise would be at the mercy of various human error elements as well as higher risks of data breaches. This, as well as recognizing manual reporting as time-consuming labour, are some key issues we don’t need to worry about any more by virtue of processing automation; Kibana being one of those favorable products.
Focus on What Matters
As a result of businesses adopting bots as part of our everyday processes, we’re left with the far more creative aspects of information science (which automation hasn’t quite caught up with yet). Naturally, Elasticsearch’s aesthetically enhanced data delivery is one of its chief selling points: users are able to explore unchartered data with clear-cut digital graphics at their very disposal. This significant upgrade in data technology has allowed us to possess more varied and complex insights; it’s more exciting now than it has ever been before.
In contrast, however, tedious tasks such as email deliveries of reports to customers, compliance managers and other stakeholders remain arduous and time-consuming; deterring attention from more stimulating in-depth data analysis. What we know to be necessary is for analysts to have the time available to devote themselves to exploring Tableau’s analytics, instead of undergoing mundane processes such as manual spreadsheet creation, generating, email exporting, and distributing.
Automate Kibana Reports
Perhaps it’s possible that you’ve already started utilizing Kibana without realizing the perks of automated scheduling. Luckily, Skedler can completely undertake those prosaic tasks, at an affordable price. As an automated scheme which meets full compliance and operations requirements, Skedler allows your peers, customers and other stakeholders to be kept informed in a virtually effortless and secure way. Comprehensive exporting preferences such as PDF, XLS and PNG are also serviceable; allowing you the luxury of consigning instant or scheduled report generation in the format you desire.
Additionally, Skedler’s reporting motions are facilitated through its prestigious dashboard system, which automatically discovers your existing Kibana dashboards and saved searches to make them available for reporting – again, saving you time creating, scheduling and sending Kibana reports. All your filtered reporting and data chartering is available on a single, versatile platform; meaning you won’t spend extensive amounts of time searching through your outgoing email reports for a specific item.
Skedler simply allows you to examine all of your criteria through one umbrella server with clear functionalities to separate the stunning data visualization deliveries, and the slightly less exciting archive of manual spreadsheet generation and handling for other departments, which it can totally manage by itself.
Ready to start saving time by creating, scheduling and distributing Kibana reports automatically? Try Skedler for free.