What’s New in Skedler

The release of Skedler in November came with many improvements, such as auto-scaling support for Grafana dashboard layout reports and an updated user interface. In the December release, we came up with more features like Autoscaling support for charts in Kibana and the option to configure proxy URL. We are very proud of these releases, but the team is always looking forward to new ways of making Skedler better for you. We are already improving our product further and wanted you to know about our newly added features and UI.So, before we end the year, we want to update you on the features we released and go through some of the important ones in this blog.

Halt your reporting schedules for Specific Days

Want to make sure you are not sending your reports on a holiday? We got you covered! You can now choose the days you do not wish to schedule reports with our new Weekday feature.

Weekday feature

Autoscaling support for charts in Kibana

Skedler now supports autoscaling of charts in Kibana. You do not have to worry about your reports being messy or missing out on important information when you add more data to your chart because Skedler will automatically take care of that.

Autoscaling in Kibana

Added an auto-scaling support for Grafana dashboard layout reports 

You can now stop worrying about your graphs and modules getting distorted in your reports as Skedler has added auto-scaling support for generating reports from Grafana Dashboard.

Autoscaling in Grafana

 Added a privilege to super admin users to change their email id

Super Admins can now update their email ID in their profile. You can add a new Mail ID instead of the one you used when you opened your account.

Super Admin User

 Generate reports using Grafana dashboard timezone

You can now generate reports in Skedler as per your Grafana time window by selecting “use dashboard time” in Skedler. You do not have to worry about missing or skipping any reports.

Dashboard Timezone

Support for fiscal year time window in Grafana dashboards. 

Grafana 8.2  has the option of the configurable fiscal year in the time picker. This option enables fiscal quarters as time ranges for business-focused and executive dashboards. Skedler now supports this feature too!

Fiscal Time Year Window

Added support for Outlook SMTP

Skedler now supports Outlook. So you can set up Outlook as your notification channel in your Skedler account.

Outlook SMTP

These are just some of the new features of Skedler. For more details on these features, do check out our release notes.

If you would like to stay updated on the latest release news or know about upcoming features, please feel free to reach out to the team and keep an eye out for our monthly newsletters.

An Easy Way to Export / Import Scheduled Reports from Skedler

Here are the highlights of what’s new and improved in Skedler Reports 4.20.0. For detailed information about this release, check the release notes.

In Skedler Reports 4.20.0 we are now able to import and export the scheduled reports from one user to another user with ease. 

Export and Import anonymous user to skedlerAdmin user

To do the export/import in your existing skedler for anonymous users and an admin user please follow the below steps, prior to that could you please back up your skedler index or internal DB for safety purposes.

To know how to back up the Skedler index or internal DB please click here

1.Please select all the scheduled reports in the dashboard by clicking the select all option check box in the anonymous user

2. Now pause all the scheduled reports which you are going to import in the admin user login.

3. Once you click the export button, the _reports.json file will be generated. 

4. Now go to the reporting.yml file and navigate to the SKEDLER SECURITY SETTINGS. Uncomment skedler_anonymous_access to set the value to “false”

5. Restart skedler and enter the credentials and now go to the scheduled reports skedlerAdmin user and click import.

6. Open the _reports.json file and all the reports which were showing in anonymous user will also be showing in the admin user.

7. Now resume the scheduled report and check whether the reports are generated. 

Note: If you wish to re-import the _reports.json file. Select all the reports and delete them. Also, delete the burst filters and templates if they are added from the JSON file.

Skedler Reports v4.19.0 & Alerts v4.9.0 now supports ELK 7.10

Here are the highlights of what’s new and improved in Skedler Reports 4.19.0 & Alerts 4.9.0. For detailed information about this release, check the release notes.

Indexing speed improvement

Elasticsearch 7.10 improves indexing speed by up to 20%. We’ve reduced the coordination needed to add entries to the transaction log. This reduction allows for more concurrency and increases the transaction log buffer size from 8KB to 1MB. However, performance gains are lower for full-text search and other analysis-intensive use cases. The heavier the indexing chain, the lower the gains, so indexing chains that involve many fields, ingest pipelines or full-text indexing will see lower gains which can now be utilized in Skedler v4.19.0.

More space-efficient indices

Elasticsearch 7.10 depends on Apache Lucene 8.7, which introduces higher compression of stored fields, the part of the index that notably stores the _source. On the various data sets that we benchmark against, we noticed space reductions between 0% and 10%. This change especially helps on data sets that have lots of redundant data across documents, which is typically the case of the documents that are produced by our Observability solutions, which repeat metadata about the host that produced the data on every document.

Elasticsearch offers the ability to configure the index.codec setting to tell Elasticsearch how aggressively to compress stored fields. Both supported values default and best_compression will get better compression with this change.

Data tiers

7.10 introduces the concept of formalized data tiers within Elasticsearch. Data tiers are a simple, integrated approach that gives users control over-optimizing for cost, performance, and breadth/depth of data. Prior to this formalization, many users configured their own tier topology using custom node attributes as well as using ILM to manage the lifecycle and location of data within a cluster.

With this formalization, data tiers (content, hot, warm, and cold) can be explicitly configured using node roles, and indices can be configured to be allocated within a specific tier using index-level data tier allocation filtering. ILM will make use of these tiers to automatically migrate data between nodes as an index goes through the phases of its lifecycle.

Newly created indices abstracted by a data stream will be allocated to the data_hot tier automatically, while standalone indices will be allocated to the data_content tier automatically. Nodes with the pre-existing data role are considered to be part of all tiers.

AUC ROC evaluation metrics for classification analysis

The area under the curve of the receiver operating characteristic (AUC ROC) is an evaluation metric that has been available for outlier detection since 7.3 and now is available for classification analysis. AUC ROC represents the performance of the classification process at different predicted probability thresholds. The true positive rate for a specific class is compared against the rate of all the other classes combined at the different threshold levels to create the curve.

Custom feature processors in data frame analytics

Feature processors enable you to extract process features from document fields. You can use these features in model training and model deployment. Custom feature processors provide a mechanism to create features that can be used at search and ingest time and they don’t take up space in the index. This process more tightly couples feature generation with the resulting model. The result is simplified model management as both the features and the model can easily follow the same life cycle.

Points in time (PITs) for search

In 7.10, Elasticsearch introduces points in time (PITs), a lightweight way to preserve index state over searches. PITs improve the end-user experience by making UIs more reactive supported by Skedler v4.19.0

By default, a search request waits for complete results before returning a response. For example, a search that retrieves top hits and aggregations returns a response only after both top hits and aggregations are computed. However, aggregations are usually slower and more expensive to compute than top hits. Instead of sending a combined request, you can send two separate requests: one for top hits and another one for aggregations. With separate search requests, a UI can display top hits as soon as they’re available and display aggregation data after the slower aggregation request completes. You can use a PIT to ensure both search requests run on the same data and index state.

New thread pools for system indices

We’ve added two new thread pools for system indices: system_read and system_write. These thread pools ensure system indices critical to the Elastic Stack, such as those used by security or Kibana, remain responsive when a cluster is under heavy query or indexing load.

system_read is a fixed thread pool used to manage resources for reading operations targeting system indices. Similarly, system_write is a fixed thread pool used to manage resources for write operations targeting system indices. Both have a maximum number of threads equal to 5 or half of the available processors, whichever is smaller.

Export your Kibana Dashboard to PDF Report in Minutes with Skedler. Fully featured free trial.

Skedler Reports v4.18.0 now supports Grafana 7.3.0

Here are the highlights of what’s new and improved in Skedler Reports 4.18.0 & Alerts 4.10.0. For detailed information about this release, check the release notes.

Table improvements and new image cell mode

The table has been updated with improved hover behavior for cells that have longer content than what fits the current column width. Another new feature that can be seen in the image above is the new image cell display mode. If you have a field value that is an image URL or a base64 encoded image you can configure the table to display it as an image. 

Table color scheme

Another thing to highlight is that all these new color schemes are theme aware and adapt to the current theme. As this new option is a standard field option it works in every panel which is supported in Skedler v4.18.1

Shorten URL for dashboards and Explore

The new share shortened link capability allows you to create smaller and simpler URLs of the format /goto/:uid instead of using longer URLs that can contain complex query parameters. In Explore, you can create a shortened link by clicking on the share button in the Explore toolbar. In the dashboards, a shortened URL option is available through the share panel or dashboard button.

Auditing

Auditing tracks important changes to your Grafana instance to help you manage and mitigate suspicious activity and meet compliance requirements. Grafana logs events (as JSON) to file or directly to Loki.

Skedler Reports v4.17.0 now supports Grafana 7.2.0

Here are the highlights of what’s new and improved in Skedler Reports 4.17.0 & Alerts 4.9.0. For detailed information about this release, check the release notes.

New date formatting options added

You can now customize how dates are formatted in Grafana. Custom date formats apply to the time range picker, graphs, and other panel visualizations which is now supported for Skedler.

Generate Report from Kibana in Minutes with Skedler. Fully featured free trial.

This screenshot shows both a custom full date format with a clock and am / pm suffix. The graph also shows the same 12-hour clock format and a customized month and day format compared to the Grafana default MM/DD format.

Date formats are set for a Grafana instance by adjusting server-wide settings in the Grafana configuration file. We hope to add org- and user-level settings in the future.

[date_formats]

full_date = MMM Do, YYYY @ hh:mm:ss a

interval_second = hh:mm:ss a

interval_minute = hh:mm a

interval_hour = MMM DD hh:mm a

interval_day = MMM DD

interval_month = YYYY-MM

interval_year = YYYY

Field options are now available with full swing!

Table Column filters added

You can now dynamically apply value filters to any table column. This option can be enabled for all columns or one specific column using an override rule.

New field override selection options

You can now add override rules that use a regex matcher to choose which fields to apply rules to. The Field options content has been updated as a result of these changes.

New transformations and enhancements

Grafana 7.2 includes the following transformation enhancements supported to Skedler,

  • A new Group By transformation that allows you to group by multiple fields and add any number of aggregations for other fields.
  • The Labels to field transformation now allows you to pick one label and use that as the name of the value field.
  • You can drag transformations to reorder them. Remember that transformations are processed in the order they are listed in the UI, so think before you move something!

Drag to reorder queries

The up and down arrows, which were previously the only way to change query order, have been removed. Instead, there is now a grab icon that allows you to drag and drop queries in a list to change their order.

Inspect queries in Explore

The query inspector information provided in your dashboards can now also be reviewed in Explore. You can open the query inspector tab by clicking the button next to the query history.

$__rate_interval for Prometheus

You can now use the new variable $__rate_interval in Prometheus for rate functions. $__rate_interval, in general, is one scrape interval larger than $__interval but is never smaller than four times the scrape interval (which is 15s by default).

Export your Kibana Dashboard to PDF Report in Minutes with Skedler. Fully featured free trial.

What’s new in Skedler Reports 4.16.0 & Alerts 4.8.0

Here are the highlights of what’s new and improved in Skedler Reports 4.16.0 & Alerts 4.8.0. For detailed information about this release, check the release notes.

EQL

EQL (Event Query Language) is a declarative language dedicated to identifying patterns and relationships between events. Consider using EQL if you use Elasticsearch for threat hunting or other security use cases. Search time-series data or logs, such as network or system logs

Data streams

A data stream is a convenient, scalable way to ingest, search, and manage continuously generated time-series data. They provide a simpler way to split data across multiple indices and still query it via a single named resource.

Enable fully concurrent snapshot operations

Snapshot operations can now execute in a fully concurrent manner.

  • Create and delete operations can be started in any order
  • Delete operations wait for snapshot finalization to finish and are batched as much as possible to improve efficiency. 
  • Snapshot creation is completely concurrent across shards, but per shard, snapshots are linearized for each repository, as are snapshot finalizations

Indexing metrics and backpressure

ELK 7.9 now tracks metrics about the number of indexing request bytes that are outstanding at each point in the indexing process (coordinating, primary, and replication) which is supported by Skedler. These metrics are exposed in the node stats API. Additionally, the new setting indexing_pressure.memory.limit controls the maximum number of bytes that can be outstanding, which is 10% of the heap by default. Once this number of bytes from a node’s heap is consumed by outstanding indexing bytes, Elasticsearch will start rejecting new coordinating and primary requests.

Inference in pipeline aggregations

In this release, the inference is even more flexible! You can reference a pre-trained data frame analytics model in aggregation to infer on the result field of the parent bucket aggregation. The aggregation uses the model on the results to provide a prediction. This addition enables you to run classification or regression analysis at search time. If you want to perform analysis on a small set of data, you can generate predictions without the need to set up a processor in the ingest pipeline.

Skedler Reports v4.15.0 now supports Grafana 7.1.0

Now Skedler supports Grafana 7.1.0. with all its new features in Skedler Reports v4.15.0

Influx data source

Support for Flux and Influx v2 has been added, now Build Grafana Dashboards with InfluxDB, Flux, and InfluxQL and explain the changes in depth.

Query history search

In Grafana v 7.1 we are introducing search functionality in Query history. You can search across queries and your comments. It is especially useful in combination with a time filter and data source filter. 

Explore modes unified

Grafana 7.1 includes a major change to Explore: it removes the query mode selector. Many data sources tell Grafana whether a response contains time series data or logs data. Using this information, Explore chooses which visualization to use for that data. This means that you don’t need to switch back and forth between Logs and Metrics modes depending on the type of query that you want to make. Grafana 7.1 includes a major change to Explore: it removes the query mode selector.

Internal links for Elasticsearch

The new internal linking feature for Elasticsearch allows you to link to other data sources from your logs. You can now create links in Elastic configuration that point to another data source (similar to an existing feature in Loki). An example would be using a trace ID field from your logs to be able to link to traces in a tracing data source like Jaeger.

Ad hoc filtering in the new table panel

Ad hoc filtering, a way to automatically add filters to queries without having to define template variables is now supported in the new Table panel.

Provisioning of apps

Grafana v7.1 adds support for the provisioning of app plugins. This allows app plugins to be configured and enabled/disabled using configuration files. 

Skedler Reports & Alerts now supports ELK 7.7.0 & Grafana 7.0.0

Now Skedler supports ELK 7.7.0 and Grafana 7.0. with all its new features in Skedler Reports v4.13.0 & Skelder Alerts v4.6.0

Features which supports Grafana 7.0 for Skelder 4.13.0

New panel editor and unified data model

The first visible change is the separated panel display settings to a right-hand side panel that you can collapse or expand depending on what your focus is on. With this change, we are also introducing our new unified option model and UI for defining data configuration and display options. A unified data configuration framework allows a single user interface to set data options through visualizations as well as to control and override all data display settings.

New tracing UI

This release provides substantial support for distributed tracing, including a telemetry mode to complement the existing metrics and log support. Traces allow you to monitor how single requests move through a distributed network. More workflows and integrations should be introduced in the future so that identification between measurements, logs, and traces is much easier.

Table panel

Grafana 7.0 comes with a new table panel (with the old one deprecated). This new table panel allows horizontal scrolling and resizing of columns. Paired with the detailed transformation of new Organize fields above you can reorder, hide & rename columns. This new panel also supports new display modes for cells, such as showing a bar gauge within a cell.

Auto grid mode for Stat panel and Gauge

This new function of 7.0 applies to the gauge and stat panels. Before, only horizontal or vertical stacking was enabled by stat and gage: The auto-layout mode only selected vertical or horizontal stacking depending on the dimensions of the panel (whatever was highest). But in 7.0 the auto layout for these two panels will allow for dynamic grid layouts where Grafana will attempt to optimize space use and map out each sub-visualization within a grid.

Features which supports ELK 7.7.0 for Skelder 4.13.0 & Skelder Alerts 4.6.0

Fixed index corruption on shrunk indices

Applying deletions or updates to the index after it has been truncated is likely to corrupt the index. We advise Elasticsearch 6.x users who opt-in for soft deletes on some of their indices and all Elasticsearch 7.x users to update to 7.7 as soon as possible so that this corruption vulnerability will no longer be applicable to them.

Significant reduction of heap usage of segments

This Elasticsearch release substantially reduces the amount of heap memory required to hold Lucene segments accessible. This helps to reduce costs, in addition to helping with cluster stability, by storing much more data per node before reaching memory limits.

Query speed-up for sorted queries on time-based indices

We optimized sorted, top-document-only queries that run on indices based on time. The optimization is due to the fact that the ranges of timestamps (document) in the shards do not overlap. This is enforced by rewriting the requests for shard searches based on partial results that are already available from other shards.

A new aggregation: top_metrics

The new top metrics aggregation “selects” a metric from a document on a given, separate field according to the requirements. At present, the criterion is the greatest or the smallest “type” value. It’s fairly close in spirit to top hits, but since it’s more constrained, top metrics uses less memory and is therefore quicker.

Skedler Single sign-on with OpenId and Azure Active Directory

Skedler supports security plugins like X-pack, open distro, search guard, Nginx, and security onion. Now our latest version of Skedler v4.12 supports SSO with OpenId Connect and Azure AD

What is SSO in Azure AD:

Single sign-on (SSO) adds security and convenience when users sign-on to applications in Azure Active Directory (Azure AD). This article describes the single sign-on methods and helps you choose the most appropriate SSO method when configuring Skedler application.

  • With single sign-on, users sign in once with one account to access domain-joined devices, company resources, software as a service (SaaS) applications, and web applications. After signing in, the user can launch applications from the Office 365 portal or the Azure AD MyApps access panel. Administrators can centralize user account management, and automatically add or remove user access to applications based on group membership.
  • Without single sign-on, users must remember application-specific passwords and sign in to each application. IT staff needs to create and update user accounts for each application such as Office 365, Box, and Salesforce. Users need to remember their passwords, plus spend the time to sign in to each application.

Choosing a single sign-on authentication method in Skedler V4.12:

There are several ways to configure Skedler application with a security plugin but we choose open distro security plugin because Open Distro for Elasticsearch gives you a comprehensive set of features to help you keep your data secure and stay compliant with regulations such as GDPR, HIPAA, PCI, and ISO. Whether you want to encrypt data-in-transit, authenticate users against Active Directory, use Kerberos or JSON web tokens for single sign-on (SSO), or monitor and log any malicious access attempts, Open Distro for Elasticsearch has you covered.

On-premises applications can use password-based, Integrated Windows Authentication, header-based, linked, or disabled methods for single sign-on.

Password-based SSO:

With password-based sign-on, users sign on to the application with a username and password the first time they access it. After the first sign-on, Azure AD supplies the username and password to the application.

Password-based single sign-on uses the existing authentication process provided by the application. When you enable password single sign-on for an application, Azure AD collects and securely stores usernames and passwords for the application. User credentials are stored in an encrypted state in the directory.

Skedler Reports v4.11.1 now supports Grafana 6.7.x

Now Skedler supports Grafana 6.7.x with all its new features in Skedler Reports v4.11.1

Enforce minimum dashboard refresh interval

Allowing a low dashboard refresh interval can cause severe load on data sources and Grafana. Grafana v6.7 allows you to restrict the dashboard refresh interval so it cannot be set lower than a given interval. This provides a way for administrators to control dashboard refresh behavior on a global level.

Stackdriver project selector

A Stackdriver data source in Grafana is configured for one service account only. That service account is always associated with a default project in the Google Cloud Platform (GCP). Depending on your setup in GCP, the service account might be granted access to more projects than just the default project.

In Grafana 6.7, the query editor has been enhanced with a project selector that makes it possible to query different projects without changing the data source.

White labeling customizes application title

A new white-labeling option is introduced to customize the application title. As with all configuration options, you can also be set with environment variables.

Translate »