With a mindset shift in most organizations adopting DevOps & Agile practices, one of the usual starting points in their transformation journey was to break down their monolith into several microservices. This, not only, helped with the continuous integration, delivery, and testing tenet but also expedited changes that previously used to take weeks or even months to execute. However, this transformation in architecture presented its own set of challenges with respect to monitoring. For traditional architectures, monitoring was relegated to understanding a known set of failures based on usage thresholds & parsing content off logs. With the architectural shift, monitoring alone no longer served the purpose of understanding the state of the system given that failures within the newer systems were never linear. 

Observability & Telemetry

Thus was born observability as a discipline to complement monitoring with its data-driven approach. In short, monitoring systems didn’t die out but were supplemented with more data to understand the internal state of the architecture & navigate from effect to cause more easily with the discipline of observability. Founded on the three pillars of logs, metrics, and tracing, commonly known as telemetry data, observability systems enabled us to understand our systems & their failures better.

However, with an increasing demand for observability systems, there also was another challenge on the rise – the lack of standardization in the offerings. In addition to this, the ones that were adopted lacked portability across languages. A combination of the above two challenges resulted in the overhead of implementations being maintained by the developer/SRE staff within the organization contributing to an increase in complexity & workload.

Thus was born OpenTelemetry: Built-in, high-quality telemetry for all

In 2019, the maintainers of OpenTracing (a CNCF vendor-agnostic tracing project) & OpenCensus (a vendor-agnostic tracing & metrics library led by Google) merged towards solving some of these challenges and standardizing the telemetry ecosystem with OpenTelemetry. As outlined in this excellent announcement post, the vision of the project was to provide a unified set of instrumentation libraries and specifications towards providing built-in high-quality telemetry for all. 

With an open, vendor-agnostic standard that was backward-compatible with both of its founding projects, OpenTelemetry’s aim was to allow for cross-platform & streamlined observability that would allow for more focus on delivering reliable software without getting mired down by the various available options. Because in the end, isn’t that the end goal of every business?

The nitty-gritty

A CNCF incubating project as of writing this post, OpenTelemetry is composed of the following main components as of v 0.11.0 released on 8th October 2021

1. Proto files to define language independent interface types such as collectors, instrumentation libraries etc. 
2. Specifications to describe the cross-language requirements for all implementations. 
3. APIs containing the interfaces & implementations of the specifications
4. SDKs implementing the APIs with processing & exporting capabilities
5. Collectors to receive, process, and export the telemetry data in a vendor-agnostic manner
6. Instrumentation libraries towards enabling observability for other libraries via manual & automatic instrumentation

As aforementioned, both manual & automatic instrumentation are supported; automatic instrumentation,  being the simpler of the two, involves only the addition of dependencies and configuration via environment variables or language-specific means such as system properties in Java. Manual instrumentation, on the other hand, would involve significant code dependencies on the API & SDK components in addition to the actual creation & exporting of the telemetry data. While extremely useful, a significant drawback is that manual instrumentation can lead to redundancies & inconsistencies in the way we treat observability data along with being a massive expenditure of manual efforts.

So where are we headed?

As of today, 14 vendors support OpenTelemetry. With a focus on developing the project on a signal-by-signal basis, the project aims to stabilize & improve LTS for instrumentation. With support for over 11 languages, there are also efforts expended towards expanding & improving the instrumentation across a wider variety of libraries as also incorporating testing & CI/CD tooling towards writing & verifying the quality of instrumentation offered.

With a vibrant community & extensive documentation around the project, there has never been a better time to get involved in the efforts towards standardizing the efforts for built-in high-quality telemetry.

Automate your  Grafana and  Kibana Reports Today!
Reporting Made Simple.

Start your free trial