## 4 Results of the Software-Prototyping
### From Logging to Structured Logging
Mostimportantly, this study has placed its focus on the instrumentation side of observability and less on the database and time series analysis side. ### Existing Instrumenting Libraries and Observability Solutions

Although the academic coverage of the observability field is expandable, in practice, there is an extensive set of existing solutions, especially for time series analysis and instrumentation. A complete listing is beyond the scope of this paper. However, from the disproportion of the number of academic papers to the number of real existing solutions, one quickly recognizes the practical relevance of the topic. Table 1 contains a list of existing database products often used for telemetry data consolidation to give the reader an overview without claiming completeness. This study used ElasticSearch as an analytical database. Table 2 lists several frequently used forwarding solutions that developers can use to forward data from the point of capture to the databases listed in Table 1. In the context of this study, FileBeat was used as a log forwarding solution. It could be proved that this solution is also capable of forwarding traces and metrics if applied in a structured logging setting. An undoubtedly incomplete overview of instrumentation libraries for different products and languages is given in Table 3, presumably because each programming language comes with its own form of logging in the shape of specific libraries. To avoid this language-binding is hardly possible in the instrumentation context unless one pursues \"esoteric approaches\" like [43]. The logging library prototype is strongly influenced by the Python standard logging library but also by structllog for structured logging but without actually using these libraries. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Product** & **Organization** & **License** & **Often Seen Scope** \\ \hline APM [9] & Elastic & Apache 2.0 & Tracing (add-on to ElasticSearch database) \\
**ElasticSearch**\(\otimes\)[47] & Elastic & Apache/Elastic License 2.0 & Logs, Tracing, (rarely Metrics) \\ InfluxDB [48] & Influxdata & MIT & Metrics \\ Jaeger [49] & Linux Foundation & Apache 2.0 & Tracing \\ OpenSearch [50] & Amazon Web Services & Apache 2.0 & Logs, Tracing, (rarely Metrics); fork from ElasticSearch \\ Prometheus [51] & Linux Foundation & Apache 2.0 & Metrics \\ Zipkin [52] & OpenZipkin & Apache 2.0 & Tracing \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Often seen databases for telemetry data consolidation. Products used in this study are marked **bold**\(\otimes\), without claiming completeness. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Product** & **Organization** & **License** \\ \hline Fluentd [53] & FluentD Project & Apache 2.0 \\ Flume [54] & Apache & Apache 2.0 \\ LogStash [55] & Elastic & Apache 2.0 \\
**FileBeat**\(\otimes\)[56] & Elastic & Apache/Elastic License 2.0 \\ Rsyslog [57] & Adiscon & GPL \\ syslog-ng [58] & One Identity & GPL \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Often seen forwarding solutions for log consolidation. Products used in this study are marked **bold**\(\otimes\), without claiming completeness. ### Standards

There are hardly any observability standards. However, a noteworthy standardization approach is the OpenTelemetry Specification [10] of the Cloud Native Computing Foundation [66], which tries to standardize the way of instrumentation. This approach corresponds to the core idea, which this study also follows. Nevertheless, the standard is still divided into Logs [67], Metrics [68] and Traces [69], which means that the conceptual triad of observability is not questioned. On the other hand, approaches like the OpenTelemetry Operator [70] for Kubernetes enable injecting auto-instrumentation libraries for Java, Node.js and Python into Kubernetes operated applications, which is a feature that is currently not addressed by the present study. However, so-called service meshes [71; 72] also use auto-instrumentation. A developing standard here is the so-called Service Mesh Interface (SMI) [73].