## 4 Results of the Software-Prototyping
### From Logging to Structured Logging
## 8 Conclusions and Future Research Directions

Cloud-native software systems often have a much more decentralized structure and many independently deployable and (horizontally) scalable components, making it more complicated to create a shared and consolidated picture of the overall decentralized system state [74; 75]. Today, observability is often understood as a triad of collecting and processing metrics, distributed tracing data, and logging--but why except for historical reasons? This study presents a unified logging library for Python [29] and a unified logging architecture (see Figure 4) that uses a structured logging approach. The evaluation of four use cases shows that several thousand events per minute are easily processable and can be used to handle logs, traces, and metrics the same. At least, this study was able with a straightforward approach to log the world-wide Twitter event stream of stock market symbols over a period of six months without any noteworthy problems. As a side effect, some interesting aspects of how crypto-currencies are reflected on Twitter could be derived. This might be of minor relevance for this study but shows the overall potential of a unified and structured logging based observability approach. The presented approach relies on an easy-to-use programming language-specific logging library that follows the structured logging approach. The long-term observation results of more than six months indicate that a unification of the current observability triad of logs, metrics, and traces is possible without the necessity to develop utterly new toolchains. The reason is the flexibility of the underlying structured logging approach. This kind of flexibility is a typical effect of data format standardization. The trick is to

* use structured logging and
* apply log forwarding to a central analytical database
* in a systematic infrastructure- or platform-provided way. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Product** & **Use Case** & **Organization** & **License** & **Remark** \\ \hline
**APM Agents**\(\otimes\)[9] & Tracing & Elastic & BSD 3 & \\ Jaeger Clients [49] & Tracing & Linux Foundation & Apache 2.0 & \\ log [59] & Logging & Go Standard Library & BSD 3 & Logging for Go \\ log4j [60] & Logging & Apache & Apache 2.0 & Logging for Java \\
**logging**\(\otimes\)[61] & Logging & Python Standard Library & GPL compatible & Logging for Python \\ Micrometer [62] & Metrics & Pivotal & Apache 2.0 & \\ Open Telemetry [10] & Tracing & Open Telemetry & Apache 2.0 & \\ prometheus [51] & Metrics & Linux Foundation & Apache 2.0 & \\ Splunk APM [63] & Tracing & Splunk & Apache 2.0 & \\
**structlog**\(\otimes\)[64] & Logging & [PERSON] & Apache 2.0, MIT & structured logging for Python \\ winston [65] & Logging & [PERSON] & MIT & Logging for node.js \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Often seen instrumenting libraries. Products that inspired the research prototype are marked **bold**\(\otimes\), without claiming completeness. Further research should therefore be concentrated on the instrumenting and less on the log forwarding and consolidation layer. If we instrument logs, traces, and metrics in the same style using the same log forwarding, we automatically generate correlatable data in a single data source of truth, and we simplify analysis. Thus, the observability road ahead may have several paths. On the one hand, we should standardize the logging libraries in a structured style like log12 in this study or the OpenTelemetry project in the \"wild\". Logging libraries should be comparably implemented in different programming languages and shall generate the same structured logging data. Thus, we have to standardize the logging SDKs and the data format. Both should be designed to cover logs, metrics, and distributed traces in a structured format.