DOI: 10.5281/zenodo.6374398
title: syslrn: Learning What to Monitor for Efficient Anomaly Detection [Dataset]

description: This repository includes the dataset for the paper: <em>D. Sanvito, G. Siracusano, S. Santhanam, R. Gonzalez, R. Bifulco</em><br> <strong><em>syslrn: Learning What to Monitor for Efficient Anomaly Detection </em></strong><br> <em>ACM EuroMLSys 2022</em> The dataset contains two directories at the root level: <em><strong>raw_dataset</strong></em> <strong><em>processed_dataset</em></strong> Each folder in the <strong><em>raw_dataset</em> </strong>directory contains the raw monitoring data used to generate the graph associated to a single experiment together with additional metadata files.<br> Each folder in the <strong><em>processed_dataset</em> </strong>directory contains the graph associated to a single experiment as a set of three CSV files: two for the graph edges (<em>pid_childof_pid_df.csv</em> and <em>pid_speakswith_pid_df.csv</em>) and one for the graph nodes (<em>proc_df.csv</em>).<br> We provide below a code snippet to parse a graph from <strong><em>processed_dataset</em> </strong>directory. In both folders the name of each sub-folder is based on the following schema: <strong><em>[SCENARIO]_[W]wl/test_[TEST_ID]</em></strong> where: <em>[SCENARIO]</em> reports the target component for the failure injection (<em>cinder_failure</em>, <em>neutron_failure</em>, <em>nova_failure</em>). <em>ff</em> indicates instead a failure-free execution <em>[W]</em> reports the number of concurrent workloads <em>[TEST_ID] </em>reports the ID of the specific failure scenario injected (same ID selected by the OpenStack failure injection framework [1] ) Each experiment includes the following data in the <strong><em>raw_dataset</em></strong> sub-folders: <em>audit_raw_logs_[TEST_ID]/</em>: raw audit monitoring data <em>bpf_tools_[TEST_ID]/</em>: raw ebpf tools monitoring data <em>instance-[INSTANCE_ID]/</em>: workload-specific metadata files, e.g. stdout/stderr (generated by the OpenStack failure injection framework [1] ) <em>logs_workload_[TEST_ID]/:</em> OpenStack application logs <em>perf_tools_[TEST_ID]/</em>: raw perf tools monitoring data <em>audit_filtered_[TEST_ID].log:</em> audit data pre-processed by <em>ausearch</em> (e.g. numerical entities are resolved to symbols) <em>failure_[TEST_ID].info</em>: metadata information about the specific failure scenario (generated by the OpenStack failure injection framework [1] ) <em>timestamps_[TEST_ID]:</em> timing information <em>[1] D. Cotroneo, L. De Simone, P. Liguori, R. Natella, N. Bidokhti - How Bad Can a Bug Get? An Empirical Analysis of Software Failures in the OpenStack Cloud Computing Platform [ACM ESEC/FSE 2019]</em> Example: parsing a graph from <strong><em>processed_dataset</em> </strong>directory <pre><code class="language-python">import pandas as pd import networkx as nx def parse_csv(path): processes_df = pd.read_csv('%sproc_df.csv' % path, index_col=0).reset_index(drop=True) speakswith_edges_df = pd.read_csv('%spid_speakswith_pid_df.csv' % path, index_col=0) speakswith_edges_df['type'] = 'speaksWith' childof_edges_df = pd.read_csv('%spid_childof_pid_df.csv' % path, index_col=0) childof_edges_df['type'] = 'childOf' return processes_df, pd.concat([speakswith_edges_df, childof_edges_df], ignore_index=True) def make_graph(nodes_df, edges_df): G = nx.MultiGraph() for _, node in nodes_df.iterrows(): G.add_node(node.pid, **node) for _, edge in edges_df.iterrows(): G.add_edge(edge.pid1, edge.pid2, type=edge.type) return G PATH = 'processed_dataset/ff_1wl/test_1/' nodes_df, edges_df = parse_csv(PATH) G = make_graph(nodes_df, edges_df) nx.draw_networkx(G, node_size=10, with_labels=False)</code></pre> If you use this dataset for your research, please cite the following paper: <pre><code>@inproceedings{sanvito2022syslrn, title={syslrn: Learning What to Monitor for Efficient Anomaly Detection}, author={Sanvito, Davide and Siracusano, Giuseppe and Santhanam, Sharan and Gonzalez, Roberto and Bifulco, Roberto}, booktitle={2nd European Workshop on Machine Learning and Systems (EuroMLSys '22)}, year={2022}, address = {Rennes, France}, publisher = {ACM}, month = apr, } </code></pre>
description: {"references": ["D. Sanvito, G. Siracusano, S. Santhanam, R. Gonzalez, R. Bifulco - syslrn: Learning What to Monitor for Efficient Anomaly Detection [ACM EuroMLSys 2022]"]}