#  1 Introduction

The Commission is running a flexible pilot under Horizon 2020 called the Open
Research Data (ORD) Pilot. The ORD pilot aims to improve and maximize access
to and re-use of research data generated by Horizon 2020 projects and
considers the need to balance openness and protection of scientific
information, commercialization and Intellectual Property Rights (IPR), privacy
concerns, security, as well as data management and preservation aspects.

As a participating project, DiSIEM is required to develop a Data Management
Plan (DMP), identified as deliverable D8.2. The DMP is a key element of good
data management, describing the data management life cycle for the data to be
collected, processed and/or generated. The goal is to make research data
findable, accessible, interoperable and re-usable (FAIR).

All partners have contributed to the document, completing a project-wide
questionnaire that was then used to determine each partner’s role in creating
and/or processing data.

##  1.1 Organization of the Document

Since each partner will generate and/or manipulate data, the document is
organized with one section per partner (Sections 3-9). Each of these sections
is structured in five subsections:

  1. **Dataset description** contains a textual description of the dataset. It aims at explaining, in a short paragraph, what the dataset contains and what its goal is; 

  

  2. **Standards and metadata** focuses on explaining the internals of the dataset, namely how a user can find syntactical and semantic information; 

  

  3. **Data sharing** addresses the issues related to data access, and privacy concerns, namely if the dataset is going to be indexed, and how and to whom it will be made accessible; 

  

  4. **Archiving and presentation** covers the aspects related to data availability, during and beyond the project, as well as the actions taken and planned to support availability; 

  

  5. **Data details** goes into the specifics of each partner’s dataset, describing its content. 

  

  

Besides these per-partner sections, the document also contains a general
description of our overall methodology in terms of data collection and sharing
in Section 2. The summary and conclusions of the Data Management Plan are in
Section 10. In the appendix, we included the questionnaire each partner filled
to prepare the document.

#  2 Methodology

In this section, we explain some general policy we defined to store and share
the data sets produced during the project and the overall methodology used for
producing this document.

##  2.1 DiSIEM Policy for Storage and Sharing of Datasets

One of the most important aspects of the methodology is how datasets are to be
stored and used during the project.

A first general concern is how the produced datasets are to be stored. The
consortium decided to do that in three ways, for different types of datasets:

  * For the public datasets, i.e., the ones we can share outside the consortium, we plan to publish them on the project webpage (or in another public repository to be referred by the project webpage). 

  

  * For controlled datasets, i.e., the ones that will be anonymized and shared within the consortium for enabling partners to do exploratory studies, we created a special directory in the project repository for storing them. The idea is to have a subdirectory for each dataset containing not only the dataset files but also a _info.txt_ text file with a brief description and metadata of the dataset. 

  

  * For privacy-sensitive datasets, i.e., those that contain critical information from partners and therefore require special care in sharing, we decided that partners need to agree on the specifics of how sharing can be done. This might include the signing of specific agreements and protocols between the involved partners. In any case, this should be done between partners, without any direct influence from the consortium. 

  

Regarding the storage of controlled datasets, they will be kept in our project
repository, which is maintained in a dedicated KVM virtual machine hosted by
FCUL. This VM can only be directly accessed by DI-FCUL system administrators
and is externally visible only through the gitlab web interface and through
the git protocol over SSL/TLS. All accesses require authentication using valid
credentials and access control is enforced. Therefore, we believe an adequate
level of protection is provided for these datasets.

As will be clear in the next sections, the preferred formats for datasets are
CSV (Comma Separated Values, as specified in RFC4180 [1]) and JSON, since both
are text-based and easily parsed by any tool or service being used within the
project.

##  2.2 Data Collection Methodology

To compile the data management plan, a questionnaire was first elaborated
covering the main questions that need to be answered in the template provided
by the European Commission [2].

In the second phase, each project partner responded to the questionnaire,
filling it with as much detail as possible at this stage of the project.
Completed questionnaires were stored for analysis and traceability in the
project’s git repository.

In the third phase, the Data Management Plan was created as a synthesis of the
questionnaire results, attempting to take advantage of commonalities between
responses to provide a simple view of data management procedures within the
consortium.

Further revisions of the document will be based on updates to partner
questionnaires. Therefore, the DMP will be updated at least by the mid-term
and final reports to be able to accommodate any new data forms and
requirements that cannot be estimated in this current stage of the project.

#  3 Dataset FFCUL

FFCUL is an academic partner in the project therefore it is not expected to
contribute with datasets about monitored infrastructures. However, it plans to
contribute with some OSINT datasets that might be useful for evaluating the
tools and techniques proposed for processing such kinds of data.

##  3.1 Dataset Description

In principle, FFCUL will provide a collection of tweets classified as
“relevant or not” for a given reference infrastructure, a list of operating
systems vulnerabilities collected from NVD and enriched with information from
other databases, and a list of compromised IP addresses collected from several
security feeds on the Internet.

##  3.2 Standards and metadata

The dataset will contain data formatted using the common Comma-Separated
Values (CSV) standard.

##  3.3 Data Sharing

Since all these datasets are being collected from public feeds from the
Internet, FFCUL intends to make them publicly available, respecting possible
data protection legislation.

##  3.4 Archiving and presentation

The dataset will be made available as companion papers exploring them are
published. The idea is to have papers using the datasets for validating tools
built within the project. Once the papers are made public, the datasets will
be made available either through the project webpage or through DI-FCUL
webpage.

##  3.5 Data details

FFCUL will provide three different types of OSINT datasets that can be used to
validate different DiSIEM innovations:

  * A collection of tweets gathered from 80 cybersecurity-related accounts such as sans_isc, e_kaspersky, alienvault, vuln_lab, etc. These tweets will be manually classified as relevant or not to some synthetic organization infrastructure; 

  

  * A list of operating systems vulnerabilities collected from NVD and enriched with information about exploits and patches obtained from other vulnerability databases such as ExploitDB and OSVDB; 

  

  * A list of compromised IPs collected from more than a hundred security feeds organized by published date and source. 

  

Notice that “the operating system vulnerabilities” dataset is somewhat similar
to the data offered by the vepRisk tool from City (see next section). In the
future, we will try to integrate these datasets to avoid duplicating efforts.

#  4 Dataset CITY

City, being an Academic partner in the project, will be primarily a data
consumer rather than a data producer. We plan to analyse the data provided by
the project partners to evaluate and test our extensions and plug-ins for
diversity and data visualisation.

##  4.1 Dataset Description

We do plan to also deploy our own testbed to evaluate and test the extensions
we build for diversity and probabilistic modelling. The data will consist of
synthetically generated network data, as well as data collected from a
University honeypot.

We are also building a tool that gathers public data on vulnerabilities,
patches and exploits. The tool is made available from the following site (the
URL may be updated and change in the future):
_http://veprisk.city.ac.uk/sampleapps/vepRisk/_

##  4.2 Standards and metadata

The data from our testbed will consist of network traffic, in the _pcap_
format, as well as the alerts of the Intrusion Detection Systems (IDS) we will
test: Snort, Suricata and Bro. These will be generated in the respective alert
format of the tool vendors.

The data from the vepRisk tool can be downloaded from the site in CSV format.

##  4.3 Data Sharing

Synthetic data from our testbed will be shared with DiSIEM partners without
restriction. Data from honeynets, would need to be anonymized first to remove
sensitive, confidential and/or private information. Data from vepRisk is
available from the public page of the tool.

##  4.4 Archiving and presentation

The dataset will be disseminated to the consortium via the Git repository.

##  4.5 Data details

For the vepRisk tool, the data is taken from the public databases on
vulnerabilities, patches and exploits and the information on these data are
available from the repositories where this data is collected namely, NVD  1  ,
Exploitdb  2  and various patch databases (e.g. Microsoft  3  , Ubuntu  4
etc.)

Regarding our testbed, we expect the data will include network flows (source

and destination IP addresses, source and destination ports, network protocol,
timestamp etc.) and the alerts from the IDS platforms.

#  5 Dataset EDP

##  5.1 Dataset Description

Having an operating SIEM platform that receives over 10.000 events per second,
EDP – Energias de Portugal, SA. has the capability to provide realistic and
meaningful data for analysis. The dataset will consist of a significant subset
of real events, comprising data from multiple and diverse sources, after
adequate pre-processing to ensure that no confidential information is
wrongfully distributed.

##  5.2 Standards and metadata

The dataset will contain data formatted using the common Comma-Separated
Values (CSV) standard, as specified in RFC4180 [1].

##  5.3 Data Sharing

EDP will make data available for the project partners. The specific
information to be shared depends on the need presented by the partners, as
well as a risk assessment to guarantee legal and business policy compliance.
The final dataset details will be indicated in a later release of the DMP.

Information retrieved from EDP’s SIEM platform should not be made publicly
available due to the critical nature of the data and user privacy concerns.

EDP is investigating tools to enable data masking and/or anonymization. We
identified and started performing tests with two of such tools: Python Faker (
_http://blog.districtdatalabs.com/a-practical-guide-to-anonymizing-datasets-
withpython-faker)_ and ARX ( _http://arx.deidentifier.org/)_ .

##  5.4 Archiving and presentation

The dataset will be disseminated to the consortium via the official Git
repository.

##  5.5 Data details

The most relevant SIEM events collected in EDP’s platform, with a summary of
the respective field set, are presented in the following table.  
  
<table>  
<tr>  
<th>

</th>  
<th>

</th>  
<th>

</th>  
<th>

**Event source**

</th>  
<th>

</th>  
<th>

</th> </tr>  
<tr>  
<td>

**Field**

</td>  
<td>

Firewall

</td>  
<td>

IPS

</td>  
<td>

User authentication

</td>  
<td>

VPN

access

</td>  
<td>

Server access

</td>  
<td>

Antivirus

</td> </tr>  
<tr>  
<td>

Event name

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td> </tr>  
<tr>  
<td>

Source username

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td> </tr>  
<tr>  
<td>

Source address

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td> </tr>  
<tr>  
<td>

Source port

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td> </tr>  
<tr>  
<td>

Source geo country

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

X

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

X

</td> </tr>  
<tr>  
<td>

Destination username

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td> </tr>  
<tr>  
<td>

Destination address

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

√

</td> </tr>  
<tr>  
<td>

Destination port

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td> </tr>  
<tr>  
<td>

Destination geo country

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

X

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

X

</td> </tr>  
<tr>  
<td>

Application protocol

</td>  
<td>

√

</td>  
<td>

√

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td> </tr>  
<tr>  
<td>

File name

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

√

</td> </tr>  
<tr>  
<td>

Policy name

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

X

</td>  
<td>

√

</td> </tr> </table>

**Table 1 – Data details (EDP)**

Field format:

Event name: String (255-character limit);

Source username: String (255-character limit);

Source address: IP Address (IPv4);

Source geo country: String (255-character limit);

Destination username: String (255-character limit);

Destination address: IP Address (IPv4);

Destination port: Integer from 1 to 65535

Destination geo country: String (255-character limit);

Application protocol: String (255-character limit); File name: String
(255-character limit);

Policy name: String (255-character limit).

#  6 Dataset AMADEUS

##  6.1 Dataset Description

Amadeus can provide real datasets from different log sources: applications,
Firewalls, OS syslog, Antiviruses, Proxy, VPN, IDS, DNS, etc. We need to
preprocess and anonymise the data before sharing it with partners.

##  6.2 Standards and metadata

Two data format will be used for the shared datasets: 1. Comma-Separated
Values (CSV);

2\. JSON.

A documentation will be provided with each type of dataset to be shared with
the partners.

##  6.3 Data Sharing

Amadeus datasets will be shared with DiSIEM partners depending on the needs
presented. However, partners need to ensure that shared datasets should not be
made publicly available in any case, due to legal and business policy
restrictions.

##  6.4 Archiving and presentation

The dataset will be disseminated to the consortium via the official Git
repository, or any secure file sharing method (in the case of privacy-
sensitive data).

##  6.5 Data details

A summary of the datasets to be shared with DiSIEM partners can be found in
the table below:  
  
<table>  
<tr>  
<th>

**Source**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

LSS ASM logs

</td>  
<td>

An administration tool for an authentication and access control management
application

</td> </tr>  
<tr>  
<td>

HTTP access logs

</td>  
<td>

HTTP logs from an e-commerce application

</td> </tr>  
<tr>  
<td>

Cisco, Palo Alto Network

</td>  
<td>

Firewall logs

</td> </tr>  
<tr>  
<td>

McAfee

</td>  
<td>

Antivirus

</td> </tr>  
<tr>  
<td>

Suricata, Palo Alto, Bro

</td>  
<td>

IDS

</td> </tr>  
<tr>  
<td>

Cisco VPN

</td>  
<td>

VPN

</td> </tr> </table>

**Table 2 – Data details (AMADEUS)**

The next sections provide a description of the data fields for each dataset.

####  6.5.1 LSS ASM logs

The logs of an administration tool for an authentication and access control
management application. The dataset to be provided is a set of user actions. A
user session is a set of user actions with the same session id (PFX, see table
below):  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

PFX

</td>  
<td>

Session id

</td> </tr>  
<tr>  
<td>

Orga

</td>  
<td>

Organisation

</td> </tr>  
<tr>  
<td>

Action

</td>  
<td>

Type of action performed

</td> </tr>  
<tr>  
<td>

userId

</td>  
<td>

User issuing the action

</td> </tr>  
<tr>  
<td>

officeId

</td>  
<td>

Office from which the user is connecting

</td> </tr>  
<tr>  
<td>

Country

</td>  
<td>

Country Code

</td> </tr>  
<tr>  
<td>

IP

</td>  
<td>

IP address

</td> </tr>  
<tr>  
<td>

*Browser 
</td>  
<td>

Client browser used

</td> </tr>  
<tr>  
<td>

*browserEngine 
</td>  
<td>

Client browser Engine

</td> </tr>  
<tr>  
<td>

*OS 
</td>  
<td>

Client operating system

</td> </tr> </table>

*These fields are derived from the useragent string. 

**Table 3 – LSS ASM logs (AMADEUS)**

####  6.5.2 HTTP access logs

This dataset will be extracted from a web server of an e-commerce application.
The fields are the default HTTP request fields with some additional nested
fields extracted from the IP address and the useragent string. More details in
the table below:  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Datetime

</td>  
<td>

Timestamp

</td> </tr>  
<tr>  
<td>

Method

</td>  
<td>

HTTP method

</td> </tr>  
<tr>  
<td>

Urlpath

</td>  
<td>

URI path

</td> </tr>  
<tr>  
<td>

Status

</td>  
<td>

HTTP status code

</td> </tr>  
<tr>  
<td>

http_referrer

</td>  
<td>

HTTP referrer

</td> </tr>  
<tr>  
<td>

Useragent

</td>  
<td>

Useragent String

</td> </tr>  
<tr>  
<td>

Accespt_language

</td>  
<td>

Accept Language in the HTTP header

</td> </tr>  
<tr>  
<td>

Duration

</td>  
<td>

Request processing time

</td> </tr>  
<tr>  
<td>

Hostname

</td>  
<td>

Target HTTP hostname

</td> </tr>  
<tr>  
<td>

Referrer_uri_proto

</td>  
<td>

Referrer URI protocol

</td> </tr>  
<tr>  
<td>

Referrer_hostname

</td>  
<td>

Referrer Hostname

</td> </tr>  
<tr>  
<td>

Referrer_uri_path

</td>  
<td>

Referrer URI path

</td> </tr>  
<tr>  
<td>

Referrer_params

</td>  
<td>

Referrer Parameters

</td> </tr>  
<tr>  
<td>

Ua

</td>  
<td>

Nested Useragent object

</td> </tr>  
<tr>  
<td>

remoteclientipaddress

</td>  
<td>

End User or CDN IP address

</td> </tr>  
<tr>  
<td>

client_ip

</td>  
<td>

Private IP address of HTTP server

</td> </tr>  
<tr>  
<td>

Geoip

</td>  
<td>

Nested Geo coordinates object

</td> </tr>  
<tr>  
<td>

isp

</td>  
<td>

Nested ISP object

</td> </tr>  
<tr>  
<td>

edge_proxy_cip

</td>  
<td>

End User or CDN IP address

</td> </tr>  
<tr>  
<td>

x_forwarded_for

</td>  
<td>

End User or CDN IP address

</td> </tr>  
<tr>  
<td>

Jsessionid

</td>  
<td>

The session id of a given request

</td> </tr> </table>

**Table 4 – HTTP access logs (AMADEUS)**

####  6.5.3 Suricata IDS

This dataset is extracted from the Open source IDS/NSM engine Suricata. A
brief description of the most relevant fields is provided in the table below:  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Category

</td>  
<td>

Threat category

</td> </tr>  
<tr>  
<td>

Dest

</td>  
<td>

Destination IP address

</td> </tr>  
<tr>  
<td>

Severity

</td>  
<td>

Threat severity

</td> </tr>  
<tr>  
<td>

Signature

</td>  
<td>

Threat Signature

</td> </tr>  
<tr>  
<td>

Src

</td>  
<td>

Source IP address

</td> </tr>  
<tr>  
<td>

Answer

</td>  
<td>

DNS server answer

</td> </tr>  
<tr>  
<td>

Date

</td>  
<td>

Timestamp

</td> </tr>  
<tr>  
<td>

Dest_nt_host

</td>  
<td>

Destination IP organization

</td> </tr>  
<tr>  
<td>

Dest_port

</td>  
<td>

Destination port number

</td> </tr>  
<tr>  
<td>

Dns

</td>  
<td>

Nested DNS response object

</td> </tr>  
<tr>  
<td>

http

</td>  
<td>

Nested HTTP request object

</td> </tr>  
<tr>  
<td>

Eventtype

</td>  
<td>

Suricata event type

</td> </tr>  
<tr>  
<td>

Message_type

</td>  
<td>

Request/Reply

</td> </tr>  
<tr>  
<td>

Proto

</td>  
<td>

Transport Layer Protocol

</td> </tr>  
<tr>  
<td>

Src_nt_host

</td>  
<td>

Same as Dest_nt_host

</td> </tr>  
<tr>  
<td>

Ssl_issuer_common_name

</td>  
<td>

SSL certificate issuer name

</td> </tr>  
<tr>  
<td>

Ssl_issuer_organization

</td>  
<td>

SSL certificate issuer organization

</td> </tr>  
<tr>  
<td>

Ssl_publickkey

</td>  
<td>

SSL certificate public key

</td> </tr>  
<tr>  
<td>

Ssl_subject_common_name

</td>  
<td>

SSL subject name

</td> </tr>  
<tr>  
<td>

SSL_subject_organization

</td>  
<td>

SSL subject organization

</td> </tr>  
<tr>  
<td>

Ssl_version

</td>  
<td>

SSL/TLS version

</td> </tr>  
<tr>  
<td>

TLS

</td>  
<td>

Nested TLS requests object

</td> </tr> </table>

**Table 5 – Suricata IDS (AMADEUS)**

####  6.5.4 Cisco Firewall logs

Within the context of a security incident, administrators can use cisco syslog
messages to understand communication relationships, timing, and, in some
cases, the attacker's motives and/or tools.  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

acl

</td>  
<td>

Access control list

</td> </tr>  
<tr>  
<td>

Action

</td>  
<td>

The status of the actions (e.g. allowed, blocked etc.)

</td> </tr>  
<tr>  
<td>

Cisco_ASA_action

</td>  
<td>

The status of the Cisco Adaptive Security Appliance (e.g. allowed, blocked
etc.)

</td> </tr>  
<tr>  
<td>

Cisco_ASA_message_id

</td>  
<td>

The id of the Cisco message

</td> </tr>  
<tr>  
<td>

Description

</td>  
<td>

The Description of the firewall event

</td> </tr>  
<tr>  
<td>

Dest_category

</td>  
<td>

The category destination of the event

</td> </tr>  
<tr>  
<td>

Dest_dns

</td>  
<td>

Destination DNS

</td> </tr>  
<tr>  
<td>

Dest_mac

</td>  
<td>

The physical address of the mac destination

</td> </tr>  
<tr>  
<td>

Dest_nt_host

</td>  
<td>

Destination network host

</td> </tr>  
<tr>  
<td>

Dest_port

</td>  
<td>

The port destination

</td> </tr>  
<tr>  
<td>

Dest_zone

</td>  
<td>

The server destination of the event

</td> </tr>  
<tr>  
<td>

Eventtype

</td>  
<td>

The type of the event

</td> </tr>  
<tr>  
<td>

Group

</td>  
<td>

The group of servers

</td> </tr>  
<tr>  
<td>

Message_id

</td>  
<td>

the ID of the message

</td> </tr>  
<tr>  
<td>

Rule_name

</td>  
<td>

The name of the rule

</td> </tr>  
<tr>  
<td>

Severity_level

</td>  
<td>

The severity level of the rule

</td> </tr> </table>

**Table 6 – Cisco firewall logs (AMADEUS)**

####  6.5.5 Next-Generation Firewall – Palo Alto Networks (PAN)

This next-generation firewall classifies all traffic, including encrypted
traffic, based on application, application function, user and content.  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Action

</td>  
<td>

The action taken by the IDS

</td> </tr>  
<tr>  
<td>

Application

</td>  
<td>

The application on which the alert was raised

</td> </tr>  
<tr>  
<td>

Client_ip

</td>  
<td>

The IP of the client

</td> </tr>  
<tr>  
<td>

Client_location

</td>  
<td>

The location of the client

</td> </tr>  
<tr>  
<td>

Date

</td>  
<td>

Timestamp

</td> </tr>  
<tr>  
<td>

Dest_asset_id

</td>  
<td>

The asset destination ID

</td> </tr>  
<tr>  
<td>

Dest_dns

</td>  
<td>

The dns of the destination

</td> </tr>  
<tr>  
<td>

Dest_interface

</td>  
<td>

The destination network interface

</td> </tr>  
<tr>  
<td>

Dest_ip

</td>  
<td>

The IP of the destination

</td> </tr>  
<tr>  
<td>

Dest_zone

</td>  
<td>

The zone of the destination

</td> </tr>  
<tr>  
<td>

Dest_nt_host

</td>  
<td>

Destination network host

</td> </tr>  
<tr>  
<td>

Eventtype

</td>  
<td>

The type of event (e.g. allowed, blocked etc.)

</td> </tr>  
<tr>  
<td>

dstPort

</td>  
<td>

Destination port

</td> </tr>  
<tr>  
<td>

Protocol

</td>  
<td>

The communication protocol being used

</td> </tr>  
<tr>  
<td>

RuleName

</td>  
<td>

The name of the rule

</td> </tr>  
<tr>  
<td>

Server_IP

</td>  
<td>

The IP of the server

</td> </tr> </table>

**Table 7 – Palo Alto Networks (AMADEUS)**

####  6.5.6 Palo Alto IDS

This dataset is also extracted from Palo Alto Networks next-generation
firewalls. It contains the events tagged as threats. A description of the most
relevant fields is provided below:  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Action

</td>  
<td>

Action taken by the IDS

</td> </tr>  
<tr>  
<td>

Application

</td>  
<td>

The application that raised the alert

</td> </tr>  
<tr>  
<td>

Category

</td>  
<td>

Category of the intrusion

</td> </tr>  
<tr>  
<td>

Client_ip

</td>  
<td>

Client local IP address

</td> </tr>  
<tr>  
<td>

Client_location

</td>  
<td>

Location of the client in the network

</td> </tr>  
<tr>  
<td>

Date

</td>  
<td>

timestamp

</td> </tr>  
<tr>  
<td>

Dest_ip

</td>  
<td>

Destination IP address

</td> </tr>  
<tr>  
<td>

Dest_hostname

</td>  
<td>

Destination hostname

</td> </tr>  
<tr>  
<td>

Dest_interface

</td>  
<td>

Destination network interface

</td> </tr>  
<tr>  
<td>

Dest_nt_host

</td>  
<td>

Destination IP organization

</td> </tr>  
<tr>  
<td>

Dest_port

</td>  
<td>

Destination Port number

</td> </tr>  
<tr>  
<td>

DestinationZone

</td>  
<td>

Destination network zone

</td> </tr>  
<tr>  
<td>

IngressInterface

</td>  
<td>

Ingress network interface

</td> </tr>  
<tr>  
<td>

Proto

</td>  
<td>

Transport Layer protocol

</td> </tr>  
<tr>  
<td>

Session_id

</td>  
<td>

Communication session id

</td> </tr>  
<tr>  
<td>

Severity

</td>  
<td>

Severity level (1 to 5)

</td> </tr>  
<tr>  
<td>

Signature

</td>  
<td>

Vulnerability signature

</td> </tr>  
<tr>  
<td>

SourceUser

</td>  
<td>

Source Username

</td> </tr>  
<tr>  
<td>

Src_bunit

</td>  
<td>

Source user business unit

</td> </tr>  
<tr>  
<td>

Src_category

</td>  
<td>

Source category

</td> </tr>  
<tr>  
<td>

Src_dns

</td>  
<td>

Source DNS server name

</td> </tr>  
<tr>  
<td>

Src_mac

</td>  
<td>

Source MAC address

</td> </tr>  
<tr>  
<td>

Src_nt_host

</td>  
<td>

Source IP Organization

</td> </tr>  
<tr>  
<td>

Src_owner

</td>  
<td>

Source IP Owner Name

</td> </tr>  
<tr>  
<td>

Src_port

</td>  
<td>

Source Port Number

</td> </tr>  
<tr>  
<td>

Src_zone

</td>  
<td>

Source IP network zone

</td> </tr>  
<tr>  
<td>

Threat:category

</td>  
<td>

Threat category

</td> </tr>  
<tr>  
<td>

Threat:name

</td>  
<td>

Threat Name

</td> </tr>  
<tr>  
<td>

User

</td>  
<td>

Username

</td> </tr>  
<tr>  
<td>

User_watchlist

</td>  
<td>

Boolean, true if User in watch list

</td> </tr> </table>

**Table 8 – Palo Alto IDS (AMADEUS)**

####  6.5.7 McAfee ePO

McAfee ePolicy Orchestrator, a centralized security management software for
antiviruses, is the source of this dataset.  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Action

</td>  
<td>

Action taken by McAfee Antivirus

</td> </tr>  
<tr>  
<td>

Category

</td>  
<td>

Threat category

</td> </tr>  
<tr>  
<td>

Date

</td>  
<td>

Date

</td> </tr>  
<tr>  
<td>

Dest

</td>  
<td>

Office ID

</td> </tr>  
<tr>  
<td>

Dest_bunit

</td>  
<td>

Destination business unit

</td> </tr>  
<tr>  
<td>

Dest_ip

</td>  
<td>

Destination IP address

</td> </tr>  
<tr>  
<td>

Dest_mac

</td>  
<td>

Destination MAC address

</td> </tr>  
<tr>  
<td>

Dest_nt_domain

</td>  
<td>

Destination IP network domain

</td> </tr>  
<tr>  
<td>

Dest_nt_host

</td>  
<td>

Destination hostname

</td> </tr>  
<tr>  
<td>

Dest_owner

</td>  
<td>

Destination User name

</td> </tr>  
<tr>  
<td>

Detection_method

</td>  
<td>

Firewall detection method

</td> </tr>  
<tr>  
<td>

Devent_description

</td>  
<td>

Firewall event description

</td> </tr>  
<tr>  
<td>

File_name

</td>  
<td>

Suspicious filename

</td> </tr>  
<tr>  
<td>

Fqdn

</td>  
<td>

Fully Qualified domain name

</td> </tr>  
<tr>  
<td>

Is_laptop

</td>  
<td>

Boolean, 1 if Laptop used

</td> </tr>  
<tr>  
<td>

Logon_user

</td>  
<td>

Username

</td> </tr>  
<tr>  
<td>

Mcafee_epo_os

</td>  
<td>

OS name

</td> </tr>  
<tr>  
<td>

Os_build

</td>  
<td>

OS build number

</td> </tr>  
<tr>  
<td>

Os_version

</td>  
<td>

OS version

</td> </tr>  
<tr>  
<td>

Process

</td>  
<td>

Process name

</td> </tr>  
<tr>  
<td>

Product

</td>  
<td>

Component creating the event

</td> </tr>  
<tr>  
<td>

Severity

</td>  
<td>

Threat severity level

</td> </tr>  
<tr>  
<td>

Severity_id

</td>  
<td>

A number mapped to severity

</td> </tr>  
<tr>  
<td>

Src

</td>  
<td>

Source IP address

</td> </tr>  
<tr>  
<td>

Src_bunit

</td>  
<td>

Source IP business unit

</td> </tr>  
<tr>  
<td>

Src_category

</td>  
<td>

Source IP category

</td> </tr>  
<tr>  
<td>

Src_mac

</td>  
<td>

Source MAC address

</td> </tr>  
<tr>  
<td>

Src_nt_host

</td>  
<td>

Source IP network zone

</td> </tr>  
<tr>  
<td>

Src_owner

</td>  
<td>

Source IP owner name

</td> </tr>  
<tr>  
<td>

Src_priority

</td>  
<td>

Same as dest_priority

</td> </tr>  
<tr>  
<td>

Threat_handled

</td>  
<td>

Boolean for whether threat is handled

</td> </tr>  
<tr>  
<td>

Threat_type

</td>  
<td>

Threat Type

</td> </tr>  
<tr>  
<td>

User_email

</td>  
<td>

User email address

</td> </tr> </table>

**Table 9 – McAfee ePO (AMADEUS)**

####  6.5.8 Bro IDS

This dataset is extracted from Bro, an open source network analysis framework.
Below is a description of the Bro events fields.  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Body

</td>  
<td>

Threat description

</td> </tr>  
<tr>  
<td>

Category

</td>  
<td>

Threat category

</td> </tr>  
<tr>  
<td>

Date

</td>  
<td>

Timestamp

</td> </tr>  
<tr>  
<td>

Dest

</td>  
<td>

Destination IP address

</td> </tr>  
<tr>  
<td>

Dest_nt_host

</td>  
<td>

Destination IP network zone

</td> </tr>  
<tr>  
<td>

Dest_port

</td>  
<td>

Destination port number

</td> </tr>  
<tr>  
<td>

Eventtype

</td>  
<td>

Bro event type

</td> </tr>  
<tr>  
<td>

File_desc

</td>  
<td>

Suspicious file

</td> </tr>  
<tr>  
<td>

O

</td>  
<td>

Organization

</td> </tr>  
<tr>  
<td>

Src

</td>  
<td>

Source IP address

</td> </tr>  
<tr>  
<td>

Src_nt_host

</td>  
<td>

Same as dest_nt_host

</td> </tr>  
<tr>  
<td>

Src_port

</td>  
<td>

Source Port number

</td> </tr>  
<tr>  
<td>

Tag::eventtype

</td>  
<td>

Event type

</td> </tr>  
<tr>  
<td>

Uid

</td>  
<td>

User ID

</td> </tr> </table>

**Table 10 – Bro IDS (AMADEUS)**

####  6.5.9 Cisco VPN

This dataset contains events from a Cisco VPN server. A description of the
dataset fields is in the summary below.  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Assigned_ip

</td>  
<td>

Private IP assigned to the user session

</td> </tr>  
<tr>  
<td>

Cisco_ASA_user

</td>  
<td>

Username

</td> </tr>  
<tr>  
<td>

Date

</td>  
<td>

Timestamp

</td> </tr>  
<tr>  
<td>

Duration

</td>  
<td>

VPN session duration in seconds

</td> </tr>  
<tr>  
<td>

Eventtype

</td>  
<td>

Cisco VPN event type

</td> </tr>  
<tr>  
<td>

Group

</td>  
<td>

Remote Access Group

</td> </tr>  
<tr>  
<td>

IP

</td>  
<td>

User Public IP address

</td> </tr>  
<tr>  
<td>

Reason

</td>  
<td>

Connection Lost reason

</td> </tr>  
<tr>  
<td>

User_email

</td>  
<td>

User email address

</td> </tr>  
<tr>  
<td>

User_identity

</td>  
<td>

Full username

</td> </tr>  
<tr>  
<td>

Username

</td>  
<td>

Username

</td> </tr> </table>

**Table 11 – Cisco VPN (AMADEUS)**

#  7 Dataset DigitalMR

DigitalMR works with OSINT and has infrastructure to fetch information to
create datasets. We intend to fetch information from security related blogs
and tweets for a specific timeline of interest. These datasets will be
available during the project.

##  7.1 Dataset Description

Our data consists of openly available content on the Internet from sources
including blogs, forums, news, and social networks like Twitter, Instagram and
Facebook. This data is either scraped from the sources using our specially
built crawlers or fetched using the built-in API of the data sources such as
the ones provided by Twitter and Facebook.

##  7.2 Standards and metadata

The format of the data is in JSON which is widely supported by several
applications and is semi-structured. The size of the data can be up to 5
million posts on the Internet depending on the scope of the project.

##  7.3 Data Sharing

Given that the content of the data might contain information such as
usernames, and privacy laws might vary between countries; it is the
responsibility of the user of the dataset to make sure that the applicable
legislations are respected.

##  7.4 Archiving and presentation

The dataset will be shared to the consortium via the official Git repository
in JSON and will be available for use by the partners.

##  7.5 Data details

Some of the common fields in the data include the following:

  * Author Username 

  

  * Author profile URL 

  

  * Post URL 

  

  * Parent tweet URL (for twitter content) 

  

  * Location 

  

  * Content/Post (actual content of the data) 

  

  * Date 

  

  * Tags (added by DigitalMR) 

  

  * Relevance (added by DigitalMR) 

  

  * Sentiment (added by DigitalMR) 

  

#  8 Dataset FRAUNHOFER

Fraunhofer does not plan to produce any dataset during DiSIEM. Instead, data
provided from the project partners will be analysed using machine learning and
visual analytics methods. This may lead to the development of novel
representations of the event data produced by the SIEM platforms, as well as
the discovery of user- or session-clusters. These results can be used to
develop novel visualization tools for SIEM data.

To represent event sequences, Fraunhofer evaluates the embedding, including
the bag-of-words approach, event occurrence frequencies within a given
sequence and the TF-IDF-score (term frequency multiplied with the inverse
document frequency) of events with respect to a given sequence database.
Another approach is to define a similarity measure for sequences. To that
extend, Fraunhofer developed an embedding of event types into a metric space,
where the distance between events correspond to the co-occurrence frequencies
within a given sequence database. These feature representations of event
sequences will be used to embed the data in 2D or 3D for visualization, as
well as to find clusters of sequences and users and to predict whether a
sequence is a potential threat.

#  9 Dataset ATOS

##  9.1 Dataset Description

Atos dataset will be generated in a testbed specifically prepared for DiSIEM.
The dataset will consist of:

  * Events generated by applications or sensors installed in the testbed (e.g. Snort, OSSec, netfilter, JBoss, linux kernel, etc), once normalized to the event format used by the XL-SIEM component; 

  

  * Alarms generated by XL-SIEM component. 

  

OSINT data or IoC from external feeds such as AlienVault Open Threat Exchange
( _OTX_ )  5  could also be used by XL-SIEM in Atos testbed.

Since data will be generated in the testbed, no confidential information will
be provided in the dataset.

##  9.2 Standards and metadata

Currently, data generated in Atos testbed can be provided in two formats:

  * Comma-Separated Values (CSV); 

  

  * JSON. 

  

No documentation or metadata is provided currently with the dataset. The need
for such additional documentation will be analysed for a later release of the
DMP.

##  9.3 Data Sharing

Atos will make data available for the remaining DiSIEM partners. Information
retrieved from Atos’ SIEM platform should not be made publicly available
without previous authorization. The specific information to be shared depends
on the needs presented by the partners, as well as a risk assessment to
guarantee legal and business policy compliance.

##  9.4 Archiving and presentation

The dataset will be disseminated to the consortium via the official Git
repository. Data generated in Atos testbed can be also shared to DiSIEM
partners using _Advanced Message Queuing Protocol_ (AMQP) protocol such as
RabbitMQ Server.

##  9.5 Data details

SIEM events to be collected in Atos testbed and the final dataset details will
be indicated in a later release of the DMP.

Some event sources to be considered are:

  * Firewall; 

  

  * Server access; 

  

  * Network Intrusion Detection System. 

  

Currently, SIEM events collected (once normalized by the plugins included in
the XL-SIEM agent for each specific data source) have the following fields:  
  
<table>  
<tr>  
<th>

**Field**

</th>  
<th>

**Description**

</th> </tr>  
<tr>  
<td>

Type

</td>  
<td>

Type of plugin: detector or monitor

</td> </tr>  
<tr>  
<td>

Date

</td>  
<td>

Date (timestamp) on which the event is received from the sensor

</td> </tr>  
<tr>  
<td>

Device

</td>  
<td>

IP address of the XL-SIEM agent generating the event in the normalized format

</td> </tr>  
<tr>  
<td>

Plugin_id

</td>  
<td>

Identifier of the data source of event generated

</td> </tr>  
<tr>  
<td>

Plugin_sid

</td>  
<td>

Type of event within the data source specified in plugin_id

</td> </tr>  
<tr>  
<td>

Protocol

</td>  
<td>

Protocol (TCP, UDP, ICMP…)

</td> </tr>  
<tr>  
<td>

Src_ip

</td>  
<td>

IP which the sensor generating the original event identifies as the source of
this event

</td> </tr>  
<tr>  
<td>

Src_port

</td>  
<td>

Source port

</td> </tr>  
<tr>  
<td>

Dst_ip

</td>  
<td>

Ip which the sensor generating the original event identifies as the
destination of this event

</td> </tr>  
<tr>  
<td>

Dst_port

</td>  
<td>

Destination port

</td> </tr>  
<tr>  
<td>

Log

</td>  
<td>

Event data that the specific plugin considers as part of the log and which is
not accommodated in the other fields.

</td> </tr>  
<tr>  
<td>

Data

</td>  
<td>

Raw event's payload, although the plugin may use this field for anything else.

</td> </tr>  
<tr>  
<td>

Userdata1

to

Userdata9

</td>  
<td>

Fields defined in the normalized event format to allocate relevant information
from the specific event's payload. They can contain any alphanumeric
information, and on choosing one or another, the type of display they have in
the event viewer will change.

</td> </tr>  
<tr>  
<td>

Organization

</td>  
<td>

Identify the organization where the agent is deployed.

</td> </tr> </table>

**Table 12 – Data details (Atos)**

#  10 Summary and Conclusions

The Data Management Plan of DiSIEM describes partners’ activity related to
datasets. It contains a summary of all the information available as of
February 28  th  , 2017. All (but one) partners intend to create datasets and
make them available within the consortium.

With respect to _dataset descriptions_ , most of the data manipulated by the
DiSIEM project is related to security events collected from SIEM systems and
processed using various exploratory methods.

With respect to _standards and metadata_ , the most prevalent form of data
format is Comma-Separated Values (CSV), a textual description of data that is
highly common and widely used in the SIEM and big data communities. This
format is very easy to manipulate, particularly adapted to sharing over git
(as text files are easily versioned) and is understood by a wide range of
tools.

With respect to _sharing_ , several partners intend to share the datasets for
further research and publication, at least in the academic community. Academic
research and innovation is the main objective of the data managed in the
DiSIEM project. All partners are aware of data sharing limitations due to
privacy concerns and legal obligations. When necessary, information will be
anonymized or truncated in compliance with the applicable legislation.

With respect to _archiving and presentation_ , partners plan to use internal
resources and have them available at the time of writing.

Since it is very early in the project, this document only presents preliminary
proposals in terms of sharing, volume and archiving. The project is aware of
these aspects and will tackle them by updating the present document during the
development of the specifications of the experimentations. Therefore,
information in this document is subject to change.