deardaniel/secdata-raw huggingface dataset:
OWASP Threat Modeling: README

## OWASP Threat Dragon Site

This is a Vue project that serves as the front end of the OWASP Threat Dragon website project,
and also provides the electron desktop project

## Project setup

`npm install`

### Compiles and hot-reloads for development

`npm run serve`

### Compiles and minifies for production

`npm run build`

### Builds the desktop application

`npm run electron:build`

### Runs the desktop application for development

`npm run electron:serve`

Clean the distribution with

`npm run clean`

### Run unit tests

`npm test`

For continuous testing:

`npm run test:unit -- --watch`

Jest coverage will only show coverage for files containing executable javascript.
This means that vue components without any JS logic will not be included in the coverage report.
This is a known limitation, so if you are reading this and there is a workaround
or you know of another way of getting better coverage for `.vue` files, please open an issue or submit a PR! :)

### Run e2e tests

`npm run test:e2e`

### Lints and fixes files

`npm run lint`

## Styles

SCSS is used.  For most things, you can use a scoped scss style block inside your `.vue` file.
For global variables, put it in the appropriately named file in the `src/styles` directory.
Any variables or mixins defined there will be available in all components.

## Font Awesome

Rather than loading all FA icons, we selectively choose the ones that we need.
These are defined in `src/plugins/fontawesome-vue.js`.
To bring in a new icon, import it from the appropriate node_module, and then add it to the `library.add(...)` call.
You do not need to import anything in your components or pages, they are globally available.

## Bootstrap

This project uses [bootstrap-vue](https://www.npmjs.com/package/bootstrap-vue), and it is available globally as well.

## Adding providers

Add a new service named <provider>.provider.js in `src/service/provider`.
See [github.provider.js](src/service/provider/github.provider.js) as an example.
This will need the following:

- `dashboardActions`: An array of objects that describe the actions a user can take from the dashboard
    (after selecting the provider)

## Local Storage

[vuex-persist](https://github.com/championswimmer/vuex-persist) is used to save stores / state from vuex to session storage.
By default, all stores are persisted to session storage.
This is configured in [vuex-persist](src/plugins/vuex-persist.js).
Elastic SIEM Detection Rules: Troubleshooting

## Troubleshooting Python 3.12 Virtual Environment Installation and Activation

When installing and managing virtual environments in Ubuntu (and other Linux operating systems), it is important to remember that the OS may require a specific version of python to perform operating system functions. For detection-rules we use Python version 3.12 which has a number of differences between older versions of Python, most notably no longer including `distutils` which can cause some unusual error messages. 

This section of the guide offers some solutions to some common problems that can occur if one inadvertently overwrites the system's Python 3 installation with Python version 3.12. 

### Issue: Python 3.12 system-wide installation replaced distutils links

If you installed Python 3.12 as a system-wide (or at least WSL wide) installation, it would have replaced the links to distutils from 3.10 with 3.12. However, since Python 3.12 no longer includes distutils as a core package, this will cause issues. 

Try installing python3.12-distutils using apt:

```bash
sudo apt install python3.12-distutils
```

If this command throws an error, you may need to fix apt first.

### Issue: apt's python3 management

If you're having issues with apt's python3 management, it might be due to a faulty python3-apt installation. Try reinstalling python3-apt with the following commands:

```bash
sudo apt remove --purge python3-apt
sudo apt autoclean
sudo apt install python3-apt
```

### Issue: Ubuntu needs to install python3.12-venv via apt

On Ubuntu, you need to install python3.12-venv via apt:

```bash
sudo apt install python3.12-venv
```


### Issue: Python 3.12 removed get-pip.py

Python 3.12 removed get-pip.py, but you can install pip using curl. If you don't have curl installed, you can install it using `sudo apt install curl`. Then, run the following commands:

```bash
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python3.12 get-pip.py
```

### Issue: ensurepip not installed

If ensurepip is not installed, you can install it via pip:

```bash
python3.12 -m pip install ensurepip
```
Elastic SIEM Detection Rules: NOTICE

Detection Rules
Copyright 2021 Elasticsearch B.V.

---
This product bundles rules based on https://github.com/BlueTeamLabs/sentinel-attack
which is available under a "MIT" license. The rules based on this license are:

- "Potential Evasion via Filter Manager" (06dceabf-adca-48af-ac79-ffdf4c3b1e9a)
- "Process Discovery via Tasklist" (cc16f774-59f9-462d-8b98-d27ccd4519ec)
- "Potential Modification of Accessibility Binaries" (7405ddf1-6c8e-41ce-818f-48bea6bcaed8)
- "Potential Application Shimming via Sdbinst" (fd4a992d-6130-4802-9ff8-829b89ae801f)
- "Trusted Developer Application Usage" (9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae1)

MIT License

Copyright (c) 2019 Edoardo Gerosa, Olaf Hartong

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

---
This product bundles rules based on https://github.com/FSecureLABS/leonidas
which is available under a "MIT" license. The rules based on this license are:

- "AWS Access Secret in Secrets Manager" (a00681e3-9ed6-447c-ab2c-be648821c622)

MIT License

Copyright (c) 2020 F-Secure LABS

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Elastic SIEM Detection Rules: typosquatting rule

# Generating detection rule to alert on traffic to typosquatting or homonym domains

## What does the rule do?

This rule helps detect spoofing attacks on domains that you want to protect.


## Steps

### 1. Run [dnstwist](https://github.com/elceef/dnstwist) on the domain you want to watch

Eg: `dnstwist --format json elastic.co | jq`

This should give you a json file consisting of potentially malicious lookalike domains for your domain.

### 2. Index the lookalike domains into Elasticsearch

In order to detect network activity on the lookalike domains using a threat match rule, you would first need to index these domains into an Elasticsearch index using the following CLI command:

`python -m detection_rules typosquat create-dnstwist-index [OPTIONS] INPUT_FILE`

### 3. Prep rule to alert on generated indexes

Run the following CLI command to generate the typosquat rule file, which you will then import into Kibana.

`python -m detection_rules typosquat prep-rule [OPTIONS] AUTHOR`


### 4. Import the rule into Kibana

Import the ndjson rule file generated in the previous step, into Kibana, via the Detection rules UI. 

### 5. Detect potentially malicious network activity targeting your organization!


## Note

- You DO NOT need to re-import the rule file each time you have an additional domain to track. For each new domain, you'd run Step 1 to generate the json file consisting of lookalike domains for that domain, followed by the CLI command in Step 2 to index these domains into a new index. This index will automatically be picked up by the rule you imported the very first time.

- For advanced users, the threat indicator indices (`dnstwist-*`) also contain additional context about the lookalike domains, such as fuzzer information. You can query these indices if you would like to get such context about domains that have been alerted on.
Elastic SIEM Detection Rules: deprecating

# Deprecating rules

Rules that have been version locked (added to [version.lock.json](../detection_rules/etc/version.lock.json)), which also means they
have been added to the detection engine in Kibana, must be properly [deprecated](#steps-to-properly-deprecate-a-rule).

If a rule was never version locked (not yet pushed to Kibana or still in non-`production` `maturity`), the rule can
simply be removed with no additional changes, or updated the `maturity = "development"`, which will leave it out of the
release package to Kibana.


## Steps to properly deprecate a rule

1. Update the `maturity` to `deprecated`
2. Move the rule file to [rules/_deprecated](../rules/_deprecated)
3. Add `deprecation_date` and update `updated_date` to match
4. Add `deprecated_reason` in `[metadata]` with a short explanation (e.g. "Replaced by <rule name>"). Required in the
   same PR that flips `maturity = "deprecated"`; surfaced in Kibana on stacks >= 9.4 and ignored on older stacks.

Next time the versions are locked, the rule will be added to the [deprecated_rules.json](../detection_rules/etc/deprecated_rules.json)
file, and `deprecated_reason` is copied into the package asset (gated at build time by `MIN_STACK_VERSION_DEPRECATED_STUBS`).


### Using the deprecate-rule command

Alternatively, you can run `python -m detection_rules dev deprecate-rule <rule-file>`, which will perform all the steps
Elastic SIEM Detection Rules: DGA

**The setup instructions in this document have been deprecated. Please follow the steps outlined in [this](https://www.elastic.co/security-labs/detect-domain-generation-algorithm-activity-with-new-Kibana-integration) blog to enable DGA detection in your environment.**
# Machine Learning on Domain Generation Algorithm (DGA)

To create and use supervised DGA ML models to enrich data within the stack, check out these Elastic blogs:
* Part 1: [Machine learning in cybersecurity: Training supervised models to detect DGA activity](https://www.elastic.co/blog/machine-learning-in-cybersecurity-training-supervised-models-to-detect-dga-activity)
* Part 2: [Machine learning in cybersecurity: Detecting DGA activity in network data](https://www.elastic.co/blog/machine-learning-in-cybersecurity-detecting-dga-activity-in-network-data)

You can also find some supplementary material and examples [here](https://github.com/elastic/examples/tree/master/Machine%20Learning/DGA%20Detection)

We also released a blog about getting started with DGA using the CLI and Kibana, which also includes a case study of the process applied to the 2020 [SolarWinds supply chain attack](https://www.elastic.co/blog/elastic-security-provides-free-and-open-protections-for-sunburst):
* [Combining supervised and unsupervised machine learning for DGA detection](https://www.elastic.co/blog/supervised-and-unsupervised-machine-learning-for-dga-detection)


For questions, please reach out to the ML team in the #machine-learning channel of the 
[Elastic community Slack workspace](https://www.elastic.co/blog/join-our-elastic-stack-workspace-on-slack)

The team can also be reached by using the `stack-machine-learning` tag in the [discuss forums](https://discuss.elastic.co/tags/c/elastic-stack/stack-machine-learning)

*Note: in order to use these ML features, you must have a platinum or higher [subscription](https://www.elastic.co/subscriptions)*
*Note: the ML features are considered experimental in Kibana as well as this rules CLI*


## Detailed steps

#### 1. Upload and setup the model file and dependencies

Run `python -m detection_rules es <args_or_config> experimental ml setup -t <release-tag>`

*If updating a new model, you should first uninstall any existing models using `remove-model`*

You can also upload files locally using the `-d` option, so long as the naming convention of the files match the 
expected pattern for the filenames.

#### 2. Update packetbeat configuration

You will need to update your packetbeat.yml config file to point to the enrichment pipeline

Under `Elasticsearch Output` add the following:

```yaml
output.elasticsearch:
  hosts: ["your-hostname:your-port"]
  pipeline: dns_enrich_pipeline
```

#### 3. Refresh your packetbeat index

You can optionally choose to refresh your packetbeat index mapping from within Kibana:
* Navigate to `Stack Management > (Kibana) Index Patterns` 
* Select the appropriate packetbeat index
* Click `refresh field list`

#### 4. Verify enrichment fields

Any packetbeat documents with the field `dns.question.registered_domain` should now be enriched with `ml_is_dga.*`
Elastic SIEM Detection Rules: readme


# Experimental machine learning

This repo contains some additional information and files to use experimental[*](#what-does-experimental-mean-in-this-context) machine learning features and detections

## Features
* [DGA](DGA.md)
* [ProblemChild](problem-child.md)
* [HostRiskScore](host-risk-score.md)
* [URLSpoof](url-spoof.md)
* [UserRiskScore](user-risk-score.md)
* [experimental detections](experimental-detections.md)

## Releases

There are separate [releases](https://github.com/elastic/detection-rules/releases) for:
* DGA: `ML-DGA-*`
* ProblemChild: `ML-ProblemChild-*`
* Host Risk Score: `ML-HostRiskScore-*`
* URL Spoof: `ML-URLSpoof-*`
* experimental detections: `ML-experimental-detections-*`

Releases will use the tag `ML-TYPE-YYYMMDD-N`, which will be needed for uploading the model using the CLI.


##### What does experimental mean in this context?

Experimental model bundles (models, scripts, and pipelines), rules, and jobs are components which are currently in 
development and so may not have completed the testing or scrutiny which full production detections are subjected to.

It may also make use of features which are not yet GA and so may be subject to change and are not covered by the support 
SLA of general release (GA) features. Some of these features may also never make it to GA.
Elastic SIEM Detection Rules: experimental detections


# Experimental ML Jobs and Rules

The ingest pipeline enriches process events by adding additional fields, which are used to power several rules. 
The experimental rules and jobs are staged separately from the model bundles under [releases](https://github.com/elastic/detection-rules/releases), with the tag `ML-experimental-detections-YYYMMDD-N`. New releases with this tag may contain either updates to existing rules or new experimental detections. 

Note that if a rule is of `type = "machine_learning"`, then it may be dependent on uploading and running a machine
learning job first. If this is the case, it will likely be annotated within the `note` field of the rule.

### Uploading rules

Unzip the release bundle and upload these rules individually.

Rules are now stored in ndjson format and can be imported into Kibana via the security app detections page.

Earlier releases stored the rules in toml format. These can be uploaded using the 
[7.12 branch](https://github.com/elastic/detection-rules/tree/7.12) CLI using the 
[kibana import-rules](../../CLI.md#uploading-rules-to-kibana) command
Elastic SIEM Detection Rules: problem child

**The setup instructions in this document have been deprecated. Please follow the steps outlined in [this](https://www.elastic.co/security-labs/detecting-living-off-the-land-attacks-with-new-elastic-integration) blog to enable Living off the Land (LotL) detection in your environment.**
# ProblemChild in the Elastic Stack 

ProblemChild helps detect anomalous activity in Windows process events by:
1) Classifying events as malicious vs benign
2) Identifying anomalous events based on rare parent-child process relationships

An end-to-end blog on how to build the ProblemChild framework from scratch for your environment can be found [here](https://www.elastic.co/blog/problemchild-detecting-living-off-the-land-attacks).

You can also find some supplementary material for the blog and examples [here](https://github.com/elastic/examples/tree/master/Machine%20Learning/ProblemChild)

We also released a blog about getting started with ProblemChild using the CLI and Kibana:
* [ProblemChild Release Blog](link to blog)


*Note: in order to use these ML features, you must have a platinum or higher [subscription](https://www.elastic.co/subscriptions)*
*Note: the ML features are considered experimental in Kibana as well as this rules CLI*


## Detailed steps

#### 1. Upload and setup the model file and dependencies

Run `python -m detection_rules es <args_or_config> experimental ml setup -t <release-tag>`

*If updating a new model, you should first uninstall any existing models using `remove-model`*

You can also upload files locally using the `-d` option, so long as the naming convention of the files match the 
expected pattern for the filenames.

#### 2. Update index pipeline configuration

You will need to update your index (containing Windows process event data) settings to point to the ProblemChild enrichment pipeline.

You can do this by running the following command in your Dev Tools console:
```
PUT your-index-pattern/_settings
{
  "index": {
    "default_pipeline": "ML_ProblemChild_ingest_pipeline"
  }
}
```

If you wish to stop enriching your documents using ProblemChild, run the following command in your dev Tools console:
```
PUT your-index-pattern/_settings
{
  "index": {
    "default_pipeline": null
  }
}

```

#### 3. Refresh your indexes

You can optionally choose to refresh your index mapping from within Kibana:
* Navigate to `Stack Management > (Kibana) Index Patterns` 
* Select the appropriate indexes
* Click `refresh field list`

#### 4. Verify enrichment fields

Any documents corresponding to Windows process events should now be enriched with `problemchild.*`. By default, the enrichment pipeline also consists of a script processor for a blocklist, so you might also see the field `blocklist_label` appear in documents that match the blocklist.
Elastic SIEM Detection Rules: README

# rules/

Rules within this folder are organized by solution or platform. The structure is flattened out, because nested file hierarchies are hard to navigate and find what you're looking for. Each directory contains several [.toml](https://github.com/toml-lang/toml) files, and the primary ATT&CK tactic is included in the file name when it's relevant (i.e. [`windows/execution_via_compiled_html_file.toml`](windows/execution_via_compiled_html_file.toml))

| folder                              |  description                                                         |
|-------------------------------------|----------------------------------------------------------------------|
| `.`                                 | Root directory where rules are stored                                |
| [`apm/`](apm)                       | Rules that use Application Performance Monitoring (APM) data sources |
| [`cross-platform/`](cross-platform) | Rules that apply to multiple platforms, such as Windows and Linux    |
| [`integrations/`](integrations)     | Rules organized by Fleet integration                                 |
| [`linux/`](linux)                   | Rules for Linux or other Unix based operating systems                |
| [`macos/`](macos)                   | Rules for macOS                                                      |
| [`ml/`](ml)                         | Rules that use machine learning jobs (ML)                            |
| [`network/`](network)               | Rules that use network data sources                                  |
| [`promotions/`](promotions)         | Rules that promote external alerts into detection engine alerts      |
| [`windows/`](windows)               | Rules for the Microsoft Windows Operating System                     |


Integration specific rules are stored in the [`integrations/`](integrations) directory:

| folder                                                 |  integration                         |
|--------------------------------------------------------|--------------------------------------|
| [`aws/`](integrations/aws)                             | Amazon Web Services (AWS)            |
| [`azure/`](integrations/azure)                         | Microsoft Azure                      |
| [`cyberarkpas/`](integrations/cyberarkpas)             | Cyber Ark Privileged Access Security |
| [`endpoint/`](integrations/endpoint)                   | Elastic Endpoint Security            |
| [`gcp/`](integrations/gcp)                             | Google Cloud Platform (GCP)          |
| [`google_workspace/`](integrations/google_workspace)   | Google Workspace (formerly GSuite)   |
| [`o365/`](integrations/o365)                           | Microsoft Office                     |
| [`okta/`](integrations/okta)                           | Okta                                 |
Elastic SIEM Detection Rules: NOTICE

This product bundles rules based on https://github.com/FSecureLABS/leonidas
which is available under a "MIT" license. The rules based on this license are:

- "AWS Access Secret in Secrets Manager" (a00681e3-9ed6-447c-ab2c-be648821c622)

MIT License

Copyright (c) 2020 F-Secure LABS

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.