semvec/pypi-packages huggingface dataset:
ddlk 0.0.0.2
Deep direct likelihood knockoffs
Requires Python: >=3.6

# DDLK: Deep direct likelihood knockoffs

This package implements DDLK, a method for variable selection with explicit control of the false discovery rate.
Install with:
```
pip install ddlk
```

![](img/ddlk_train.gif)

## Controlled variable selection with DDLK

Suppose you have a set of features and a response.
DDLK identifies the features most predictive of the response at a pre-specified false discovery rate (FDR) threshold.
For example, if you choose an FDR of 20%, DDLK can guarantee that no more than 20% of the selected features will be unimportant.
To learn more about how it works, check out [our paper](https://arxiv.org/abs/2007.15835).

## Running DDLK

Variable selection with DDLK involves three stages:

1. Fit a joint distribution to model features
2. Fit a knockoff generator
3. Sample knockoffs and apply [knockoff filter](https://arxiv.org/abs/1610.02351) to select variables at a pre-specified FDR

To see a complete working example, check our [synthetic data example](examples/ddlk_synthetic_example_CPU.ipynb), used to generate the gif above.
Below is an exceprt of how to run DDLK.

### Fitting a joint distribution

This implementation of DDLK uses the fast and easy [PyTorch Lightning framework](https://github.com/PytorchLightning/pytorch-lightning) to fit `q_joint`:

```python
# initialize data
x, y = ...
# put your data in standard PyTorch format
trainloader = ...
# initialize joint distribution model with mean and std of data
((X_mu, ), (X_sigma, )) = utils.get_two_moments(trainloader)
hparams = argparse.Namespace(X_mu=X_mu, X_sigma=X_sigma)
q_joint = mdn.MDNJoint(hparams)
```


```python
# create and fit a PyTorch Lightning trainer
trainer = pl.Trainer()
trainer.fit(q_joint, train_dataloader=trainloader)
```

### Fitting a knockoff generator

```python
# initialize and fit a DDLK knockoff generator
q_knockoff = ddlk.DDLK(hparams, q_joint=q_joint)
trainer = pl.Trainer()
trainer.fit(q_knockoff, train_dataloader=trainloader)
```

### Variable selection

Using the knockoff generator, we sample knockoffs, and run a [Holdout Randomization Test](https://arxiv.org/abs/1811.00645):

```python
xTr_tilde = q_knockoff.sample(xTr)
knockoff_test = hrt.HRT_Knockoffs()
knockoff_test.fit(xTr, yTr, xTr_tilde)
```


# Citing this code
If you use this code, please cite the following paper ([available here](https://arxiv.org/abs/2007.15835)):
```
Deep Direct Likelihood Knockoffs
M. Sudarshan, W. Tansey, R. Ranganath
arXiv preprint arXiv:2007.15835
```
Bibtex entry:
```
@misc{sudarshan2020deep,
    title={Deep Direct Likelihood Knockoffs},
    author={Mukund Sudarshan and Wesley Tansey and Rajesh Ranganath},
    year={2020},
    eprint={2007.15835},
    archivePrefix={arXiv},
    primaryClass={stat.ML}
}
```
thirdparty-yaml 0.1.0
Tools for managing yaml configurations.
Requires Python: ~=3.7

# thirdparty.yaml

YAML Processing Extensions (e.g. Overlays, Recursive Merging, Includes)

## Usage Synopsis

```
from thirdparty.yaml.overlay import YamlOverlay

yaml_overlay = YamlOverlay()
obj = yaml_overlay.load(yaml_string)
print(yaml_overlay.dump(obj))
```

## Overview

`thirdparty.yaml` enables yaml content creators to add include and recursive merge directives within the yaml itself.

Example YAML with recursive_merge and include:

```
---
thing:
  value: 4

<<<:
  thing:
    additional: 6

!recursive_merge :
  thing:
    yet_another_recursive_merge: true

<<<: !include test_simple_mapping.yaml
```

The result of the above is:

```
---
thing:
  value: 4
  additional: 6
  yet_another_recursive_merge: true

otherthings:
  myval: 45
```

## Install

```
python3 -m pip install --upgrade thirdparty-yaml
```

## Contributing

Any contributions and feature requests are welcome! Please submit all contributions for `thirdparty.yaml` as an issue or pull-request (PR) on [thirdparty.yaml's Github Page](https://github.com/third-party-dev/yaml).

## Authors

- Paul Kohout
- Vincent Agriesti

## License

```
MIT License

Copyright (c) 2020 third-party-dev

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

```
dface 1.1.5
Face detection and recognition library that focuses on speed and ease of use.

## dFace

Face detection and recognition library that focuses on speed and ease of use.
This is a stripped down version of timesler's [facenet](https://github.com/timesler/facenet-pytorch) repo with some improvements notably on memory overflows. This repository is actively developed and used in production.

[MTCNN](https://arxiv.org/abs/1604.02878) is used for face detection and [FaceNet](https://arxiv.org/abs/1503.03832) is used for generating face embeddings.

### Installation

Both algorithms can be used directly by copying the source file into your code base. They are single file and single model implementations with no interdependence.

Alternatively you can install with `pip3 install dface` command. Weights will be  downloaded from github automatically in the first run.

### MTCNN

This implementation takes a list of frames and returns a list of detected faces. The input and output lists always have the same size. If no face is detected for a frame, the corresponding list element is returned `None`, otherwise it's a tuple of three elements; `(bounding_boxes, probabilities, landmarks)`.

```python
mtcnn = MTCNN('cuda')
...
result = mtcnn.detect(frames)
for res in result:
    if res is None:
        continue
    boxes, probs, lands = res
```

Bounding boxes is a list of four coordinates `(x1, y1, x2, y2)` for each face detected in the frame. Probabilities is a list of confidence scores between 0 and 1 for each face. Landmarks is a list of facial landmark points. There are five points for each face.

The implementation is at [mtcnn.py](dface/mtcnn.py)

### FaceNet

This implementation takes a list of cropped faces and returns a list of face embedding vector of 512 dimensions. These embeddings then can be used for detecting facial similarities, clustering identities and so on.

```python
facenet = FaceNet('cuda')
...
embeds = facenet.embedding(faces)
```

The implementation is at [facenet.py](dface/facenet.py)

Both algorithms accept model path as the second argument if you want to load models from a custom location.

### Full working example

The repo contains an [example](example.py) script that uses both MTCNN and Facenet to read a video and cluster the faces. It uses [OpenCV](https://pypi.org/project/opencv-python/) to read video frames and [DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html) to cluster faces.

You can run the example by simply typing: `python3 example.py video.mp4`. A directory will be created with the same name as video that contains clustered identities. The following is an example image.

![the dude](https://i.imgur.com/npt6W0l.png)

The implementation is at [example.py](example.py)
numdatasmexample 0.0.1
A simple package calculating values for a single number
Keywords: package numbers calculations

# NumData Documentation
***
## Installation
numdata can be installed using python pip. The package is hosted on [PYPI](https://pypi.org/) and [PYPI-test](https://test.pypi.org/)
```sh
pip install numdata
```
***
## Usage
Initialization is not required for using methods in numdata
```python
from numdata import calculate

# Calculate square of a number
print(calculate.calculateSquare(8))
# 64

# Calculate cube of a number
print(calculate.calculateCube(8))
# 512
```
***