subsample-test-reweigh 0.0.2
SubsampleTestReweigh algorithm for transfer learning
Requires Python: >=3.8


# Standard Readme

## Table of Contents

- [Source code](#source-code)
- [Background](#background)
- [Install](#install)
- [Usage](#usage)
- [Contributors](#contributors)
- [License](#license)

## Source code

The source code and full documentation:
[source-code](https://github.com/AnonymousICML2022/SampleTestReweigh)

## Background

This project is a simulation of the non-private version Subsample-Test-Reweigh. The simulation and the results are described in the paper: "Transfer Learning In Differential Privacy's Hybrid-Model", written by anonymous. 
(submitted to ICML-2022).

For now, it is possible to run it only on the example of the distributions described in the paper, but in the near future, we intend to extend it to more general distributions and for the private version of the algorithm. 

## Install

This project exists as a Python package in [Pypi](https://pypi.org/project/subsample-test-reweigh) repository and can be installed to your python environment using **pip** command:

```sh
$ pip install -u subsample-test-reweigh
```
It is recommended to use **virtualenv** or **conda** to create a clean python environment.

The project was tested on python3.8 in the Windows or Linux operating system.

The following dependencies are required and will be installed automatically with the above command:

NumPy, Scipy, Pandas, Scikit-Learn, Matplotlib.

## Usage

#### **Step 1:** running the simulation on different sample sizes from the source distribution:

The main script called: run_str_simulation.py
	
The script runs the simulation with one set of parameters. In the simulation described in the paper, we run it several times, each with a different sample size from the source distribution. 

Following the command for running 50 repetitions of the simulation (each with a different seed of the random numbers generator), with a sample size of 90K examples from the source distribution. We run it in parallel on 25 processors (so 25 repetitions run first and after them another group). 

```sh
$ python run_str_simulation.py --output-dir str-output-dir --title paperExample --multiproc 25 --num-rep 50 --sample-size-s 90000
```
In a similar way, we run the following similar command for the sample size of 100K and so on until 140K.

Notice that we put the results under the same --output-dir folder and the same --title, in order to enable us to aggregate and plot the results from all runs.

```sh
$ python run_str_simulation.py --output-dir str-output-dir --title paperExample --multiproc 25 --num-rep 50 --sample-size-s 100000
```

The running time of one repetition is ~9-14 hours (more time for a small sample size).

Full help on the parameters of the script you can achieve with the command:

```sh
$ python run_str_simulation.py --help  
```
The output is: [run-str-simulation-help](str-output-dir/toyRunForTest/run_str_sumulation_cmd.txt)

#### **Step 2:** agreggate and plot the results.

After the all runs (with the different sample sizes) ended, we run another script for aggregating the output files and creating plots:

```sh
$ python create_plots.py --input-dir str-output-dir/paperExample --sample-size-s-list 90000 100000 110000 120000 130000 140000 --num-rep 50 --run-parallel
```
which means that the script needs to aggregate outputs of 50 repetitions of each of the runs in the --sample-size-s-list list. For proper aggregation, we also must indicate that the runs are made in parallel by adding the --run-parallel flag.

The plots will be located in str-output-dir/paperExample/plots folder.

Full help on the parameters of the script you can achieve with the command:
```sh
$ python create_plots.py --help  
```
The output is: [create-plots-help](str-output-dir/toyRunForTest/create_plots_cmd.txt)


### Toy example of run:

In order to illustrate the running of the script, you can run a toy example that takes only several minutes and can be run on any computer (with OS of Linux or Windows):

In the toy example, we use the parameter: --std-k-t 0.04, which means that the standard deviation of the K different coordinates of the target distribution is 0.04 instead of 0.01 in our real example in the paper. In this example, the algorithm converges very fast because no the distributions are not very far from each 
other.

```sh
$ python run_str_simulation.py --output-dir str-output-dir --title toyRunForTest --std-k-t 0.04 --multiproc 10 --num-rep 10 --sample-size-s 50000
$ python run_str_simulation.py --output-dir str-output-dir --title toyRunForTest --std-k-t 0.04 --multiproc 10 --num-rep 10 --sample-size-s 100000
$ python run_str_simulation.py --output-dir str-output-dir --title toyRunForTest --std-k-t 0.04 --multiproc 10 --num-rep 10 --sample-size-s 150000
```
```sh
$ python create_plots.py --input-dir str-output-dir\toyRunForTest --sample-size-s-list 50000 100000 150000 --num-rep 10 --run-parallel --skip-iterations 1  
```

Example of the output files of the toy example you can find under 
[str-output-dir](str-output-dir) folder , and the plots are located under the [plots](str-output-dir/toyRunForTest/plots) folder.


### Advanced parameters:
A full list of parameters can be found in [params.py](SubsampleTestReweigh/params.py) file. The parameters in the command line override the values in this file. 

## Contributors

The code has been developed by Anonymous: anonymous@domain.com under the supervision of Anonymous anonymous@domain.com.

## License

[BSD 3-Clause](LICENSE)