hpo-gui 0.0.6
Graphical user interface for hyperparameter optimization on CERN computing grid
Requires Python: >=3.6

# HyperParameter Optimization Graphical User Interface (HPOGUI)

# Table of Content
1. [Installation](#installation)
2. [Introduction](#introduction)
3. [Search Space Configuration](#search-space-configuration)
4. [Job Configuration](#job-configuration)


# Installation 
<p> This package requires python>=3.6 and can be installed as follows

```
git clone ssh://git@gitlab.cern.ch:7999/tupham/hpogui.git -b master
cd hpogui
virtualenv ve --python=[your link/alias to python >= 3.6]
sourve ve/bin/activate
pip install dist/hpo_gui-0.0.5-py3-none-any.whl 
```
</p>

# Introduction
<p>
This is a package that provides interactive user interface for hyperparamter optimization (HPO) job submission to PANDA. Currently, it includes tools to do user authentication, to define and submit a job. More functionalities will be added as the package is developed. 
</p>
<p> 
An HPO job comprises 3 elements:
<ol>
<li> The search space.</li>
<li> The job configurations.</li>
<li> Training and execution scripts.</li>
</ol>
An example of job submission using a payload (training scripts) from <a href="https://gitlab.cern.ch/zhangruihpc/iddsWF/-/tree/master/submit_rnn">this repo</a> by Rui Zhang is used the demonstrate the workflow. This payload is incorporated in a container image ready for use, so there is no need create your own container. More on how to create an evaluation container later. Let us go through each of the above-mentioned elements.

# Search space configuration. 
An HPO job is essentially <strong> a parallelised search on a grid of hyperparamter points </strong> (the search space) for the optimal (lowest) value of the loss function. Therefore, it requires a search space configuration to define the search space and instructs the steering container how to generate search points for loss function evaluation. 
</p>
<p> One uses hpogui.SearchSpace.SearchSpaceForm to create the search space configuration. After installing the package, create  In a jupyter notebook,

```
from hpogui.SearchSpace import SearchSpaceForm
form=SearchSpaceForm()
form.define_parameters()
```
A widget appears and lets the user define a search space. Each bar represents a HP and comprises a new, an optimization method and corresponding parameters. 

<img src="images/searchSpace.png">

To add more HPs, click on "Add HP" button. To delete HPs, check on the "Remove Hyperparameter" box to mark for removal, then click on "Delete HP". To submit the search space, click on "Submit", upon which the form will be closed, the validity of all HPs checked and the search space saved if all HPs are valid. Finally, to cancel the form, click on "Cancel". The form will close, and none of the input will be saved.

To save the search space file in json or yaml format, 
```
form.to_json("search_space.json")
# save as a json file.
form.to_yaml("search_space.json")
# save as a yaml file.
```

If no name is passed to these methods, a json or yaml string is returned for checking. 
```
print(form.to_json())
print(form.to_yaml("search_space.json"))
```

To understand the optimization methods, checkout the section 5, part 4 of the README in <a href="https://gitlab.cern.ch/aml/hyperparameter-optimization/alkaid-qt/hpogrid/-/blob/master/README.md">this repo</a>.

SearchSpaceForm is a subunit of the Phpo module, which submits the HPO task. This means that the user can submit the task without referencing an external the search space file. One can save the file if so desire, but it is not necessary.

# Job Configuration 
The job configiration defines the parameters required to run the job on the grid. Much like the search space, the job configs can be define by a JobConfigForm as follows
```
from hpogui.JobConfiguration import JobConfigForm
jconf=JobConfigForm()
jconf.define_configurations()
```

<img src="images/jobConfig.png">

The form contains 4 categories of configurations. The detail on each configurations can be found at <a href="https://panda-wms.readthedocs.io/en/latest/client/phpo.html">this documentation</a>.