Spaces:
Running
Running
# Evaluating BatDetect2 | |
This script evaluates a trained model and outputs several plots summarizing the performance. It is used as follows: | |
`python path_to_store_images/ path_to_audio_files/ path_to_annotation_file/ path_to_trained_model/` | |
e.g. | |
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar` | |
By default this will just evaluate the set of test files that are already specified in the model at training time. However, you can also specify a single set of annotations to evaluate using the `--test_file` flag. These must be stored in one annotation file, containing a list of the individual files. | |
e.g. | |
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar --test_file yucatan_TEST.json` | |
You can also specify if the plots are saved as a .png or .pdf using `--file_type` and you can set title text for a plot using `--title_text`, e.g. `--file_type pdf --title_text "My Dataset Name"` | |
### Comparing to Tadarida-D | |
It is also possible to compare to Tadarida-D. For Tadarida-D the following steps are performed: | |
- Matches Tadarida's detections to manually annotated calls | |
- Trains a RandomForest classifier using Tadarida call features | |
- Evaluate the classifier on a held out set | |
Uses precomputed binaries for Tadarida-D from: | |
`https://github.com/YvesBas/Tadarida-D/archive/master.zip` | |
Needs to be run using the following arguments: | |
`./TadaridaD -t 4 -x 1 ip_dir/` | |
-t 4 means 4 threads | |
-x 1 means time expansions of 1 | |
This will generate a folder called `txt` which contains a corresponding `.ta` file for each input audio file. Example usage is as follows: | |
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar --td_ip_dir /data1/bat_data/baselines/tadarida_D/` | |