bat_detect/evaluate/readme.md

# Evaluating BatDetect2
This script evaluates a trained model and outputs several plots summarizing the performance. It is used as follows:  
`python path_to_store_images/ path_to_audio_files/ path_to_annotation_file/ path_to_trained_model/`

e.g.  
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar`

By default this will just evaluate the set of test files that are already specified in the model at training time. However, you can also specify a single set of annotations to evaluate using the `--test_file` flag. These must be stored in one annotation file, containing a list of the individual files.  

e.g.    
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar --test_file yucatan_TEST.json`  

You can also specify if the plots are saved as a .png or .pdf using `--file_type` and you can set title text for a plot using `--title_text`, e.g. `--file_type pdf --title_text "My Dataset Name"`




### Comparing to Tadarida-D
It is also possible to compare to Tadarida-D. For Tadarida-D the following steps are performed:  
- Matches Tadarida's detections to manually annotated calls   
- Trains a RandomForest classifier using Tadarida call features  
- Evaluate the classifier on a held out set  

Uses precomputed binaries for Tadarida-D from:  
`https://github.com/YvesBas/Tadarida-D/archive/master.zip`  

Needs to be run using the following arguments:  
`./TadaridaD -t 4 -x 1 ip_dir/`  
-t 4 means 4 threads  
-x 1 means time expansions of 1  

This will generate a folder called `txt` which contains a corresponding `.ta` file for each input audio file. Example usage is as follows:  
`python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar --td_ip_dir /data1/bat_data/baselines/tadarida_D/`