Spaces:
Running
A newer version of the Gradio SDK is available:
4.36.1
Evaluating BatDetect2
This script evaluates a trained model and outputs several plots summarizing the performance. It is used as follows:python path_to_store_images/ path_to_audio_files/ path_to_annotation_file/ path_to_trained_model/
e.g.python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar
By default this will just evaluate the set of test files that are already specified in the model at training time. However, you can also specify a single set of annotations to evaluate using the --test_file
flag. These must be stored in one annotation file, containing a list of the individual files.
e.g.python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar --test_file yucatan_TEST.json
You can also specify if the plots are saved as a .png or .pdf using --file_type
and you can set title text for a plot using --title_text
, e.g. --file_type pdf --title_text "My Dataset Name"
Comparing to Tadarida-D
It is also possible to compare to Tadarida-D. For Tadarida-D the following steps are performed:
- Matches Tadarida's detections to manually annotated calls
- Trains a RandomForest classifier using Tadarida call features
- Evaluate the classifier on a held out set
Uses precomputed binaries for Tadarida-D from:https://github.com/YvesBas/Tadarida-D/archive/master.zip
Needs to be run using the following arguments:./TadaridaD -t 4 -x 1 ip_dir/
-t 4 means 4 threads
-x 1 means time expansions of 1
This will generate a folder called txt
which contains a corresponding .ta
file for each input audio file. Example usage is as follows:python evaluate_models.py ../../plots/results_compare_yuc/ /data1/bat_data/data/yucatan/audio/ /data1/bat_data/annotations/anns_finetune/ ../../experiments/2021_12_17__15_58_43/2021_12_17__15_58_43.pth.tar --td_ip_dir /data1/bat_data/baselines/tadarida_D/