Package: antispoofing.fusion_faceverif
This package contains methods for fusion of verification algorithms with anti-spoofing methods at decision and score-level, as well as well as for evaluation of the fused systems under spoofing attacks. In particular, the scripts in this package enable fusion of face verification and anti-spoofing algorithms on theReplay-Attackface spoofing database. The fusion scripts require score files for both the verification and anti-spoofing algorithm. Hence, at least two score files are needed for each video in Replay-Attack: one for the verification and one for the anti-spoofing scores. Each score file contains the scores for all the frames in the corresponding video of Replay-Attack. If there is no score for a particular frame, the score value needs to be Nan. The format of the score files is .hdf5. The score files should be organized in the same directory structure as the videos in Replay-Attack.Some of the scripts can receive multiple score files as input, enabling fusion of more then one verification algorithm with more then one anti-spoofing algorithm.To summarize, the methods in this package enable the user to:parse score files in 4 column or 5 column format (formatspecified inBob) and extract the necessary information from that to organize the score files for each video into Replay-Attack directory structure.evaluate the threshold of a classification system on the development setapply the threshold on an evaluation or any other setdo: AND, SUM, LR and PLR fusion of verification and anti-spoofing system(s)plot performance curvesplot score distributionsplot scatter plots and decision boundariesIf you use this package and/or its results, please cite the following
publications:Bobas the core framework used to run the experiments:@inproceedings{Anjos_ACMMM_2012,
    author = {A. Anjos AND L. El Shafey AND R. Wallace AND M. G\"unther AND C. McCool AND S. Marcel},
    title = {Bob: a free signal processing and machine learning toolbox for researchers},
    year = {2012},
    month = oct,
    booktitle = {20th ACM Conference on Multimedia Systems (ACMMM), Nara, Japan},
    publisher = {ACM Press},
}Anti-spoofing in action: joint operation with an verification system@INPROCEEDINGS{Chingovska_CVPRWORKSHOPONBIOMETRICS_2013,
     author = {Chingovska, Ivana and Anjos, Andr{\'{e}} and Marcel, S{\'{e}}bastien},
   keywords = {biometric recognition, Counter-Measures, Fusion, Spoofing, trustworthy, vulnerability},
   projects = {Idiap, TABULA RASA, BEAT},
      month = jun,
      title = {Anti-spoofing in action: joint operation with a verification system},
  booktitle = {Proceedings of CVPR 2013},
       year = {2013},
   location = {Portland, Oregon},
}InstallationNoteIf you are reading this page through our GitHub portal and not through PyPI,
notethe development tip of the package may not be stableor become
unstable in a matter of moments.Go tohttp://pypi.python.org/pypi/antispoofing.fusion_faceverifto download the latest
stable version of this package. Then, extract the .zip file to a folder of your choice.Theantispoofing.fusion_faceverifpackage is a satellite package of the free signal processing and machine learning libraryBob. This dependency has to be downloaded manually. This version of the package depends onBobversion 2 or greater. To installpackages of Bob, please read theInstallation Instructions. ForBobto be able to work properly, some dependent Bob packages are required to be installed. Please make sure that you have read the Dependencies for your operating system.The most simple solution is to download and extractantispoofing.fusion_faceverifpackage, then to go to the console and write:$ cd antispoofing.fusion_faceverif
$ python bootstrap-buildout.py
$ bin/buildoutThis will download all required dependentBoband other packages and install them locally.Requirements and dependenciesAs mentioned before, this satellite package requires verification and anti-spoofing score files. To generate the face verification scores for the results in the paperAnti-spoofing in action: joint operation with a verification system, we usedFaceRecLib. To generate the anti-spoofing scores, we used the anti-spoofing algorithm from the following satelite packages:antispoofing.lbptopfor LBP and LBP-TOP counter-measures andantispoofing.motionfor motion-based counter-measure. Both the face verification and anti-spoofing scores were generated on per-frame basis. Of course, you can experiment with different verification and anti-spoofing algorithms, as long as your score files are organized as the directory structure of Replay-Attack.This satellite package relies on the following satellite packages:antispoofing.utilsandantispoofing.fusion.User guideThis section explains the step by step procedure how to generate the results presented in the paperAnti-spoofing in action: joint operation with a verification system. The code is tied toReplay-Attackdatabase at the moment.Step 1: Generate score files from the verification and anti-spoofing algorithmsThe first step is to train a face verification algorithm and to create models for each user into it. To generate the face verifications scores, you need to create a protocol for matching real-access (LICIT protocol) and spoof (SPOOF protocol) samples to user models that the algorithms has learned. The LICIT protocol is created by exhaustively matching each real access sample to the user model belonging to the sample’s user and to all the other models. The SPOOF protocol is created my matching the spoof samples to the user model belonging to the sample’s user. In our case, the algorithms work on a frame-by-frame basis. Due to computational limitations, we computed the scores only for every 10th frame of each video. The matching files for the LICIT and SPOOF protocol were then fed into FaceRecLib.To generate the anti-spoofing scores, simply pick your favourite face verification algorithm and feed Replay-Attack to it.This user guide does not give details on the exact commands how to generate the scores. To learn how to do it, please refer directly to FaceRecLib and the anti-spoofing algorithm of your choice.Step 2: Convert the score files to the requested directory structureAs explained before, the score files need to be organized as the directory structure of Replay-Attack. While the anti-spoofing algorithms we use already give the scores in this format, FaceRecLib outputs score files in 4-column format (format), particularly, separate score files for the real accesses (LICIT protocol) and attacks (SPOOF protocol) videos. So, the first step is to convert them into the required format. For example, to convert the licit scores in the development set, run the following command:$ ./bin/four_column_to_dir_structure.py score_file out_dir -t licit -s devel replayThe argumentsscore_fileandout_dirrefer to the 4-column score file which is input, and the directory for the converted scores, respectively. To see all the options for the scriptfour_column_to_dir_structure.py.
just type--helpat the command line. If you want to do the conversion for a particular protocol of Replay-Attack (eg. print protocol), state that protocol at the end of the command:$ ./bin/four_column_to_dir_structure.py score_file out_dir -t licit -s devel replay printDo not forget to do this step for all the dataset subsets (train, development and test set) and the two protocols (LICIT and SPOOF), using the appropriate input files and script options. Depending on the protocol, the scores will be saved into subdirectories calledlicitandspoofwithinout_dir.The score files in 4-column format generated by the recognition algorithm of FaceRecLib used in our work are supplied in this satellite package for your convenience. They can be found in the directory namedsupplemental_data.If it happens that your face verification or anti-spoofing algorithms output the scores in different format, feel free to implement your own convertor to get the scores into Replay-Attack directory structure.Step 3: Decision-level fusionAND decision fusion is supported via the scriptand_decision_fusion.py. AND decision fusion depends on the decision thresholds of the verification and anti-spoofing algorithms separately. Therefore, we first need to compute them:$ ./bin/antispoof_threshold.py as_score_dir replay
$ ./bin/faceverif_threshold.py fv_score_dir replayThe arguments as_score_dir and fv_score_dir refer to the directory with the score files for the anti-spoofing and face verification threshold respectively. The thresholds calculated with these methods are then fed as an input to theand_decision_fusion.pyscript:$ ./bin/and_decision_fusion.py -s fv_score_dir -a as_score_dir --ft fv_thr --at as_thr -v replayThe script directly prints the error rates. To see all the options for the scriptand_decision_fusion.pyjust type--helpat the command line.Step 4: Score-level fusionThree strategies for score-level fusion are available: SUM, LR and PLR. The score-fusion can be performed using the scriptfusion_fvas.py:$ ./bin/fusion_fvas.py -s fv_score_dir -a as_score_dir -o outdirThe script writes the fused scores for each file in the specified output directory in a 4-column format. Having them, you can easily run any script for computing the performance or plotting. Note that you need to run this script separately for the LICIT and the SPOOF protocol for both development and test set at least. This will result in a total of 4 score files. To see all the options for the scriptfusion_fvas.py, just type--helpat the command line. A very important parameter is--spthat will save the normalization parameters and the machine of the fusion for further use.Step 5: Compute performanceTo compute the performance using the 4-column format score-files containing the fused scores, you can use the scriptseval_threshold.pyto calculate the threshold on the development set andapply_threshold.pyto compute the performance using the obtained treshold. Do not forget that you have 4 score files (one for development and one for test set for the LICIT and SPOOF protocol), and depending on your needs, you can use any of them for evaluation or application of the threshold.:$ ./bin/eval_threshold.py -s develset_score_file -c eer
$ ./bin/apply_threshold.py -s testset_score_file -t thrUsually, the development set score file of the LICIT protocol is used to evaluate the threshold. That threshold can be used on any of the 4 score files.To see all the options for the scripts, just type--helpat the command line.Step 6: Plot performance curvesUsing the scriptplot_on_demand.py, you can choose to plot many different plots like score distributions or DET curves on the LICIT, SPOOF protocol or both. Just see at the documentation of the script to see what input you need to specify for the desired curve. As mandatory input, you need to give the score files for the LICIT and SPOOF protocol for both the development and test set.:$ ./bin/plot_on_demand.py devel_licit_scores eval_licit_scores devel_spoof_scores eval_spoof_scores -b eer -i 2This will plot the DET curve of the LICIT protocol overlayed with the DET curve of the SPOOF protocol. To see all the options for the scriptplot_on_demand.py, just type--helpat the command line.Step 7: Scatter plotsScatter plots plot the verification and anti-spoofing scores in the 2D space, together with a decision boundary depending on the algorithms used for their fusion. To plot a scatter plot for LLR fused scores, type:$ ./bin/scatter_plot.py -s fv_score_dir -a as_score_dir -m machine_file -n norm_file -d thr -f LLRThe devel threshold (specified with-dparameter) is a mandatory argument in this script. In the case of AND fusion (option-fAND), two thresholds need to be specified with this argument. Normalization parameter (parameter-nnorm_file) needs to be specified for the score fusion algorithms (i.s. option-fSUM,-fLLRor-fLLR_P), where norm_file is a file containing the normalization parameters. Usually, this is the file saved when the option--spis set when running the scriptfvas_fusion.pyin Step 4. Similarly, the score fusion algorithms require the parameter-mmachine_file, where machine_file contains the of the fusion algorithm. It is also saved when the option--spis set when running the scriptfvas_fusion.pyin Step 4.To see all the options for the scriptscatter_plot.py, just type--helpat the command line.Additional informationThe package contains an additional scriptdir_to_four_column.pythat might be useful in some cases. It converts scores from Replay-Attack directory structure to 4-column file structure.ProblemsIn case of problems, please contactivana.chingovska@idiap.ch