Package: xbob.paper.tpami2013
This package contains scripts that show how to use the implementation
of the scalable formulation of Probabilistic Linear Discriminant Analysis
(PLDA), integrated intoBob, as
well as how to reproduce experiments of the article mentioned below.
It is implemented and maintained viagithub.ContentsReferencesInstallationUsing an automatic installerUsingzc.buildoutPLDA tutorialReproducing experimentsNote for Grid UsersLabeled Faces in the Wild datasetGetting the features and converting them into HDF5 formatPCA+PLDA toolchain on LFWSummarizing the results as in Table 2Multi-PIE datasetGetting the dataFeature extractionDimensionality reductionProposed system: PLDA modeling and scoringBaseline 1: PCA on the LBP histogramsBaseline 2: LDA on the PCA projected LBP histogramsBaseline 3: LBP histogram classification with Chi square scoringSummarizing the results as in Table 3Reporting bugsReferencesIf you use this package and/or its results, please cite the following
publications:The original paper with the scalable formulation of PLDA explained
in details:@article{ElShafey_TPAMI_2013,
  author = {El Shafey, Laurent and McCool, Chris and Wallace, Roy and Marcel, S{\'{e}}bastien},
  title = {A Scalable Formulation of Probabilistic Linear Discriminant Analysis: Applied to Face Recognition},
  year = {2013},
  month = jul,
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  volume = {35},
  number = {7},
  pages = {1788-1794},
  doi = {10.1109/TPAMI.2013.38},
  pdf = {http://publications.idiap.ch/downloads/papers/2013/ElShafey_TPAMI_2013.pdf}
}Bob as the core framework used to run the experiments:@inproceedings{Anjos_ACMMM_2012,
  author = {A. Anjos and L. El Shafey and R. Wallace and M. G\"unther and C. McCool and S. Marcel},
  title = {Bob: a free signal processing and machine learning toolbox for researchers},
  year = {2012},
  month = oct,
  booktitle = {20th ACM Conference on Multimedia Systems (ACMMM), Nara, Japan},
  publisher = {ACM Press},
  url = {http://publications.idiap.ch/downloads/papers/2012/Anjos_Bob_ACMMM12.pdf},
}If you decide to use the Multi-PIE database, you should mention the
following paper, where it is introduced:@article{Gross_IVC_2010,
  author = {Gross, Ralph and Matthews, Iain and Cohn, Jeffrey and Kanade, Takeo and Baker, Simon},
  title = {Multi-PIE},
  journal = {Image and Vision Computing},
  year = {2010},
  month = may,
  volume = {28},
  number = {5},
  issn = {0262-8856},
  pages = {807--813},
  numpages = {7},
  doi = {10.1016/j.imavis.2009.08.002},
  url = {http://dx.doi.org/10.1016/j.imavis.2009.08.002},
  acmid = {1747071},
}If you only use the Multi-PIE annotations, you should cite the following paper
since annotations were made for the experiments of this work:@article{ElShafey_TPAMI_2013,
  author = {El Shafey, Laurent and McCool, Chris and Wallace, Roy and Marcel, S{\'{e}}bastien},
  title = {A Scalable Formulation of Probabilistic Linear Discriminant Analysis: Applied to Face Recognition},
  year = {2013},
  month = jul,
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  volume = {35},
  number = {7},
  pages = {1788-1794},
  doi = {10.1109/TPAMI.2013.38},
  pdf = {http://publications.idiap.ch/downloads/papers/2013/ElShafey_TPAMI_2013.pdf}
}If you decide to use the Labeled Faces in the Wild (LFW) database, you should
mention the following paper, where it is introduced:@TechReport{LFWTech,
  author = {Gary B. Huang and Manu Ramesh and Tamara Berg and Erik Learned-Miller},
  title = {Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments},
  institution = {University of Massachusetts, Amherst},
  year = {2007},
  number = {07-49},
  month =  oct,
}InstallationNoteIf you are reading this page through our GitHub portal and not through PyPI,
notethe development tip of the package may not be stableor become
unstable in a matter of moments.Go tohttp://pypi.python.org/pypi/xbob.paper.tpami2013to download the latest
stable version of this package.There are two options you can follow to get this package installed and
operational on your computer: you can use automatic installers likepip(oreasy_install) or manually download, unpack and
usezc.buildoutto create a
virtual work environment just for this package. In both cases, you must
first installBob(>= 1.2.0), whose installation process is described
in theuser guide.Using an automatic installerUsingpipis the easiest (shell commands are marked with a$signal):$ pip install xbob.paper.tpami2013You can also do the same witheasy_install:$ easy_install xbob.paper.tpami2013This will download and install this package plus any other required
dependencies. It will also verify if the version of Bob you have installed
is compatible.This scheme works well with virtual environments byvirtualenvor if you have root access to your
machine. Otherwise, we recommend you use the next option.Usingzc.buildoutDownload the latest version of this package fromPyPIand unpack it in your
working area:$ wget http://pypi.python.org/packages/source/x/xbob.paper.tpami2013/xbob.paper.tpami2013-1.0.0.zip
$ unzip xbob.paper.tpami2013-1.0.0.zip
$ cd xbob.paper.tpami2013-1.0.0The installation of the toolkit itself usesbuildout. You don’t need to understand its inner workings
to use this package. Here is a recipe to get you started:$ python bootstrap.py
$ ./bin/buildoutThese two commands should download and install all non-installed dependencies and
get you a fully operational test and development environment.NoteThe python shell used in the first line of the previous command set
determines the python interpreter that will be used for all scripts developed
inside this package. Because this package makes use ofBob,
you must make sure that thebootstrap.pyscript is called with thesameinterpreter used to build Bob, or
unexpected problems might occur.If Bob is installed by the administrator of your system, it is safe to
consider it uses the default python interpreter. In this case, the above 3
command lines should work as expected. If you have Bob installed somewhere
else on a private directory, edit the filebuildout.cfgbeforerunning./bin/buildout. Find the section namedbuildoutand edit or
add the lineprefixesto point to the directory where Bob is installed or
built. For example:[buildout]
...
prefixes=/home/laurent/work/bob/buildPLDA tutorialThe following example consists of a simple script, that makes use of
Probabilistic Linear Discriminant Analysis (PLDA) modeling on the
Fisher’s iris dataset. It performs the following tasks:Train a PLDA model using the first two classes of the datasetEnroll a class-specific PLDA model for the third class of the datasetCompute (verification) scores for both positive and negative samplesPlot the distribution of the scores and save it into a fileTo run this simple example, you just need to execute the following command:$ ./bin/plda_example_iris.py --output-img plda_example_iris.pngReproducing experimentsIt is currently possible to reproduce all the experiments of the article
on both Labeled Faces in the Wild and Multi-PIE using the PLDA algorithm.
In particular, the value of the accuracy reported in Table 2, the
Figure 2 and the HTER reported on Table 3 can be easily reproduced, by
following the steps described below.Be aware that all the scripts provide several optional arguments that
are very useful if you wish to use your own features or your own
parameters.Keep in mind that the results published in the paper were obtained with
a pre-release of Bob (older than 1.0.0). You might hence observe slight
differences when running the scripts with Bob 1.2.0.Note for Grid UsersAt Idiap, we use the powerful Sun Grid Engine (SGE) to parallelize our
job submissions as much as we can. At the Biometrics group, we have developed
a little toolboxgridtkthat can
submit and manage jobs at the Idiap computing grid through SGE.The following sections will explain how to reproduce the paper results in
single (non-gridified) jobs. If you are at Idiap, you could run the
following commands on the SGE infrastructure, by applying the ‘–grid’
flag to any command. This may also work on other locations with an SGE
infrastructure, but will likely require some configuration changes in the
gridtk utility.Labeled Faces in the Wild datasetThe experiments of this section are performed on the LFW (Labeled Faces
in the Wild) protocol. The features are publicly available and will be
automatically downloaded fromthis webpageif you follow the
instruction below. They were extracted on the LFW images aligned with the
funneling algorithm.Getting the features and converting them into HDF5 formatThe following command will download a tarball with the SIFT features,
extract the content of the archive and convert the features into a
suitable HDF5 format for Bob:$ ./bin/lfw_features.py --output-dir /PATH/TO/LFW/DATABASE/PCA+PLDA toolchain on LFWOnce the features have been extracted, the dimensionality is reduced
using Principal Component Analysis (PCA), before applying PLDA modeling.
These steps are combined in the following script, that will run the
PCA+PLDA toolchain on the specified protocol:$ ./bin/toolchain_pcaplda.py --features-dir /PATH/TO/LFW/DATABASE/lfw_funneled --protocol view1 --output-dir /PATH/TO/LFW/OUTPUT_DIR/To report the final performance on LFW, it is required to run
10 experiments on view 2 in a leave-one-out cross validation scheme.
We provide the following script for this purpose:$ ./bin/experiment_pcaplda_lfw.py -features-dir /PATH/TO/LFW/DATABASE/lfw_funneled --output-dir /PATH/TO/LFW/OUTPUT_DIR/NoteThe previous script is monothreaded and will run the 10 independent
view 2 experiments in a sequence. If you have a multi-core CPU, you
could split this script into several shorter jobs, by splitting the
loop below, which will at the end be equivalent to the previous
command:$ for k in `seq 1 10`; do \
    ./bin/toolchain_pcaplda.py --features-dir /PATH/TO/LFW/DATABASE/lfw_funneled --protocol view2-fold${k} --output-dir /PATH/TO/LFW/OUTPUT_DIR/ ; \
  doneSummarizing the results as in Table 2Once the previous experiments have successfully completed, you can use
the following script to plot Table 2, that will estimate the mean
accuracy as well as the standard error of the mean on the 10 experiments
of LFW view2:$ ./bin/plot_table2.py --output-dir /PATH/TO/LFW/OUTPUT_DIR/NoteCompared to the results published in the article, there are slight
differences caused by both the order of the training files when applying
PCA, and the lists used to split the LFWtrainingset into atrainingset and avalidationset (The validation set is use to select the
verification threshold to apply on the test set).Multi-PIE datasetThe experiments of this section are performed on the U protocol of the
Multi-PIE dataset. The filelists associated with this protocol can be found
onthis website.Getting the dataYou first need to buy and download the Multi-PIE database:http://multipie.org/and to download the annotations available here:http://www.idiap.ch/resource/biometric/Feature extractionThe following command will extract Local Binary Patters (LBP) histograms
features. You should set the paths to the data according to your own
environment:$ ./bin/lbph_features.py --image-dir /PATH/TO/MULTIPIE/IMAGES --annotation-dir /PATH/TO/MULTIPIE/ANNOTATIONS --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/NoteThe output directory /PATH/TO/MULTIPIE/OUTPUT_DIR/ is a base directory
for the output of all experiments on Multi-PIE. Make sure to use the
same directory for all the experiments below, otherwise the following
commands might not work as expected. You can look at the options
of the scripts if you need more flexibility or want to use alternate
features vectors, etc.Dimensionality reductionOnce the features has been extracted, they are projected into a lower
dimensional subspace using Principal Component Analysis (PCA):$ ./bin/pca_features.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/NoteEquivalently, this can also be achieved by running the following
individual commands:$ ./bin/pca_train.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph --pca-dir features --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/linear_project.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph --algorithm-dir features --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/Proposed system: PLDA modeling and scoringPLDA is then applied on the dimensionality reduced features.This involves three different steps:TrainingModel enrollmentScoringThe following command will perform all these steps:$ ./bin/toolchain_plda.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/NoteEquivalently, this can also be achieved by running the following
individual commands:$ ./bin/plda_train.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/plda_enroll.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/plda_scores.py --group dev --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/plda_scores.py --group eval --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/Then, the HTER on the evaluation set can be obtained using the
evaluation script from the bob library as follows:$ ./bin/bob_compute_perf.py -d /PATH/TO/MULTIPIE/OUTPUT_DIR/U/plda/scores/scores-dev -t /PATH/TO/MULTIPIE/OUTPUT_DIR/U/plda/scores/scores-eval -xThe HTER on the evaluation set, when using the EER on the development
set as the criterion for the threshold, corresponds to the PLDA value reported
on Table 3 of the article mentioned above.If you want to reproduce the Figure 2 of the article, you can run the
following commands (instead of the previous one):$ ./bin/experiment_plda_subworld.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/plot_figure2.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/NoteEquivalently, this can also be achieved by running the following
individual commands. Be aware that the commands within the loop
are independent and monothreaded. Furthermore, you could break
the loop and call several of these commands at the same time
if your CPU has several cores:$ for k in 2 4 6 8 10 14 19 29 38 48 57 67 76; do \
    ./bin/toolchain_plda.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/ --world-nshots $k --plda-dir plda_subworld_${k}; \
  done
$ ./bin/plot_figure2.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/The previous commands will run the PLDA toolchain several times for a varying
number of training samples. Please note, that this will require a lot of time
to complete (a bit less than two days on a recent workstation such as one with an
Intel Core i7 CPU).Then, the value of the HTER on Table 3 of the article (for the PLDA system)
corresponds to the one, where the full training set is used, and might
similarly be obtained as follows:$ ./bin/bob_compute_perf.py -d /PATH/TO/MULTIPIE/OUTPUT_DIR/U/plda_subworld_76/scores/scores-dev -t /PATH/TO/MULTIPIE/OUTPUT_DIR/U/plda_subworld_76/scores/scores-eval -xNoteIf you compare your obtained figure with the Figure 2 of the published article,
you will observe slight differences. This does not affect at all the global
trends and conclusions shown in the article. This is caused by two different
aspects:The features for the paper were generated using a version of Bob that is
unofficial (which means older than the first official release), whereas the
features currently generated rely on Bob 1.2.0. Many improvements were
performed in the implementations of the preprocessing techniques (Face
cropping and Tan Triggs algorithm) as well as in the LBP implementation.The order of the files obtained (and now sorted) from the database API.
For instance, when applying PCA, the input matrix will be different depending
on the order of the file used to build this matrix.Baseline 1: PCA on the LBP histogramsThe LBP histogram features were used in combination with the PCA
classification technique (commonly called Eigenfaces in the face
recognition literature).This involves three different steps:PCA subspace trainingModel enrollmentScoring (with an Euclidean distance)The following command will perform all these steps:$ ./bin/toolchain_pca.py --n-outputs 2048 --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/NoteEquivalently, this can also be achieved by running the following
individual commands:$ ./bin/pca_train.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph --n-outputs 2048 --pca-dir pca_euclidean --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/linear_project.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph --algorithm-dir pca_euclidean --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/meanmodel_enroll.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/pca_euclidean/lbph_projected --algorithm-dir pca_euclidean --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/distance_scores.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/pca_euclidean/lbph_projected --algorithm-dir pca_euclidean --distance euclidean --group dev --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/distance_scores.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/pca_euclidean/lbph_projected --algorithm-dir pca_euclidean --distance euclidean --group eval --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/Then, the HTER on the evaluation set can be obtained using the
evaluation script from the bob library as follows:$ ./bin/bob_compute_perf.py -d /PATH/TO/MULTIPIE/OUTPUT_DIR/U/pca_euclidean/scores/scores-dev -t /PATH/TO/MULTIPIE/OUTPUT_DIR/U/pca_euclidean/scores/scores-eval -xThis value corresponds to the one of the PCA baseline reported on
Table 3 of the article (Once more, be aware of differences due
to the implementation changes in the feature extraction process
and algorithm parameters that have not been kept). These results
are obtained for a PCA subspace of rank 2048, which was
found to be the optimal PCA subspace size, when we tuned this
parameter using the LBPH features.NoteIn contrast to what one sentence of the article suggests, we did not
apply the PCA baseline on the dimensionality-reduced PCA features.
This would mean to apply consecutively twice, the same PCA
dimensionality reduction technique, which does not make much sense.
In contrast, we apply this PCA technique to the LBPH features,
tuning the PCA subspace size.Baseline 2: LDA on the PCA projected LBP histogramsThe PCA projected LBP histogram features considered for the PLDA system
were also used in combination with the Fisher’s Linear Discriminant
Analysis (LDA) classification technique (commonly called Fisherfaces
in the face recognition literature).This involves three different steps:LDA subspace trainingModel enrollmentScoring (with an Euclidean distance)The following command will perform all these steps:$ ./bin/toolchain_lda.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/NoteEquivalently, this can also be achieved by running the following
individual commands:$ ./bin/lda_train.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph_projected --lda-dir lda_euclidean --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/linear_project.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph_projected --algorithm-dir lda_euclidean --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/meanmodel_enroll.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/lda_euclidean/lbph_projected --algorithm-dir lda_euclidean --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/distance_scores.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/lda_euclidean/lbph_projected --algorithm-dir lda_euclidean --distance euclidean --group dev --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/distance_scores.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/lda_euclidean/lbph_projected --algorithm-dir lda_euclidean --distance euclidean --group eval --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/Then, the HTER on the evaluation set can be obtained using the
evaluation script from the bob library as follows:$ ./bin/bob_compute_perf.py -d /PATH/TO/MULTIPIE/OUTPUT_DIR/U/lda_euclidean/scores/scores-dev -t /PATH/TO/MULTIPIE/OUTPUT_DIR/U/lda_euclidean/scores/scores-eval -xThis value corresponds to the one of the LDA baseline reported on
Table 3 of the PLDA article (Once more, be aware of slight
differences due to the implementation changes in the feature
extraction process). These results are obtained for a LDA subspace
of rank 64, which was found to be the optimal LDA subspace size,
when we tuned this parameter using the initial LBPH features.Baseline 3: LBP histogram classification with Chi square scoringThe LBP histogram features might be used in combination with a distance such
as the Chi Square distance, to obtain a face recognition system.This involves two different steps:Model enrollmentScoring (with a chi square distance)The following command will perform all these steps:$ ./bin/toolchain_lbph.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/NoteEquivalently, this can also be achieved by running the following
individual commands:$ ./bin/meanmodel_enroll.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph --algorithm-dir lbph_chisquare --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/distance_scores.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph --algorithm-dir lbph_chisquare --distance chi_square --group dev --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/
$ ./bin/distance_scores.py --features-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/U/features/lbph --algorithm-dir lbph_chisquare --distance chi_square --group eval --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/Then, the HTER on the evaluation set can be obtained using the
evaluation script from the bob library as follows:$ ./bin/bob_compute_perf.py -d /PATH/TO/MULTIPIE/OUTPUT_DIR/U/lbph_chisquare/scores/scores-dev -t /PATH/TO/MULTIPIE/OUTPUT_DIR/U/lbph_chisquare/scores/scores-eval -xThis value corresponds to the one of the LBP histogram (chi square)
baseline reported on Table 3 of article (Once more, be aware of
slight differences due to the implementation changes on the feature
extraction process).Summarizing the results as in Table 3If you successfully run all the previous experiments, you could
get a summary of the performances, as in Table 3, by running the
following command:$ ./bin/plot_table3.py --output-dir /PATH/TO/MULTIPIE/OUTPUT_DIR/Reporting bugsThe package is open source and maintained viagithub.If you are facing technical issues to be able to run the scripts
of this package, please send a message on theBob’s mailing list.If you find a problem wrt. this satelitte package, you can file
a ticket on thegithub issue trackerof this
satellite package.If you find a problem wrt. the PLDA implementation, you can file
a ticket onBob’s issue tracker.Please followthese guidelineswhen (or even better before) reporting any bug.