Automated Tracking Turns Cell Culture into Numbers Figure 6. Reference data set A. The first image (t = 0) shows the reference data set in its initial configuration with 282 cells. After 23 h and 31 h, the observed area contained 957, respectively 1297 cells. The last micrograph (t = 48 h) displays 2211 single cells. The blue cells belong to complete paths wich have been accepted by the validation algorithm. doi:10.1371/journal.pone.0027315.g006 detectors. Both data sets, alongside with a detailed description of the data-structures, are downloadable in the supplemental material section for free use by scientists or companies (Creative Commons license, Link to the data sets). Cell detection and tracking of the reference data was carried out manually but with the assistance of our tracking software. A user interface was implemented which enabled the scientists to manually correct the initial tracking results, obtained by the unsupervised tracker. The program also monitors the (bio)logical consistency of the manually corrected tracking results and adjacency matrix (i.e. no duplicate cells and paths; uninterrupted paths; no overlapping cell masks in one frame; mitotic cells with exactly two successors). Using this approach, the cell detection and tracking results were incrementally corrected by three different scientists, i.e. each scientist corrected and improved the result of the previous. The last correction left only very few, if any errors. The final reference data not only specify the correct cell positions and shapes but each single cell is also assigned to the correct branch of its genealogic tree. III) Evaluation of tracking algorithms Comparing different tracking algorithms requires not only a sufficiently large and freely accessible reference data set, but also common criteria to evaluate tracking algorithms. The intent of this section is to propose and discuss general criteria for evaluating tracking algorithms and to illustrate them with our particular tracking data. Below, we introduce three such criteria: (1) cell detection rates; (2) path classification counts; (3) genealogical assignments. Evaluation of Cell detection (I): Cell detection rates. To measure the performance of the proposed cell detection algorithm, the false acceptance rate (FAR) and the false rejection rate (FRR) #TN are used. The FAR is defined by FAR~1{ #TNz#FP #FP #TP ~ and the FRR by FAR~1{ #TNz#FP #TPz#FN #FN ~ . TP and TN describe correct decisions of the #TNz#FP algorithm (TP = true positive and TN = true negative), while FP and FN describe false decisions (FP = false positive and FN = false negative). Figure 7 displays the false rejection rate (FRR, missed cells) and the false acceptance rate (FAR, false cells) of our cell detector for each image of reference data set A. False acceptance/rejection errors are detected analogous to overlap tracking: The intersection between a detected cell and the reference cell mask is calculated. A cell is categorized as correctly detected, if it overlaps with one and only one cell in the reference cell mask and both overlaps (between true and detected cell masks and vice versa) exceed a threshold of 30% (cf. figure 7, inset). On average, our global threshold detector finds 95% of the cells. This number compares well with other detection methods [20,22,29,30]. Figure 7 also reveals that FRR and FAR appear to be significantly correlated; high FRRs are often coupled with high FARs and vice versa. This correlation is a consequence of the fact, that a cell, which is detected with insufficient accuracy (e.g. one overlap ,30%) is counted both, as a false positive and a false negative cell (figure 7, inset). PLoS ONE | www.plosone.org 7 November 2011 | Volume 6 | Issue 11 | e27315