every cell in every single image. (2) Cell tracking, i.e. identifying and following every cell over time, thereby reconstructing their temporal continuity. Cell detection can be performed with a wide range of known image processing methods, such as level set [10], wavelet [11] or threshold segmentation methods [12] and contour-based methods [8,13]. Ultimately, all these methods decide for each pixel of a given image whether it belongs to a cell or not. Once the cells are detected, the second step, cell tracking, involves search strategies, by which a given cell is identified in subsequent images. Simple tracking methods look for the nearest cell [14,15], while more complex approaches use specific models to describe the similarity of a cell with itself, such as image registration [16] or smooth shape transitions (active contours [10,17,18,19,20]) and markov models based on simultaneous changes of a variety of cell features [14]. These methods share the assumption that cell behaviour - arguably with the exception of mitosis - is a smooth, continuous temporal process. Despite this considerable range of tracking algorithms, the most reliable results are still achieved through manual tracking [1,5,6]. In contrast to human inspection, none of the automated tracking approaches to date controls the reliability of its output. Yet, cell tracking has a particularly fatal error propagation mechanism. A small local error, like e.g. a single undetected cell, may lead to large deviations of subsequent genealogic assignments and genealogic tree topology. Consequently, even robust tracking algorithms with very low error rates can produce substantial amounts of false results, if their reliability is not assessed independently. In this work, we introduce and discuss two distinguished concepts to asses, how ‘‘true’’ tracking results are: validation and evaluation. The term validation refers to an error detection strategy which scrutinizes the intrinsic structure and logic of tracking results. Evaluation, on the other hand, relates to a method for scoring tracking results with respect to a reference data set. Here, we have devised a novel validation algorithm to find, classify and partially correct errors which may occur during tracking. These errors involve detection as well as tracking errors and the connection between them (error propagation). We give a complete classification of all possible error scenarios, alongside with a systematic strategy to find them. Turning this approach into an algorithm rejects all untrustworthy paths, leading to reliabilities of .95% (correct paths), resp. .96% (correct genealogic assignments). This validation works independently of the tracking algorithm itself and could in principle be used to validate the results of any arbitrary tracking approach. In fact, we will demonstrate that validation is mandatory to all cell tracking approaches. The high reliability of the validated tracking results allows for a fully automated extraction of biological meaningful parameters. We have e.g. automatically determined the life-time distribution of pancreatic stem cell cultures and the life-time of sibling cells (symmetry of cell divisions) with unprecedented statistical accuracy. A second point of this work pertains the fact that until now no common benchmark system exists to test and compare the different detection and tracking methods. In fact, many of the published tracking methods have not been assessed for their reliability on real cell images at all [7,21,22]. This makes it difficult to decide which method is appropriate for a given tracking problem. More general, comparability is a fundamental prereq- uisite to establish a scientific discourse. We therefore provide two large sets of real cell tracking data, from which a complete Automated Tracking Turns Cell Culture into Numbers reference set of detection and tracking results has been manually derived by scientists. These data have been used to evaluate the particular validation algorithm of this work. The reference data sets can be freely used by the community to assess the strengths or disadvantages of different tracking approaches or to extract and try new descriptors for cell behaviour. Thereby, we hope to establish a means for comparing and, ultimately, improving cell tracking methods. Material and Methods I) Cell culture Adult stem cells from exocrine parts of the pancreas of rattus norwegicus were used throughout all tracking experiments. The cells were isolated as described in Kruse et al [23]. These pancreatic stem cells are characterized by a high proliferation rate (doubling time of roughly 20 hours) and the propensity to spontaneously give rise to cell types from different germ layers [24]. They were cultured in polystyrene dishes (TPP, Switzerland) using DMEM medium (Gibco, USA), with the addition of 10% (v/v) fetal bovine serum (FBS "Gold", PAA, Germany) and 1% (v/v) penicillin/streptomycin (PAA 100x,Penicillin 10.000 Units/ml, Streptomycin 10 mg/ml). The cells were in passage 48 (data set A), in passage 39 (data set B) and passage 31 (data set C, see Figure S1). Additional experiments have been carried out with human dermal fibroblasts in passage 7 (data set D, see Figure S2 and Videos S15, S16, S17, S18). The in vitro cell population was established through explant culture from the isolated dermis and cultured similarly to the above described PSCs (DMEM, 10%FBS); the cells were in passage 7. II) Soft- and Hardware All time lapse experiments were carried out using an inverted Olympus ix-81 microscope with integrated climatization control and a motorized x/y stage. The z-stage is also motorized and compensates automatically for slow drifts of the focal plane (maximum contrast auto focus). The microscope is equipped with a CCD-camera (F-View FireWire Camera, Olympus Soft Imaging Solution) for image-acquisition and has an integrated motorized condenser and light path changer. To control the image-acquisition in long-term experiments Olympus’ CELL‘M software was used. Cell-Culture dishes were placed in a climate-controlled chamber at 37uC with 5% CO2 and about 50% humidity to prevent evaporation of the culture medium. Computational parts of the experiments were done using an Intel(R) Core(TM)2 Duo machine at 3GHz with 8GB RAM. For software-development MATLAB R2008b was used. Tracking and validating our reference data set A (209 images containing .244.000 cells) takes less than 1.5 hours on this machine. Thus, if the image acquisition rate does not exceed 2images/minute, the algorithm can be used in real time on a regular desktop computer (order of 1000cells/image;). The algorithm scales roughly linear with the number of tracked cells. The source code of the tracking and validation algorithms is provided in the supplemental material (Link). III) Image Acquisition Images were acquired using an Olympus UPLFLN4XPH 4x phase-contrast objective. The raw image data consist of 12bit gray scale images with a size of 137661038 pixels (pixel resolution 1.6mm) recorded every 10 minutes (data set B) or 15 minutes (data set A) over several days. PLoS ONE | www.plosone.org 2 November 2011 | Volume 6 | Issue 11 | e27315