## 2 Materials and Methods
### Datasets
Briefly, their automated segmentation method involves the application of two standard segmentation techniques, thresholding [37] and watershed [17], the latter of which is a built-in function of the image analysis toolbox of MATLAB. To segment an image, they first produce a series of binary images corresponding to the thresholded image at every threshold between zero and the image bit depth. After applying the watershed algorithm, these image matrices are then summed and re-scaled to produce a composite binary image that is the most correct across all thresholds, and then the watershed algorithm is applied again to segment cells in this composite image. In their method, this automated segmentation approach is first applied to the last time frame of the experiment. The user can then use the GUI CTseed to manually correct errors in the segmented image by manual adding or deleting watershed lines by clicking on areas of the image with a computer mouse. Thereafter, the user initiates cell tracking using CTtrack, which assumes that the manually corrected image of the last time frame is the best guide to optimize backwards tracking of automatically segmented cells from earlier time frames. In our application of the method to obtain the ground-truth dataset, we did not use cell tracking. We first applied their automated segmentation to the image of each time point, and we then used their CTseed GUI to each image time point as if it was the last time point, which allowed manual correction, and perfection of the cell label matrix of every time point. In addition to our dataset, images were also taken from two datasets in the Yeast Image Toolkit. The Yeast Image Toolkit (YIT) was created by the authors of CellStar [21]; they created the ground-truth labels for the images in the YIT datasets which were provided by the Batt & Hersen lab [38]. Images were taken every 3 min with a 50 ms exposure. A 100\(\times\) oil immersion objective (PlanApo 1.4 NA; Olympus) was used with an Olympus IX81 inverted microscope. Images were taken at a resolution of 512 \(\times\) 512 with a QuantEM 512 SC camera (Roper Scientific) using the \(\upmu\)Manager [39] plugin for ImageJ [40]. Cells were fixed and grew in a monolayer and did not reach confluence. YIT Dataset 1 consisted of 60 frames with a cell count starting with 14 cells and growing to 26 cells. YIT Dataset 3 consisted of 20 frames with a cell count starting with 101 cells and growing to 128 cells. These datasets were chosen due to their qualitative and quantitative differences. The associated ground-truth for these datasets does not include cell masks so we generated ground-truth masks using ImageJ [41]. The ellipse tool in Fiji was used to manually select a region of interest (ROI) for each cell in every image. The union of the selected ROIs in an image was saved as a binary mask. Additional details can be found on the Yeast Image Toolkit website ([[http://yeast-image-toolkit.org/pmwiki.php](http://yeast-image-toolkit.org/pmwiki.php)]([http://yeast-image-toolkit.org/pmwiki.php](http://yeast-image-toolkit.org/pmwiki.php)), accessed: 8 December 2019). To standardize images from different datasets prior to network inference, images were normalized on a dataset-level. The mean and standard deviation of the training set for an individual dataset were taken and used to zero center and normalize the images in the test and training set images of that dataset. Images were also re-scaled between 0 and 1.