dataesr/scientific-paragraphs-categorization huggingface dataset:
2. B. Application areas The areas of application of the benchmark relate to the field of oncology. Image types are expected to reflect the state-of-the-art in diagnostic imaging and treatment management and to adopt the most recent digitized histopathology methods and bespoke phantoms. The benchmark should be easily extendable to satisfy the needs of more application areas according to the availability of new data and new technology. The types of Volumes of Interest (VOI) included in a standardized evaluation protocol should at a minimum include disease sites established for using PET in radiotherapy treatment planning. Best estimates of reference contours (RC) may originate from various sources depending on the image type included in the dataset. We distinguish between the following types of RC: (i) Absolute truth: only available for simulated images.
(ii) Single 'best' estimates: surrogate of truth provided for physical phantom images and in the special case of patient images for which histopathology data are available. In physical phantom images, the optimal threshold in simultaneous CT images provides a uniquely best estimate but cannot be considered the absolute truth because of threshold uncertainty arising from partial volume effects and potential misalignments between PET and CT datasets. The accuracy of RC data for patient scans provided by pathology examination of excised lesions is limited due to specimen deformation during processing and Medical Physics, 44 (8), August 2017 possible metabolic changes between the time of scan and the time of excision. 7 (iii) Multiple equally 'best' estimates: they can be provided in the form of consensus manual expert delineations when no single delineation can be considered to be the best.
This functionality is expected to ultimately be related to treatment outcome and is not implemented in the first release of this benchmark. The robustness metrics should include, as suggested by Hatt et al. ,  9  the analysis of the sensitivity to the following variations: (i) across datasets, governed by differences in anatomy and physiology as reflected by the image characteristics, (ii) within a dataset, resulting from natural differences in gross tumor volume shape/size between different patients, and (iii) within an image, according to differences in image reconstruction and noise levels across different realizations of that image.
2. D. Recommendation for standard requirements The following components and functionalities are desirable for the benchmark software: A1: Open access to the software and functionalities that are understandable by both developers and users of the segmentation methods. A2: Collection of datasets representing the clinical applications requiring validation of PET-AS methods. A3: Carefully selected images and RCs checked to only include cases that can provide meaningful metric values. A4: Capability to allow future extensions by adding new datasets. A5: Implementation of PET-AS methods representing the current state-of-the-art, as described in ref. 6. A6: Capability to facilitate the addition of an algorithm that is developed outside the application's framework and that can be used to segment images and produce contours in a format compatible with the benchmark analysis routines.
2. E. Evaluation of the benchmark The evaluation of the benchmark aims at addressing the following question: how good are the chosen datasets and metrics at quantifying and comparing the performance of PET-AS methods? The implicit hypothesis is that the collection of images and metrics provided with the benchmark is appropriate and allows comparing PET-AS methods with enough accuracy and precision to distinguish between the methods and identify their specific strengths and weaknesses. Medical Physics, 44 (8), August 2017 To validate this hypothesis, we need to test both the Discriminative power (DP) and specificity of the benchmark for differentiating PET-AS methods. DP, which here relates to the ability to distinguish between two PET-AS methods that are close in performance, can be tested by evaluating the range of variation of the performance metrics across the range of PET-AS methods and images.
RESULTS This section summarizes the approach taken to implement the standard with the PETASset software while satisfying aims A1 -A9 and the recommendations given in Section 2. The PETASset code was written in the Matlab language (The Mathworks Inc, Natick, USA), including the Image Processing Toolbox. PETASset was implemented as a plugin to the Computational Environment for Radiotherapy Research (CERR) software. 10 This enabled using CERR's capabilities for handling and displaying Digital Imaging and Communications in Medicine (DICOM) data, as well as dealing with Radiotherapy Treatment (DICOM-RT) data. PETASset reference data are stored and distributed in a Matlab structure saved in the CERR file format, and are compatible with all the tools available in CERR. This format is also readable by any other Matlab-based application.
An application programming interface that can be used to read Matlab formatted data in external environments is also available and is described elsewhere. † PETASset is freely distributed along with CERR, for which user documentation is easily accessible, ‡ in line with A1. The folder structure of the PETASset package and a short description of its content are given in Fig. 1. A detailed description of PETASset's content and functionalities is given in the following section.
3. A. 1. Datasets With reference to Fig. 1 , the "Data" folder contains the image datasets (cases) distributed with PETASset. These are provided in compressed CERR file format. The cases include phantom and patient scans for three anatomical sites: H&N, lung, and pelvis. Reference contours were generated using (a) histopathology specimen, (b) simulation templates, or (c) another imaging modality (e. g. , CT). The reference contours are considered to be the best estimate of RCs for the cases provided with PETASset. A total of 66 PET studies with RCs from four different research centers are currently included in the PETASset database. The cases included in each dataset are reported in Table I and described in more details below: • UCLPTLU 11 : 10 clinical PET/CT lung cases, with two PET scans corresponding to different spatial sampling, i.
e. , different voxel sizes. • UCLPTHN 12 : seven clinical PET/CT H&N cases. • MILPPAB 13 : 11 cases corresponding to successive acquisitions of the same physical body phantom. • BRENPLU 14 : two simulated PET lung study generated with the GATE simulation toolkit. 15 BRENPHN 14 : six simulated PET H&N cases generated with the GATE simulation toolkit. • SIM 16 : a total of 30 simulated PET scans, 10 for each of the sites: pelvis (SIMPTAB), lung (SIMPTLU), and H&N (SIMPTHN). Each dataset contains two reconstructions using the Ordered Subset Expectation Maximization (OSEM) algorithms and OSEM + point spread The target volumes were chosen in line with requirement A2, with a focus on lung and H&N cancer. Cutting-edge histopathology and tissue heterogeneity data, modeled in the simulated datasets (BRENPLU, BRENPHN, SIM) were also included.
3. B. Workflow and analysis The workflow implemented in PETASset is shown in Fig. 3  and includes:  (i) Image and VOI visualization (ii) Image segmentation (iii) Data analysis and evaluation of performance metrics (iv) Structured reporting The input to PETASset is a set of contours the accuracy of which has to be evaluated. The contours can be generated using the (a) default PET-AS methods provided with PETASset, (b) the segmentation module available in CERR or (c) custom Matlab code (cf. Fig. 1 ). A number of different analyses can be carried out in PETASset as shown in Fig. 4. Depending on the used dataset and evaluation metric, a given PET-AS method can be tested in terms of absolute accuracy and/or in terms of robustness to a specific reconstruction parameter. For instance, testing a PET-AS method on UCLPTLU data with Level I and II metrics provides a quantitative measure of its performance on clinical lung tumor data.
In addition, the robustness of a PET-AS method to different image acquisitions of the same subject can be assessed using the MILPABB dataset as the standard deviation of results obtained for the same lesions over the different acquisition instances available. All evaluation strategies, represented by Level I-II metrics, operate on three types of data: • PET image • User-generated contours • Reference contour It is worth noting that in PETASset all RCs and PET-AS contours in each study are defined in the same frame of reference and in the same co-ordinate grid as the image space and resolution of the PET data are used. The evaluation strategies implemented in PETASset are described in Section 3. C. 2.
3. C. 1. Segmentation The functions in the Segmentation category are used by the PETASset segmentation tool to segment images and generate contours. The PETASset segmentation tool is accessible through a Graphical User Interface (GUI) which allows visualization and segmentation of the current PET scan. Once the PET scan is selected, PETASset automatically displays axial, sagittal, and coronal views of the volume on which the RC is defined. The segmentation algorithm can be then chosen from a drop-down list. The list includes the PET-AS methods provided by default with the PETASset package plus any additional custom algorithm implemented by the user. The result of the image segmentation can be displayed and saved together with the information on algorithm and parameters used. The PET-AS segmentation methods implemented by default in PETASset include: • fixed threshold (percentage of maximum) • fixed threshold (absolute threshold) • region growing • statistical clustering The methods were selected from common thresholding and advanced methods as defined by Hatt et al.
3. C. 2. Analysis The functions in the Analysis category implement metrics used for the evaluation of the agreement between PET-ASgenerated contours and the RC. The code files are grouped in two folders: Levels I and Level II, which correspond to different metrics that can be launched independently from the PETASset menu. The PETASset analysis tool is also accessible through a GUI that allows the user to select the PET-AS contours and the metrics to use in the study. The results of the analysis are automatically saved to file and can also be shown in tabular format. Level I and II metrics are described in the next sections. Level I: Level I metrics are used to provide basic and essential information on the delineated VOI. The metrics include: • Volume • Mean uptake value • Maximum uptake value • Centre of mass Each Level I metric can be used to quantify the deviation between the PET-AS and the RC.
The range of values that can be achieved is indicated for each metric. The metric in Eq. 8 is a variant of the Hausdorff distance (averages replaces the maximum). This formulation was implemented in PETASset following the results of Dubuisson et al. 17  showing its superiority compared to the original algorithm in quantifying the similarity between two contours. The DUV in Eq. 9 is the cardinality of the volume between the reference and test surfaces. From DUV, one can derive the average delineation uncertainty in voxels (or cm knowing the voxel dimensions) as the average thickness of this volume. An example of Level II analysis performed on a single study (1 RC) of the UCLPTLU dataset is given in error in volume ¼ ðjBj À jAjÞ jAj Â 100 ð%Þ (1) error in mean uptake ¼ ðmean uptake ðBÞ À mean uptake ðAÞÞ mean uptake ðAÞ Â 100 ð%Þ error in maximum uptake ¼ ðmaximum uptake ðBÞ À maximum uptake ðAÞÞ maximum uptake ðAÞ Â 100 ð%Þ error in center of mass ¼ kcenter of mass ðBÞ À center of mass ðAÞk (4) Medical Physics, 44 (  8 ), August 2017 Table  II  for metrics DSC, S, PPV, and HD.
It allows one or more PET-AS methods to be evaluated and ranked using different performance metrics across the whole benchmark dataset. The Global Report also provides additional statistics data such as the mean and standard deviation of metric values, for each dataset across all the selected cases. It can also be used for longitudinal studies. The structure of the Local and Global Reports is the same, except for one additional section which contains the following items: (i) A table reporting the mean and standard deviation of each metric value across the selected PET-AS contours (ii) A table containing mean metric values across cases within each dataset separately (iii) Graphs showing the mean and standard deviation of each metric value across cases within the different datasets grouped by data type (clinical, nonclinical) and tumor site.
In addition, PETASset is able to accurately and clearly rank contours with known and different inaccuracies, for example small variations in volume, which validates its specificity. It should be noted, however, that the specificity will be limited by the accuracy of the volume contour definition in CERR. Furthermore, the specificity, as described here, will vary with the RC size: larger RCs are expected to lead to smaller differences between metric values when testing volumes of similar known volume error. The results given in Table  III  are an illustration of how ALs could be defined, based on the hypothesis that datasets and PET-AS methods implemented in PETASset are representative of the current state-of-the art. In this example, with reference to Table  III , a PET-AS method would be within the ALs for the volumetric error if its absolute mean error in volume across the PETASset VOIs was lower than 59% of the true volume.
4. A. Design and implementation PETASset was designed and built following AAPM TG211 report which identified the need for developing a standard evaluation framework designed for the assessment of both existing and future PET-AS algorithms including those derived from supervised machine learning methods. 6, 23, 24 igure  4  shows the workflows available in PETASset and the large range of information that can be extracted from the current version of the benchmark. It is important to note that, although a number of datasets and evaluation metrics are included, PETASset is not required, and therefore not designed, to evaluate each PET-AS method on all datasets using all metrics. Such a requirement is unrealistic due to different assumptions made about the datasets by different PET-AS methods and metrics.
For example, a PET-AS method may operate on hybrid data, which assumes the availability of both PET and CT datasets. This assumption may hold for a patient dataset but not for numerical or physical phantoms. The imaging and RC data available in the current version of PETASset are intended to represent the state-of-the-art and have been assembled with the contribution of leading clinical and research institutions in the field. The PETASset datasets include RCs for homogeneous (physical and numerical phantoms) as well as heterogeneous tracer uptake (numerical phantom and clinical data). However, we emphasize that in order to be meaningful, the comparison of PET-AS methods developed with the same goal should be performed on the same datasets and according to the same metrics. We therefore recommend that the overall performance of a PET-AS method is evaluated on all the PET-ASset data applicable, for a given segmentation method.
PETASset can provide ALs for all evaluation metrics included in the package. This can be used to compare the performance of new and well established PET-AS methods. The quality and usefulness of these ALs will depend on the quality of data available in PETASset, which will need to be regularly updated. It should be noted that the ALs provided in this document are not recommended for the evaluation of new PET-AS, since they were derived using a small number of PET-AS methods and a limited set of images. Further work is needed to produce task-dependent and reliable ALs for PET image segmentation. The RC is hidden via encryption in PETASset when the software is distributed to the user. Even then, there is a risk that users optimize their segmentation method blindly to increase the accuracy score of certain algorithms.
4. B. Future work The current version of PETASset is a research tool that can be reliably used to evaluate the performance of PET-AS methods against reference RC data. The following additional (i) Web access: provide web access to PETASset data, tools and statistics including reports for selected datasets and segmentation methods. (ii) Level III Analysis: design and implement metrics to evaluate the clinical implications of contour accuracy in radiotherapy treatment planning. 25 It is envisaged that Level III metrics will operate on reference dose maps calculated using PETASset's RC and distributed with the benchmark. (iii) Reference data: the value of PETASset will be enhanced by adding more test data including 4D PET/CT scans and expert consensus VOIs. 26 In particular, including images with highly varying degrees of tumor size, activity, contrast and resolution will enable thorough robustness studies in fulfillment of requirement A8.
It is envisaged that synthetic datasets generated with PETSTEP will also help growing the PETASset database, in particular to include data specifically designed for testing robustness to the partial volume effect. (iv) Imaging modalities: PETASset should evolve to include the next generation of auto-contouring methods that combine information from different imaging modalities such as CT and MRI. (v) Unified performance score: performance metrics are specific and limited to only certain image or contour parameters. PETASset could be used to combine more metrics in a unified score reporting a single performance value. This would be a desirable feature and research toward such a metric is encouraged. (vi) Knowledge-based PET segmentation: continuously adding to the PETASset database standardized data on the performance of different PET-AS methods will enable us to start building models and ALs to use as a baseline for the assessment of new PET-AS algorithms and for the optimal segmentation of virtually every type of PET image.
ACKNOWLEDGMENTS This work was supported in part through  NIH/NCI  grant  R01CA172638  (WL) and  NIH/NCI Cancer Center Support  grant  P30 CA008748  ( CRS, WL, ASK ). BB and ES acknowledge support from  Cancer Research Wales  grant nos. 7061  and  2476. HZ acknowledges support from the  Swiss National Science Foundation  under grant  SNFN 31003A-149957  and the  Swiss Cancer Research Foundation  under grant  KFS-3855-02-2016. JAL is a Research Associate with the  Belgian F. R. S. -FNRS. The work on this report was funded in part by the  American Association of Physicists in Medicine.
The Image Processing, Decision Making and Command (IPDC) Module The IPDC is developed in C++. A Graphical User Interface (GUI) is also designed and developed with the QT Creator software  [33]. It receives data packets from the FEES and constructs an image from this data (cf. Figure  9 ). The IPDC analyzes the received image and compares it with the reference one. The outcomes of this comparison are employed by the IPDC to make decisions. Based on these decisions, the IPDC The devised system configures the camera frame rate and resolution according to the application requirements. The targeted application does not require a high frame rate. Therefore, the 912 KBaud rate is employed in the system. However, this implementation supports much higher baud rates as it is successfully tested up to 1 MBauds per second.
The Image Processing, Decision Making and Command (IPDC) Module The IPDC is developed in C++. A Graphical User Interface (GUI) is also designed and developed with the QT Creator software  [33]. It receives data packets from the FEES and constructs an image from this data (cf. Figure  9 ). The desired object location is marked with a blue circle in Figure  10. Its conveyance trajectory is shown, with the blue arrows, in Figure  10. Each arrow represents a corresponding solenoid valve action command. The process is summarized in Table  1. The solenoid valve action commands are stored by the FEES, in the instruction buffer, in the sequence to be executed. There is a waiting time between each execution of a solenoid valve action command. It is calculated as a function of the solenoid valves' response time, the necessary air pressure accumulation time and the considered object weight.
Conclusions In this paper, a fully, FPGA-based, embedded automatic system is devised for systematically piloting the smart surface  [1] , in order to maneuver the micro-objects. It is a closed-loop system. The feedback, regarding the micro-object's current location, is provided via the FPGA-based embedded camera to the IPDC (Image Processing, Decision Making and Command) module. The IPDC module compares this micro-object location with the desired one and generates actuation commands. Following these commands, the front-end, FPGA-based, embedded system generates actuation patterns for the smart surface. The same process is repeated in a closed-loop fashion, unless the micro-object achieves the desired position on the smart surface. The IPDC module is based on a specific and profiled C++ based development using QT tool and has no dependencies on the commercial software licenses.
The QT tool is compatible with a variety of real-time embedded operating systems such as Linux, IOS, Windows IoT, etc. It ensures the IPDC implementation on an appropriate commercial microprocessor or FPGA-based embedded system. Therefore, the devised solution is independent of a PC-based approach. Moreover, the front-end, embedded system is realized with a specific VHDL-based implementation and it can be implemented on any appropriate FPGA or even an effective ASIC (Application Specific Integrated Circuit) development can be realized. Moreover, a specific, VHDL-based, resource efficient and fully configurable camera driver is employed. The camera frame rate and resolution can be configured via this embedded camera driver with a soft command. This feature is not evidently available with the USB-based standard camera solutions.
Acknowledgments: The planned 2D static air-flow motion system was developed under  SmartSurface  project with the participation of  Réda Yahiaoui ,  Rabah Zeggari ,  Julien Malapert  and  Jean-François Manceau. The authors would like to thank  Joseph Nunes  for his technical assistance and help. The authors are also thankful to the anonymous reviewers for their valuable feedback.
2009 ] implementation that computes parametrization functions whose gradients align with the directions of the cross elds. Finally, the meshes and the parametrizations are given as input to a Quad Extraction (QEx)  [Ebke et al. 2013 ] implementation which robustly extracts the quad meshes associated with the parametrizations. We use the implementations of MIQ and QEx directly and thus we omit further discussion on these methods, and refer the interested reader to the respective papers for additional information. Technical details such as the actual code packages and the parameters we used are described in Section 6. Fig. 2. A quad mesh generated with our method using k = 100 eigenfunctions on the intrinsically symmetric bunny model. We emphasize that we optimize for a consistent cross eld and thus the quadrangulation is only approximately compatible in practice.
IMPLEMENTATION DETAILS We implemented our method using MATLAB and tested it on a Intel Xeon 3. 20GHz processor with 32GB RAM. The problems we consider in Eqs. (  7 ) and (  11 ) could be re-arranged as standard quadratic programming problems, composed of sparse components, E s and E l , and a dense element, E c (see Appendix B). Thus, we were able to use MATLAB's quadprog optimization tool with a user-handle to compute H , where H is the Hessian and is a vector, in order to avoid storing the full dense Hessian. The initial solution was the smoothest power eld (α c = 0 and α l = 0) in all our tests. For problems with 5k/11k/20k/27k/50k vertices, the power eld design part converges in 5/7/13/34/90 seconds with a point to point mapping or in 10/18/68/90/169 seconds using a reduced basis of size k = 100, respectively.