dataesr/scientific-paragraphs-categorization huggingface dataset:
Moreover, it is easier for us to visually control the camera distances, viewpoint locations, and densities since this approach can navigate the camera controller (e. g. , people, robot, etc. ) to move the camera dynamically so that most regions around the object can be covered. Once the training data are collected, we apply a set of post-processing steps such as calibrating the camera poses from the marker to the target object, data augmentation, etc. To encourage the community to adapt camera pose regression methods towards more complex domains, we collected a new benchmark dataset, Industrial10, to reflect the challenges of industrial environments. Industrial10 comprises of training and testing data of 10 industrial objects. To assess the efficiency of the proposed method, five actively used pose regression methods are employed in our experiments.
Evaluations show that the proposed method can effectively improve the camera pose regression accuracy compared to the traditional data collection method. The contributions of this work are summarized as follows:  (1)  We propose a novel training data collection method called WatchPose to improve the performance of camera pose regression in industrial environments. The proposed method is sufficiently general and can be extended to other scenarios. (2) We introduce a new dataset, Industrial10, to spur the computer vision systems community towards innovating and adapting existing camera pose regression approaches to cope with more complex environments.
Camera Pose Data Collection In general, camera pose collection methods can be classified into two ways: Direct and indirect approaches. With direct approaches, camera poses are acquired from markers or physical sensors. For example, Brachmann et al. [28]  collected the ground-truth camera poses via integrating a set of traditional markers densely surrounding the target object. Tiebe et al. [29]  collected the camera poses via a finely-controlled robot arm. In most cases, camera poses are collected indirectly. For instance, the actively used dataset 7Scenes  [30]  was first recorded from a handheld Kinect RGB-D camera  [31]. After that, the ground-truth camera poses were extracted by Kinect Fusion technique  [24]. To generate the Cambridge Landmarks dataset, Kendall et al. [5]  first captured high definition videos from around each scene using a smartphone and then proceeded to extract ground-truth camera poses using Structure from Motion (SfM) techniques  [32].
Alternatively, Wohlhart et al. [20]  first reconstructed 3D models of the target objects and then extracted training data (images and their pose annotations) by rendering the 3D models. More recently, the WoodScape  [18]  dataset were collected from a car mounted with a set of sensors (e. g. , Inertial Measurement Unit,GPS, LiDAR, multiple cameras, etc. ) along selected streets in USA, Europe, and China. Camera poses of this dataset could be extracted by fusion of LiDAR, IMU, and camera sensors. However, these indirect collection methods cannot be properly applied in industrial environments since industrial objects are normally textureless and may possess specular surfaces under strong artificial lights. For direct methods, it is also not feasible to use traditional markers or robot arms as well.
With this, the computation of feature descriptors are not required. Unlike their approach, Gupta et al. [38]  proposed a 3-layer convolutional neural network to regress coarse poses using detected and segmented object instances. However, these algorithms are constrained by the use of RGB-D images to generate the scene coordinate label, which in practice limits its use in industrial environments. To improve it, Kendall et al. [5, 6]  proposed PoseNet that directly regresses the camera pose from RGB images. However, it easily overfitted with its training data while its localization error on indoor and outdoor datasets was an order of magnitude larger. Such limitations motivated a surge of absolute pose estimation methods such as Bayesian PoseNet  [39] , MapNet  [7] , LSTM-Pose  [40] , Hourglass-Pose  [8] , SVS-Pose  [9] , BranchNet  [10] , and NNnet  [11]  to involve deeper encoders, better loss functions, and more sensor data.
Industrial10 To the best of our knowledge, there is no existing dataset specially designed for evaluating camera pose estimation in industrial environments. For this, we collected a dataset containing 10 industrial objects using our proposed WatchPose approach. As shown in Figure  9 , the main purpose of selecting these objects is to reflect the real challenges from industrial environments. For example, Object1 and Object10 have similar appearances under certain viewpoints that may confuse object detection algorithms. Object6 has limited training data since it is located in a narrow environment. Since Object9 is relatively big, only part of its appearance was captured with small camera distances. Object7 is occluded by a green pipe so its appearance could look different depending on camera pose.
For each object, we collected the training data from three camera distances: 0. 5, 1. 0, and 2. 0 m (Figure  8 ) using the proposed WatchPose approach. Figure  10  presents an example depicting the viewpoint distributions of Object1 with different camera distances. We can observe that most regions around Object1 are covered. The number of original training data for the 10 objects is also shown in Figure  9  after their respective names. For testing, each object has a set of 200 testing images that are randomly collected within 0. 5 to 2. 0 m camera distances. The Industrial10 dataset is publicly available to the community (Please check congyang. de for more details). In addition to the Industrial10 training data collected with WatchPose (named Industrial10-WatchPose), we also extracted another set of training data, named Industrial10-Traditional, using the traditional data acquisition approach.
Specifically, Industrial10-Traditional was collected by capturing a high definition video around each object from camera distances 0. 5, 1. 0, and 2. 0 m. In total, 3 × 10 videos were recorded and each video was then sub-sampled at 2Hz to generate its frames. These images are then inputted to the SfM pipeline to extract the camera poses. For fair comparison, two post-processing steps were followed: (1) We converted the camera poses from SfM to the same format as of Industrial10-WatchPose based on the pre-measured datum line. (2) We uniformly selected the same number of training images according to the training data distribution in Industrial10-WatchPose. For example, Object1 has 1793 training images in both Industrial10-WatchPose and Industrial10-Traditional datasets.
Ablation Studies of WatchPose To fully assess the proposed Watchpose scheme and its properties, we performed a set of experiments by fixing the pose regressor to PoseNet  [5]. Specifically, we first evaluate and compare the trained PoseNet using original images from both Industrial10-Traditional and Industrial10-WatchPose. The main purpose is to show the performance improvement using our proposed method against the traditional method of collection. We also evaluated the usability of our proposed data augmentation strategy on both datasets. It should be noted that the β value in PoseNet's loss function should be calibrated based on the scenario. As reported in  [6] , different β brings considerable performance gaps. Based on the preliminary experiments and discussion in  [5] , β is mainly influenced by the camera location unit and the scene type (i.
Original Images Here, we compare the pose regression performance using the original training data from Industrial10-WatchPose and Industrial10-Traditional. In other words, the images were not augmented. The mean results of each object are detailed in Table  2. We can clearly observe that our proposed WatchPose scheme significantly improved the pose regression performance in this challenging dataset by around 4. 7 times for location error and 3. 9 times for orientation error, compared to the traditional data collection approaches  [5, 7]. Based on Industrial10-WatchPose, we also experimentally verified the necessity and usability of image inpainting for pose regression in our scenario. Sample images before and after inpainting are shown in Figure  13a,c  and b,d , respectively. Particularly, we applied experiments with cross combinations of with-inpainting and without-inpainting for training and testing.
Augmentation In this section, we experimentally explore the efficacy of the data augmentation strategy introduced in Section 3. 3 on the Industrial10-WatchPose and Industrial10-Traditional datasets. Following the rotation augmentation introduced in existing reports  [6, 39] , all images were first rotated 4 times. After that, we empirically enriched each training data 4 times within each camera distance interval. For instance, augmenting 4 times within the 0. 5 m∼1. 0 m camera distance interval meant that the camera distances 0. 6, 0. 7, 0. 8, and 0. 9 m were imitated by performing zooming-in and center cropping from the 1. 0 m image. Evaluations were done on the same testing data as in Section 4. 1 and the result are detailed in Table  4. We can observe that the improvement of pose regression accuracy using augmented Industrial10-Traditional was limited, around 4%.
In contrast, the pose regression accuracy improved around 5. 3 times in location and 4. 2 times in orientation using the augmented Industrial10-WatchPose. The main reason was that the uncovered regions by the traditional approach were still not properly covered after augmentation. As a result, the performance gap between the traditional and proposed WatchPose approaches was enlarged after data augmentation. Specifically, we achieved errors of around 13. 3 mm for location and 4. 7 • for orientation using Industrial10-WatchPose after augmentation, which is around 19 times better than Industrial10-Traditional. In Figure  14 , we plot the pose estimation results in each iteration during the training process using augmented Industrial10-WatchPose. Horizontal axis represents the iteration numbers and vertical axis represents the estimated median performance with respect to camera location and orientation.
Dense Control In this section, we quantitatively evaluate the influence of viewpoint dense control parameter λ in Section 3. 3 to the final pose regression performance. To this end, we employ Object1 (λ = 0. 2) in the Industrial10-WatchPose dataset for the experiment. We also fix other factors such as β in PoseNet, the augmentation strategy, and the testing data. With the increase of λ value (from 0 to 1), original viewpoints of Object1 were proportionally and randomly selected from the virtual ball to imitate the data collection from dense to sparse. For λ = 0, 20% of the original viewpoints were randomly selected and duplicated to imitate the overlapped views. The corresponding viewpoints of original images were selected (or duplicated) to generate different training sets. Table  5  presents the pose estimation results with different λ values (the original number of training images before augmentation is also provided for reference).
Deep Absolute Pose Estimators Using the augmented Industrial10-Traditional and Industrial10-WatchPose datasets, we compare the pose regression performance of five existing approaches. Specifically, Bayesian-PoseNet  [39]  was released by the same authors of the PoseNet approach. Bayesian-PoseNet first generates some samples by dropping out the activation units of convolutional layers of PoseNet based on a probability value. The final pose is then computed by averaging over the individual samples' predictions. Meanwhile, PoseNet+  [6]  introduced a loss with learned uncertainty parameters (learnable weights pose loss) for optimizing PoseNet. Hourglass-Pose  [8]  focused on optimizing the architecture of PoseNet by suggesting an encoder-decoder architecture implemented with a ResNet34  [48]  encoder (removing the average pooling and softmax layers).
In contrast to the aforementioned approaches, a more recent method called MapNet  [7]  suggested to include additional data sources in order to constrain the loss. It is trained with both absolute and relative ground truth data. On the Industrial10-Traditional dataset, we analyzed the estimation performance of each object category so that deeper observations could be made. The final results are detailed in Table  6. We found that the performances varied among different objects. We also observed that the improved optimization of loss in PoseNet+ and the architecture design in Hourglass-Pose both improved PoseNet's accuracy by around 2 times, particularly in the location errors which were dramatically reduced. Overall, MapNet achieved the best performances, around 3 times better in location and 2 times better in orientation, compared to the poses estimated by the original PoseNet.
In contrast to Table  6 , the estimation errors in Table  7  are smaller among 5 approaches using the Industrial10-WatchPose dataset. This is because the WatchPose data covers more camera poses thereby leading to better generalization in pose regression models. In particular, while the Bayesian-PoseNet only marginally improved the pose regression accuracy (over PoseNet), the Hourglass-Pose and PoseNet+ approaches achieved around 14% improvement in location and 4% in orientation estimation. Once again, MapNet achieved the best performance, with around 226. 187 mm in location error and 57. 551 • in orientation error. It also shows that the proposed WatchPose method of collecting data enabled much higher in both location and orientation performance improvements on MapNet compared to the traditional data collection method.
Conclusions In this paper, we introduced a simple but efficient data collection method for complex industrial environments named WatchPose so as to learn effective absolute camera pose regression models. The proposed method integrated the advantageous properties of marker tracking and viewpoint visualization approaches. The features of WatchPose could properly handle the challenges of industrial-type objects: Textureless and specular surfaces under strong artificial lights, and highly variable distances and views angles. We also proposed two post-processing steps (inpainting and augmentation) to improve the robustness and stability of the trained models. To advance pose estimation research in industrial environments, we introduced a new challenging dataset, Industrial10, to represent the aforementioned challenges of industrial-type objects and environments.
Experiments showed that the proposed WatchPose method could effectively improve the pose regression performance of five widely-used approaches. In the future, we propose two further directions. Firstly, we will collect more data to cover other kinds of challenges in industrial environments. For instance, we can enrich the training data by varying an image brightness  [52]  to imitate different lighting conditions. Secondly, we will compare the motion tracking performance between ARToolKit and ARCore  [53] , which can be applied with and without markers, respectively. Finally, we will release further baselines for the Industrial10 dataset using a few other camera pose estimation methods such as SfM  [32] , 3D Scene  [30] , etc.
In this article, we evaluated and compared the performance of four encoder-decoder CNNs; FCN, SegNet, DeepLabV3+ and U-Net in segmenting prostate T2W MRI into whole gland (WG), or into three zones PZ, TZ, and CZ depending on availability of dataset label. The training and testing of the CNNs are conducted with selected image pre-processing techniques, data augmentation techniques and selected loss function for dealing with the issue of class imbalance. The paper is outlined as follows. In Section 2, we reviewed papers related to image segmentation focusing on atlas-based segmentation, model-based segmentation, deep learning-based segmentation, and patch CNN-based segmentation. This section also includes a summary of our contribution. Next, Section 3 presents materials and methods, and Section 4 presents results and discussion.
In  [29] , Ronneberger et al. proposed U-Net model, which is proposed for medical imaging segmentation tasks. The U-Net model concatenates the contracting side (encoder) and expanding side (decoder) to compensate the loss of spatial resolution due to the max pooling operations. In another work by Milletari et al. [30] , the Dice loss function was used to solve the problem of class imbalance between background and foreground pixels and obtained accurate 3D segmentation of the prostate T2W MRI using U-Net architecture. In the work by Yu et al. [31] , residual connections are incorporated into the convolutional and deconvolutional part of U-Net and the proposed architecture won the first place in 2017 Computer-Assisted Intervention Conference using the PROMISE12 challenge dataset. In recent work by Rundo et al.
[32] , a new segmentation method based on U-Net was proposed. The CNN network known as Use-Net introduced Squeeze-and-Excitation (SE) blocks in both encoder and the decoder of U-Net architecture. The network was trained and tested on multi-dataset allowing for both intra and cross dataset generalization. Their findings revealed that the SE blocks enable adaptive feature recalibration, hence, providing excellent cross-dataset generalization. Patch-based CNN segmentation methods were proven to result in better pixel-wise segmentation of biomedical images  [33] [34] [35] [36] [37] [38] [39]. In  [33] , Zhang et al. applied patch-based CNN for segmentation of the multi-modality MRI brain dataset, T1, T2, and fractional anisotropy (FA) MRI. The brain region was segmented into white matter, grey matter, and cerebrospinal fluid.
Material and Methods In this section, we cover the methodology for training and testing of the CNNs in segmenting the whole prostate gland, peripheral zone, and central gland from T2W MRI of two different datasets. Firstly, a brief description of the four CNNs architecture is discussed. Secondly, the details of the two datasets used in this work are described, covering the scanner types, image dimensions, and class labels. In the third section, methods for image pre-processing, data augmentation, and patch extraction are discussed. Next, the technique of class balancing is presented as a measure to prevent the network from being biased toward classes with a larger number of pixels. Then, the weighted cross-entropy loss function used in the optimization of the network parameters is presented.
Deep CNN Architectures for Prostate Segmentation The following deep CNN models which follow encoder-decoder structure are selected for comparative performance evaluation of semantic segmentation of prostate T2W MRI; FCN  [24] , U-Net  [29] , SegNet  [40] , and DeepLabV3+  [41]. A brief description of each CNN models is presented in the subsequent paragraphs. Fully convolutional neural network (FCNs)  [24, 42, 43]  is based on existing image classification network such as AlexNet  [44] , VGG16  [45] , and GoogleNet  [46]  to serve as the encoder module (downsampling path) and appended with a decoder module (upsampling path) with transposed convolutional layers. The encoder module is built from local connected layers such as convolution, pooling, and upsampling without any dense layer in the architecture.
MRI Dataset Two MRI datasets are used for evaluation of the CNN architectures, from (1) NCI-ISBI 2013 challenge and (2) Universiti Kebangsaan Malaysia Medical Centre (UKMMC). The summary of the two datasets is as follows: 1. NCI-ISBI 2013 is a prostate 3T collection of 40 patients having 542 T2-weighted MR slices available at the Cancer Imaging Archive (TCIA) site  [52]. The images were collected at Radboud University Medical Center, Netherlands, using Siemens TIM 3 Tesla scanners  [53]. This dataset has three different image sizes, 384 × 384, 320 × 320 and 256 × 256, having a thickness of 4 mm. The image masks are labeled to 3-class, peripheral zone (PZ), Central Gland (CG), and background.
2. UKMMC dataset is a prostate 3T collection consists of 11 patients having 229 T2-weighted MR slices that are collected from UKMMC. The images were acquired using a 3 Tesla Siemens TIM MRI scanner with a surface coil. The image dimensions are 384 × 384 and 320 × 320, with a thickness of 3 mm. The image masks are labeled to 2-class, whole prostate gland (WG), and background.
Image Resizing The images of NCI-ISBI 2013 dataset come in three sizes, 384 × 384, 320 × 320 and 256 × 256, whereas, in the UKMMC dataset, the images come in two sizes, 384 × 384 and 320 × 320. Therefore, in order to achieve common image size, image resizing is performed by scaling the images to a size of (320 × 320)-pixel using the nearest-neighbor interpolation method  [54].
Patch Extraction Training DNNs for accurate prostate segmentation requires large datasets, hence we used a patch-wise framework. The four different encoder/decoder networks are evaluated in terms of DSC for different sizes of patches, including 64 × 64, 128 × 128, and 192 × 192, similar to the work of Sekou et al. [59]. It is observed that the performance of the CNNs decreases in the smaller patch sizes because of the small prostate region compared to the background region. In smaller patch sizes of 64 × 64 and 128 × 128, the prediction of the prostate gland is lower than in higher patch size of 192 × 192. This explains why the CNN architectures will result in better segmentation in the case of (192 × 192)-patch than the (64 × 64)-patch and (128 × 128)-patch. In our case, from each MRI slices, 20 and 30 (192 × 192)-pixel random patches were generated from NCI-ISBI 2013 and UKMMC dataset, respectively.
For NCI-ISBI 2013, 460 images were selected for training from a total of 542 MRI slices, giving 9200 images, a 20-fold increase in the number of training images. On the other hand, since the UKMMC is a relatively small dataset, more patches are needed to be generated from this dataset. From 176 MRI slices, patch extraction has generated 5280 total training images, which is a 30-fold increase from the original 176 slices. A sample of patches generated from one of the datasets is shown in Figure  3.
Class Weight Balancing It is common in medical volume scan that the anatomy of interest occupies only a small region of the scan, like in the case of prostate MRI. As a result, the number of pixels for image background is higher than the prostate gland, resulting in imbalance class labels. Specifically, for the NCI-ISBI 2013 dataset, the ratio of pixels for background to central gland and peripheral zone is 95. 5:1. 9:2. 6, whereas, in the UKMMC dataset, the ratio of background to whole gland is at 94. 94:5. 06. Class imbalance can be detrimental for the learning process as the network learns more on dominant classes and biases towards these dominant classes. To improve the learning process and avoid being trapped in local minima, the class weighting approach can be used as a systematic way to improve the low dominant class  [60].
To counteract class imbalance present in the dataset, larger weights are assigned for labels with less total pixels and smaller weights for labels with more total pixels. If the number of pixels in a particular class is denoted as N c , where c corresponds to background, CG and PZ for 3-class dataset, and c corresponds to background and WG for 2-class dataset. Hence, N c is (1 × 3)-array and (1 × 2)-array for 3-class dataset and 2-class dataset, respectively. Let T represent the total number of pixels in the images, then the (1 × 2) or (1 × 3)-array of image frequency, F c , of a class is the ratio of N c to T given by F c = N c T. ( 1 ) From here, the (1 × 2) or (1 × 3)-array of class weight, W c , for a set of training data can be calculated by finding the ratio of median of F c to F c W c = median(F c ) F c.