dataesr/scientific-paragraphs-categorization huggingface dataset:
This tutorial guides students through their first AR app built with web technology. Tutorial 11 [••] AR Evaluation. This tutorial uses Google forms for administering three evaluation instruments for AR, namely SPINE, TAMARA, and TRUST, to conduct a test evaluation jointly with the participants. An associated Overleaf L A T E Xdocument uses knitr to integrate the R code required for the analysis, presenting results live and on click of a button. Tutorial 12 [•] AR portals using shaders. The tutorial is a simple introduction to Unity shaders, demonstrating how two shaders can be used to mask 3D objects in a way that they become visible only when looking through a portal window. Tutorial 13 [•] Geo-Location based AR. The tutorial shows how to work with GPS in AR, to create Pokemon-go like apps.
Tutorial 14 [••] Performance profiling. Monitoring and optimising the performance of an AR application is important to ensure a pleasant user experience. Performance profiling of AR applications is not only concerned with the responsiveness of the app. Instead, developers need to look at further measures like the application's framerate. This tutorial introduces performance metrics, performance profiling tools and performance optimization. Tutorial 15 [••] AR Unit Testing. Unity, for example, ships with a test runner that uses the NUnit framework. NUnit is part of the xUnit testing framework collection. It is the official solution by Unity and is actively used by an increasing number of people in the development of modules for the package manager that was introduced in Unity 2018. This tutorial explores the possibilities of the NUnit framework.
Renting expensive special purpose hardware as well as ordering online or shipping smaller and cheaper components are viable options. Additionally, using common technology such as mobile phones with cameras, web cameras, or low-end HMDs are potential alternatives. Also of note are platforms such as Amazon Web Services, which offers remote app streaming services 59 , alongside remote desktop services such as Citrix 60 , which ensures remote access to relevant hardware and software for teaching the curriculum, which potentially opens course participation to a global audience. Given the fact that AR technology is advancing faster than legislation addressing any relevant ethical implications, the AR model curriculum might also cover ethical issues. This can increase awareness as well as empower students in so far as what they wish to see through the technology, what data becomes accessible to whom, and how the technology can affect society as a whole.
While currently, in many places, such issues are covered by ethics clearance processes which are attached to project work (and critical analysis 58 https://github. com/rwth-acis/VIAProMa 59 For example, https://aws. amazon. com/appstream2/ 60 https://www. citrix. com/en-gb/products/citrix-virtual-apps-and-desktops/ requirements for course work), any future curriculum might speak to concerns for AR privacy, security and ethical design. Given a plethora of online AR curricula designs, one must take into account the internationalisation of the curriculum, while recognizing the global spectrum of participants. Consequently, educators should incorporate international, intercultural, and global aspects, at least as far as the AR context of the curriculum, learning outcomes, assessment tasks, teaching methods, and support services of the study program [21] are concerned.
To accurately capture the overlap between the masks, all activation layers of the U-nets are copied and concatenated on each path and sent to two additional convolutional layers before the output activation layers. The method was tested with a dataset of 7563 manually delineated individual buildings of the city of Joanópolis-SP, Brazil. On this dataset, the semantic segmentation showed an overall accuracy of 97. 67% and an F1-Score of 0. 937 and the building individual instance segmentation showed good performance with a mean intersection over union (IoU) of 0. 582 (median IoU = 0. 694).
Semantic segmentation consists of attributing a class to each pixel of an image. This type of segmentation is reasonably well-resolved for building segmentation from very high resolution images. For example, for the Inria Aerial Image Labeling Dataset  [3] , which is a dataset of buildings for 10 cities with varied urban landscapes over two different continents and with two semantic classes, namely "building" and "not building", the most recent models show intersections over union (IoU) of 80. 32% and an overall accuracy of 97. 14% (https://project. inria. fr/aerialimagelabeling/leaderboard/). This is remarkable considering that the validation samples are constituted of cities not present in the training samples  [3]. Among the deep learning frameworks for building semantic segmentation, the U-net architecture has proven to be a standard and has achieved the highest accuracy  [4, 5].
For example, the Mask R-CNN algorithm, which is probably the most widely used method nowadays, consists of two distinct modules: the first module is Faster R-CNN  [10] , which attributes a label to the object and generates the bounding box to encapsulate the object, while the second module produces a mask of the object  [6]. The most recent published works focusing on building instance segmentation have used Mask R-CNN and shown that it performs poorly for edge extraction and to preserve the integrity of the building instances  [11, 12]. The second method is based on the prediction of the the mask characteristics to further segment the instance in a posterior step. For example, the Deep Watershed Transform  [7]  creates an energy map of the image, where object instances are represented as energy basins.
In post-processing, the different energy basins are separated and correspond to the object instances. The deep learning models for instance segmentation have mostly been developed with images from large-scale online databases, such as IMAGENET  [13]  and PASCAL VOC  [14] ; from medical imagery datasets, such as images from the challenge at the International Symposium on Biomedical Imaging (ISBI); or from images obtained with cameras onboard cars, such as the Kitti Road Data  [15]. However, very little research has been done for instance segmentation of remote sensing imagery. Remotely sensed images have a characteristic which is not commonly encountered in the other types of images and which complicates the application of region-of-interest based instance segmentation (e. g. , Mask R-CNN): they can contain very large number of objects of interests.
This model is of the same type as the Deep Watershed Transform and segments the instances, even if they are adjacent, directly based on the characteristics of the mask. It is designed to be simple and efficient while resolving some of the limitations of the common instance segmentation frameworks. As it is based entirely on the U-net model, the computational resources do not increase with the number of objects in the image, such as in a region-of-interest based model (e. g. , Mask R-CNN). The U-net model is also known to be better (higher Dice score) than Mask R-CNN to predict the segmentation mask when applied on the same dataset  [17]. Our model is also designed to remove all expert-based decisions, such as the number of objects, which has to be defined by the user in Mask R-CNN and influence results and the computational resources, and definition of a threshold value to separate instances such as in the Deep Watershed Transform  [6, 7].
All these mask layers share 1 or 2 pixels of overlap. Second, the result of each U-net is combined with the two others by concatenation operation and followed by several convolution layers and returns an output which is an activation map of its mask layer. These two novelties add a lot redundancy to the mask and the model that seems to help the model to converge. That is, it not only learns the mask layer individually but also the spatial relation (overlapping) between the mask layers. As a study case, the model was tested for instance segmentation of the buildings of Joanópolis-SP (Brazil) using a sample of manually labeled individual buildings and the corresponding very high resolution satellite image of WorldView-3 with 0. 3 m of spatial resolution. The model R code is available at https://doi.
Building Dataset To train the algorithm to detect and segment individual buildings, a mask was manually delineated using the WorldView-3 image of Joanópolis, SP, Brazil. The buildings were manually delineated, resulting in 7563 polygons that have all been validated in the field. Not all the buildings of the image were delineated; however, they were delineated entirely for most neighborhoods in the central region of the city. Each polygon represents one building. The images from the manual sampling presented different types of buildings: residential, commercial, and industrial. Using the delineated polygons, a raster mask coded in RGB (three bands) was produced (Figure  3 ) with the following values: background [0,0,0]; building segment [1,0,0] (Figure  3b ); border segment [0,0,1] (Figure  3c ), which is the border of the building segment augmented by 4 pixels outside and reduced by 3 pixels inside; and inner building segment [0,1,0] (Figure  3d ), which is the building segment reduced by 2 pixels.
Algorithm The model was coded in the programming language R  [23]  with Rstudio interface to Keras  [21, 22]  and Tensorflow backend  [24]. The training of the models took ∼2-20 h using GPU on an Nvidia RTX2080 with 8 GB of dedicated memory. Prediction using GPU of a single tile of 16,384 × 16,384 pixels (∼24. 16 km 2 ) took approximately 10 min.
Results For the city of Joanópolis, the algorithm presents a good level of segmentation accuracy (layer mask 1) with an overall accuracy of 97. 67% and an F1-Score of 0. 937 (precision = 0. 936 and recall = 0. 939) (Table  1 ). The detection rate on the validation data set was of 97. 67%; that is, on the 2226 objects in the validation dataset, the algorithm found 2176 objects and missed 52. The mean intersection over union was 0. 582 and the median was 0. 694. Considering the entire WorldView-3 tile, the algorithm delineated and individualized 7477 buildings. The model segmentation for the three masks was very accurate, as seen by the segment in blue in Figure  4a-c. There are very few errors for the segment and inner segment, as seen in white in Figure  4a, c. The errors seem slightly higher for the border mask (Figure  4b ).
For Arrows 4 and 5, the roof features are very similar. Furthermore, looking at the manually delineated polygon in the bottom left corner, it can be observed that it shows contrasted building features in only one instance. The way the algorithm decides whether or not to segment these instances was likely based on the characteristics of the instances in the training sample. In the main extension of Joanópolis (Figure  7  and Table  2 ), 6106 building instances are individualized (7380 buildings in the reference dataset). The buildings cover an area of 618,077 m 2. The mean area covered by a building in Joanópolis was found to be 101. 2 m 2 (median 68. 6 m 2 ) from the results of our model. For the same spatial extent, the manually delineated buildings area was of 86. 6 m 2 (median 78. 4 m 2 ).
6 m 2 was close to the median of manually delineated building area of 78. 4 m 2. Our model missed ∼17% of the instances (Table  2 ). This happened when two or more buildings were adjacent and not fully separated (see Figures  4d  and 6b ). However, as the objects are very small, the larger buildings are about ∼50 m (or 150 pixels) width, this value should decrease with object of increasing sizes. The Joanópolis building dataset is also complex; for example, it can have two instances under the same roof, which represents a reality in the field but a difficult or impossible case to resolve by the algorithm. Because the most recent works using Mask R-CNN to segment building instances are applied to different datasets  [11, 12] , unfortunately, accuracies are not compatible with our results and comparison of the different methods on the same datasets will be made in a future work.
It was also observed that adding rotation in the data augmentation helped the model to generalize. Obviously, the Joanópolis dataset does not contain all the building morphologies that could be encountered around the world and the training sample has to be increased in order to improve generalization. However, the only information that users need to provide are additional images and mask to train the model, no parameters have to be adjusted by hand, such as in the currently most used instance segmentation frameworks such as Mask R-CNN or Deep Watershed Transform  [6, 25]. We found that the more frequent error of the algorithm was to systematically round the building angles (see Figures  5b  and 6b ). This error is recurrent in the CNN: for example, all the results of semantic segmentation in the leaderboard of the INRIA labeling dataset present this error, even those with the best accuracies (see https://project.
g. , low to high buildings density, slums and highly vegetated areas). The model has been developed in R with the keras package  [28, 29]  for RStudio  [30] , supporting that R can be used for Deep Learning model development. Our model is the first instance segmentation model made entirely from within the R environment. While for the strict Deep Learning part of the algorithm there is currently almost no difference between R and Python, that is, the R commands are translated and sent to Python with the help of the keras and reticulate packages  [31] , R possesses many convenient libraries for spatial data processing. The main spatial packages that have been used here are the raster package to work with the raster data  [32] , the sf package to work with spatial vector data  [33] , and tools to convert between vector data and raster data, such as the fasterize function  [34]  in R or outside R with function directly from the GDAL/OGR library  [35].
The main time bottleneck with the spatial data processing remains the transformation of resulting raster of the model (16,384 × 16,384 at 0. 3 m resolution) to vector data with gdal_poligonize, which can take around 20 min for one tile if there are many of objects (>50,000), even with a custom script for retiling the tile before the vectorization of the predicted objects. This conversion can be sped up with the function isoband from the R package isobands  [36] ; however, the border of the objects will be simplified during vectorization, resulting in some minor variation in the objects area between the raster and the shapefile (which is not the case when using gdal_poligonize). To conclude, R provides convenient and easy-to-use libraries for Deep Learning applications and development for remotely sensed imagery without the need to delve deeply into Python programming.
Conclusions In this work, we present a new architecture of deep learning called U-net-id, specifically designed for the instance segmentation of buildings in very high resolution satellite imagery. The architecture is built to work with image in tiles, such as remote sensing images, and to reduce the minimum information that needs to be provided by the user: only the images for prediction or only the images and masks for training. No manual threshold has to be defined by the user, which is not the case in the currently most used instance segmentation frameworks such as Mask R-CNN or Deep Watershed Transform. The U-net-id architecture achieves great performance, for instance segmentation of the instance building dataset of the city of Joanópolis-SP, Brazil. It will be further improved with samples from other Brazilian cities and applied to building instance segmentation of the main cities of Brazil.
Funding: The research leading to these results received funding from the project BIO-RED 'Biomes of Brazil-Resilience, Recovery, and Diversity', which is supported by the São Paulo Research Foundation (FAPESP, 2015/50484-0) and the U. K. Natural Environment Research Council (NERC, NE/N012542/1). F. H. W. has been funded by FAPESP (grant 2016/17652-9). R. D. acknowledges the support of FAPESP (grant 2015/22987-7). Y. T. has been funded by the project EPITOME ANR-17-CE23-0009 of the French National Research Agency (ANR). We also thank the Amazon Fund through the financial collaboration of the Brazilian Development Bank (BNDES) and the Foundation for Science, Technology and Space Applications (FUNCATE) no. 17. 2. 0536. 1 (Environmental Monitoring of Brazilian Biomes).
One of our parcellations, an atlas composed of 160 parcels, achieves a reproducibility across subjects of ≈0. 74, based on the average Dice's coefficient between subject's connectivity matrices, rather than ≈0. 73 obtained for a macro anatomical parcellation of 150 parcels. Moreover, we compared two of our parcellations with state-of-the-art atlases, finding a degree of similarity with dMRI, functional, anatomical, and multi-modal atlases. The higher similarity was found for our parcellation composed of 185 sub-parcels with another parcellation based on dMRI data from the same database, but created with a different approach, leading to 130 parcels in common based on a Dice's coefficient ≥0. 5.
, 2001). It is based on measurements of the movement of hydrogen atoms present in water molecules of biological tissues. Tractography algorithms reconstruct an estimate of the main WM tracts of the entire brain based on dMRI information  (Basser et al. , 2000; Zhang et al. , 2014). The generated datasets represent an estimation of the main WM pathways, in the format of 3D polylines, also called fibers, even though they do not represent real neural fibers  (Mori and van Zijl, 2002; Perrin et al. , 2005). This technique is indirect, and relies on models and inference, but allows a whole-brain exploration of WM structure in living humans, on large populations of subjects. The structural networks of the human cerebral cortex have not yet been comprehensively mapped  (Sporns et al. , 2005; Hagmann et al.
Unlike most of the methods proposed in the literature, which use full tractography, we use fibers labeled into bundles, according to short and long bundle atlases. The advantage is that the correspondence of connecting regions is given in advance for the different subjects in a database. Furthermore, the generation of parcels from segmented bundles could provide a better representation of the main regions or nodes of the human brain connectome, since these were identified as the main short and long connections of the brain, represented in the atlases of bundles. The resulting parcellation will then represent a subdivision of the cortex into the regions that connect the most probable bundles. The method still experiences difficulty in clearly defining the nodes (cortex parcels), knowing that the bundles from tractography are very variable across subjects and may not exist in several subjects.
This poses a big but interesting challenge. The key point of the proposed work is the automatic analysis of the density and variability of the connecting regions among subjects over the cortical mesh, so that the most probable ones are selected, merged, and homogenized. The overlapping is solved using a graph representation of the intersected regions, taking into account the degree of overlapping of their density centers, across subjects. The method was applied to a group of 79 subjects from a HARDI database. Several quantitative and qualitative evaluations were performed. Twenty parcellations were generated, based on different sets of the three parameters of the method, and compared to evaluate the similarity between them. Furthermore, a reproducibility analysis was also performed, based on the similarity of connectivity matrices across subjects, constructed with the whole tractography.
A comparison with a macro anatomical parcellation using Dice's coefficient between subject's connectivity matrices was performed, showing a slightly better reproducibility in a resultant parcellation generated with the proposed method. Moreover, other comparisons were also made with state-ofthe-art parcellations based on different MRI modalities, finding a degree of similarity with dMRI, functional, anatomical, and multi-modal atlases. A higher similarity was found for our parcellation composed of 185 sub-parcels with another parcellation containing 239 parcels, based on dMRI data from the same database, but was created with a totally different approach. This comparison led to 130 parcels being in common based on a Dice's coefficient ≥0. 5 and 75 parcels being in common with a Dice coefficient ≥ 0.
Tractography Dataset We used the ARCHI database  (Schmitt et al. , 2012) , containing anatomical MRI and HARDI data from 79 healthy subjects with special acquisition sequences of a 3T MRI scanner with a 12-channel head coil (Siemens, Erlangen). The MRI protocol included the acquisition of a T1 image dataset using a MPRAGE sequence (160 slices, TE/TR = 2. 98/2300 ms, TH = 1. 10 mm, deflection angle FA = 9, TI = 900 ms; matrix = 256 × 240; voxel size = 1 × 1 × 1. 1 mm; RBW = 240 Hz/pixel), a single-shell HARDI SS-EPI dataset along 60 optimized DW directions, and a field map B0, b = 1,500 s/mm 2 , (70 slices, TE = 93 ms, FA = 90, TH = 1. 7 mm, TR = 14,000 ms, matrix = 128 × 128, partial Fourier factor PF = 6/8, echo spacing ES = 0. 75 ms, RBW = 1,502 Hz/pixel; GRAPPA = 2, total scan time = 16 min and 46 s).
The database has the affine transformation matrices to convert the data between the spaces T1, T2 (diffusion space), and Talairach. By using the software BrainVISA/Connectomist-2. 0  (Duclap et al. , 2012) , all data were pre-processed. The main artifacts, such as noise, susceptibility effects, geometric distortions, and eddy currents were corrected. Further, defective slices were removed. The analytic Q-ball model was also computed  (Descoteaux et al. , 2007). Whole-brain streamline deterministic tractography was calculated, using a T1based brain propagation mask  (Guevara et al. , 2011a) , with one seed per voxel at T1 resolution, a maximum curvature angle of 30 • , and a tracking step of 0. 2 mm. Resulting tractography datasets contain about 1 million fibers per subject.
Approach We propose a hybrid method for the creation of fine-grained parcellations of the cortical surface, from a coarse-grained parcellation according to an anatomical atlas, based on structural connectivity information, given by segmented bundles for a population of subjects. The bundle segmentation is based on atlas bundles from three different atlases. The parcellation method receives as an input tractography and the labeled mesh of each subject, the fused bundle atlas with selected superficial and deep white matter bundles. The method returns an average parcellation atlas for the input dataset, which consists of a subdivision of the anatomical parcels (gyri) of Desikan-Killiany atlas, based on the most stable connectivity-based sub-parcels across the subjects. The data consists of the labels associated with each cortical mesh vertex.
Pre-processing: Fusion of Atlases This pre-processing aims to create a fused atlas of white matter bundles, containing the main WM connections across subjects and, consequently, to create a more complete parcellation of the cortex (see Figure  1A ). We used two atlases of superficial white matter (SWM) and one atlas of deep white matter (DWM). The first SWM atlas, swm_atlas_1, is composed of 50 bundles in both hemispheres, with a total of 7,857 fibers  (Guevara et al. , 2017a). The second SWM atlas swm_atlas_2, has 44,345 fibers and is made up of 44 bundles in the left hemisphere and 49 bundles in the right hemisphere  (Román et al. , 2017). Finally, the DWM atlas contains 18 bundles per hemisphere, corresponding to 11,755 fibers  (Guevara et al. , 2012). Those atlases were created using the ARCHI database, representing the most reproducible bundles across subjects (see Figure  S2 ).