dataesr/scientific-paragraphs-categorization huggingface dataset:
B. K. G. acknowledges the support of  STFC  through the  University of Hull  Consolidated Grant  ST/ R000840/1 , access to VIPER, the  University of Hull High-Performance Computing Facility , and the  European Unionʼs Horizon 2020 research and innovation program  ( ChETEC-INFRA- Project No. 101008324 ). This research was also partially supported by the  ANR-19-CE31-0017 ; http://www. secular-evolution. org. Y. K. is supported by  NRF-2022M3K3A1097100. J. H. S. acknowledges support from the  Korea Astronomy and Space Science Institute  under the  R&D program  (Project No. 2023-1-830-00 ) and the  National Research Foundation of Korea  grant ( 2021R1C1C1003785 ) funded by the  Ministry of Science, ICT & Future Planning. High-performance computing resources for this research were partially supported by the  Research Solution Center  and the  National Supercomputing Center for Astrophysics and Space Sciences in the Institute for Basic Science.
On the other hand, methods for estimating biomechanically accurate skeletal motion typically rely on complex motion capture systems and expensive optimization methods. What is needed is a parametric 3D human model with a biomechanically accurate skeletal structure that can be easily posed. To that end, we develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton. To enable this, we need training data of skeletons inside SMPL meshes in diverse poses. We build such a dataset by optimizing biomechanically accurate skeletons inside SMPL meshes from AMASS sequences. We then learn a regressor from SMPL mesh vertices to the optimized joint locations and bone rotations. Finally, we re-parametrize the SMPL mesh with the new kinematic parameters. The resulting SKEL model is animatable like SMPL but with fewer, and biomechanically-realistic, degrees of freedom.
We show that SKEL has more biomechanically accurate joint locations than SMPL, and the bones fit inside the body surface better than previous methods. By fitting SKEL to SMPL meshes we are able to "upgrade" existing human pose and shape datasets to include biomechanical parameters. SKEL provides a new tool to enable biomechanics in the wild, while also providing vision and graphics researchers with a better constrained and more realistic model of human articulation. The model, code, and data are available for research at https://skel. is. tue. mpg. de. CCS Concepts: • Computing methodologies → Mesh models; Motion capture; Physical simulation.
While previous work has addressed the problem of putting skeletons inside 3D body models  [Ali-Hamadi et al. 2013; Kelc 2012; Keller et al. 2022; Shetty et al. 2023 ], such approaches have not addressed the problem of precisely locating the skeleton within a moving body. The key challenge is the lack of training data that pairs the posed 3D human body shape with the ground-truth skeleton. We address this by creating a novel dataset called BioAMASS. To create BioAMASS, we take sequences of 3D bodies from the AMASS dataset  [Mahmood et al. 2019 ] that cover a wide range of body shapes and challenging poses. To obtain pseudo-ground-truth skeletons, we place virtual motion capture markers on the body surface. We then use the recent method, AddBiomechanics  [Werling et al. 2022] , to solve for the BSM skeleton given the virtual markers.
With this paired dataset, we can now solve several problems that were previously impossible. First, we train a regressor to estimate the 3D anatomical BSM joint locations of the body given a posed SMPL mesh. Note that these locations significantly differ from the joints in SMPL. This is useful for generating more relevant training data for 2D or 3D joint detectors, as today, such methods are typically trained from manually labeled joints or projected SMPL joints. Next, we re-rig the SMPL body model with the BSM biomechanical skeleton, i. e. we use the BSM parameters to drive a SMPL mesh, and we call the resulting model SKEL, which is short for "Skeletal Kinematics Enveloped by a Learned body model". To do so, it is critical that the skeleton is properly scaled, located and oriented inside the SMPL body mesh.
SKEL has several uses. Specifically, we consider the problem of taking a SMPL body model and computing the correct skeleton inside. To do so, we simply fit SKEL to the posed SMPL mesh by optimizing the SKEL pose to minimize the vertex-to-vertex distance between the meshes. We apply this process to archival datasets such as 3DPW  [Von Marcard et al. 2018]  and  BEDLAM [Black et al. 2023]. This effectively upgrades existing computer vision datasets to contain biomechanical ground truth, extending their use to biomechanics. For example, one could evaluate, or learn to directly regress biomechanical parameters from video. We evaluate two methods for estimating the skeleton from SMPL: direct regression of BSM joints from SMPL and fitting SKEL to SMPL. Accuracy is defined in terms of 3D joint location error.
Since there is no ground-truth for this task, we take the joint locations estimated by AddBiomechanics as pseudo-ground-truth. We find that both of our methods produce significantly more accurate joint predictions than SMPL. We also provide extensive qualitative experiments that show the articulated structure of SKEL and its use in upgrading existing human motion datasets to support biomechanics. SKEL can also be used in the other direction. Given an input skeleton mesh obtained after fitting a biomechanical model to mocap data, SKEL can be used to add a plausible skin surface; this is useful for visualization of mocap data. Since there are an infinite number of body shapes that are consistent with a given skeleton, the predicted shape can be constrained, e. g. with the subject's weight. To the best of our knowledge, SKEL is the first model where the body surface and anatomical skeleton are directly controlled by the same set of shape and pose parameters (𝜷, q).
METHOD OVERVIEW Our driving goal is to create SKEL, a model that combines skin and skeleton meshes in which both are synchronously rigged with the same pose parameters q, and can be reshaped by inheriting the SMPL shape space. To create this model, we must know the location of the anatomic joints and bone rotations inside the human body. There is no large-scale medical dataset of subjects in motion where one can extract both the body and skeleton meshes, and static medical scans do not fully constrain the skeleton in motion. For this, we need bodies in motion and leverage the AMASS dataset  [Mahmood et al. 2019]  to address this challenge. In Sec. 4 we first present our new custom Biomechanical Skeleton Model, BSM, and describe how to align it inside AMASS sequences of SMPL bodies in motion to obtain the new BioAMASS dataset.
THE BIOAMASS DATASET The goal of the BioAMASS dataset is to enable the learning of the location and orientation of the 3D bones inside a body surface in motion. To create BioAMASS, we use the SMPL  [Loper et al. 2015 ] model for the body surface and a new biomechanical skeleton model, BSM, for the bones. We first introduce these two models, and then describe how we fit BSM to SMPL and create BioAMASS.
2019 ] and thus re-parameterize this prior term as 𝑃 (s, 𝜷). Despite the scale prior, using a generic marker set can lead to Ad-dBiomechanics over-stretching the bones for heavy subjects. Defining personalized marker locations m 0 (𝜷) on the skeleton template as described in the previous section helps further regularize the bone scales (Fig. 4c ). We apply this optimization process to a subset of AMASS consisting of 113 subjects and 2198 motion sequences, amounting to over 9 hours of motion data. The paired SMPL meshes and BSM skeletons form the BioAMASS dataset. For each subject 𝑝 there is a SMPL body shape 𝜷 𝑝 and the scaled personalized BSM model s 𝑝. Further, for each motion frame 𝑓 it includes the bone angles q 𝑓 as well as the bone joint locations J 𝐵 𝑓. Figure  5  shows examples of the BioAMASS dataset.
THE SKEL MODEL Now we have BSM skeletons inside SMPL but we want to go further and parameterize the 3D body model with the biomechanical skeleton. To that end, we develop SKEL, which is designed to be We deduce the personalized markers location m 0 (𝜷 ) on the BSM bone template. (c) On high BMI subjects, a shape-agnostic marker definition for all subjects yields over-stretched bones (red). Using personalized marker locations m 0 (𝜷 ) defined using OSSO prevents this over-stretching (green). compatible with SMPL and posed like BSM. This allows us to leverage SMPL's learned shape space as well as all the existing datasets where SMPL bodies are estimated from different modalities. To create SKEL, we must put SMPL vertices and the BSM skeleton together in the same reference frame. We pose the BSM skeleton mesh inside a SMPL body in T-pose, the zero-pose of the SMPL body model.
To that end, in Sec. 5. 1, we learn to regress the anatomical joint locations from SMPL using the BioAMASS dataset. Then, in Sec. 5. 2 we describe how we rig the SMPL model using the BSM joint rotations. Note that, in SMPL, all joint orientations are defined in a global T-pose space with an axis-aligned frame of reference for each joint as illustrated in Fig. 6  right. This means that SMPL assumes, for example, that the elbow rotation axis is aligned with the world y-axis, independent of the orientation of the humerus. The overparametrized nature of SMPL allows plausible arm articulation by combining several axis rotations. But BSM, with its reduced degrees of freedom for the rotations, requires the local frame on which the rotation is applied to be precisely aligned with the anatomy in order to obtain a proper anatomic rigging.
Establishing the bone locations and orientations Anatomical joint locations. Given paired SMPL body meshes and their corresponding BSM anatomic joint locations, we learn a function that predicts the joints from the body surface. We proceed similarly to  Loper et al. [2015]  by learning a joint regressor J that takes as input the SMPL mesh vertices v smpl ∈ R 6890×3 and predicts the new anatomic joints J 𝐵 ∈ R 24×3. We follow the Keller et al. [2022] OSSO methodology, by formulating a non-negative least squares problem for each joint 𝑖, and solving it with an active set method  [Lawson and Hanson 1995]. We train these regressors from the posed vertices and joints of the BioAMASS dataset. Figure  6  shows the new regressed kinematic tree in green. Notice that the hip joint locations, corresponding to the femur heads, are more anatomically correct than the ones in SMPL.
To solve those two issues, we split the bone rotation 𝑅 𝑖 (𝜷) into a learned base rotation 𝑅 𝑏𝑎𝑠𝑒 𝑖 and a shape-dependant rotation 𝑅 𝜷 𝑖 (𝜷): 𝑅 𝑖 (𝜷) = 𝑅 𝜷 𝑖 (𝜷) • 𝑅 𝑏𝑎𝑠𝑒 𝑖 , (3) where 𝑅 𝑏𝑎𝑠𝑒 𝑖 is learned to define the bone's orientation around its bone axis, ensuring that bones are properly orientated wrt their parent bone. 𝑅 𝜷 𝑖 (𝜷) is computed dynamically to align the bone to the segment defined by its parent and child joints, so that the bone stays in its socket regardless of the shape of the subject. First, we learn 𝑅 𝑏𝑎𝑠𝑒 𝑖 from BioAMASS. For each bone 𝑖, we can define a corresponding SMPL joint and limb. For example, the right humerus bone corresponds to the 17th joint and the right upper arm of SMPL. Thus, for each frame 𝑓 of our dataset, we obtain the bone BSM rotation 𝑅 𝐵 𝑖,𝑓 and its SMPL rotation 𝑅 𝑆 𝑖,𝑓.
𝑅 𝑏𝑎𝑠𝑒 𝑖 is the rotation that the bone needs to undergo so that when chained to the SMPL rotation, the corresponding BSM rotation is obtained. For each bone 𝑖 we learn its base rotation 𝑅 𝑏𝑎𝑠𝑒 𝑖 by minimizing 𝑁 𝐹 ∑︁ 𝑓 =1 (𝑅 𝐵 𝑖,𝑓 -𝑅 𝑆 𝑖,𝑓 𝑅 𝑏𝑎𝑠𝑒 𝑖 ) 2 (4) over the 𝑁 𝐹 frames of the dataset. This rotation properly orients the bone around its bone axis. But, as shown in Fig. 7 , this rotation alone does not guarantee that the bones are aligned between their T-pose parent and child joints. Thus we explicitly compute 𝑅.
Joint regressors We evaluate the regressors learned in Sec. 5. 1 on unseen body meshes by comparing the regressed values with the reference BSM alignment. We train our anatomical joint regressors on the CMU  [CMU Graphics Lab 2000]  and MPI_Limits  [Akhter and Black 2015]  datasets, which are part of AMASS  [Mahmood et al. 2019 ]. CMU contains good variation in body shape, while MPI_Limits contains extreme poses. Once trained, we evaluate our regressor on the DFAUST dataset  [Bogo et al. 2017] , with various motion sequences for 10 subjects with diverse BMIs; DFAUST contains precise SMPL fits to 3D scan sequences. For each frame of the DFAUST dataset, BioAMASS provides the anatomical joint locations J 𝐵 that we consider ground truth. Then, from the frame's SMPL mesh, we use our learned joint regressor to regress the anatomical joint location J 𝑟𝑒𝑔.
SKEL fits to SMPL Since SKEL has the same surface mesh topology and shape parameters 𝜷 as SMPL, it can be directly fit to existing SMPL meshes by optimizing its pose parameters to minimize the vertex-to-vertex error. To quantitatively evaluate how similar SKEL shapes are to SMPL, we consider motion sequences from the DFAUST dataset and their SMPL fits with 10 shape parameters. We fit SKEL to each of these SMPL meshes by optimizing its pose parameters q. To evaluate the mesh fits, we compute the mean absolute difference (MAD) between SKEL skin vertices and the target SMPL vertices, and then average over all the frames. For males, we find an average difference of 1. 1 cm and an average max difference of 2. 5 cm, while for females we obtain an average mean difference of 0. 9 cm and max of 1. 9 cm.
A visualization of these differences on the SMPL body mesh is shown in Fig. 12. The larger differences can be explained by the approximate pose-dependant blend shapes inherited from SMPL, which could be retrained in future work. Fitting SKEL to SMPL provides joint locations with similar accuracy as the regressed ones, as reported on Fig. 11. Let us note that direct joint regression is faster than estimating the SKEL model fit. Applications that require the joint locations but not the skeleton pose parameters, and for which time is critical, should prefer the direct regression approach. Upgrading SMPL datasets with SKEL. Since SKEL is compatible with SMPL, we can fit SKEL to SMPL meshes from the 3DPW dataset  [Von Marcard et al. 2018 ] and the synthetic BEDLAM  [Black et al. 2023 ] dataset (Fig.
Qualitative comparisons with OSSO SKEL fits to SMPL also yield anatomically correct orientations of the bones. To illustrate this we compare the SKEL predictions to OSSO skeletons  [Keller et al. 2022 ] on the MOYO dataset  [Tripathi et al. 2023 ]. The SKEL skeletons yield more anatomically correct joint location and biomechanically relevant bone angles, as visible in Fig. 14 ; see, for example, the knee orientation as well as arm supination. See Sup. Mat. and Sup. Video for more examples.
Disentangling body shape and bone lengths Since our skeleton mesh is fully defined by the joint segment lengths, we can modify the body shape of a person, while maintaining their skeletal identity. This can be helpful for generating a plausible skin  mesh from a given biomechanical skeleton. As illustrated in Fig. 15 , we optimize the SKEL shape parameters 𝜷 to fit a subject's limb lengths with different target weights. This results in body meshes with different body shapes but the same bone lengths. In this paper we describe SKEL, a new parametric 3D human body shape model driven by anatomically sound parameters, providing consistent skin and bone geometries. SKEL is learned from BioA-MASS, a new dataset of skeletons inside SMPL meshes in diverse AMASS poses. We build BioAMASS by optimizing BSM, a new biomechanically accurate skeleton model, to fit inside SMPL mesh sequences.
SKEL extensions and future work. There are several directions for extending and improving SKEL. For instance, in the current model the hands of SKEL are rigid. Using a biomechanical model with more expressive hands, our approach could be used to put it in correspondence inside SMPL-X  [Pavlakos et al. 2019 ]. The bone locations could also be supervised with static medical data observations, such as the ones provided by  [Wang et al. 2019]. Additionally, the current SKEL model inherits the skinning weights and pose-dependent blend shapes from SMPL. These could be retrained from the BioAMASS dataset to make the skin surface deformation more accurate. Ideally, the pose-correctives should be retrained from 3D scan data with BSM as the native parameterization. This would allow the pose-corrective offsets to be directly learned as a function of BSM parameters.
ACKNOWLEDGMENTS Michael J. Black  (MJB) has received research gift funds from  Adobe ,  Intel ,  Nvidia ,  Meta/Facebook , and  Amazon. MJB has financial interests in Amazon,  Datagen Technologies , and  Meshcapade GmbH. While MJB is a consultant for Meshcapade, his research was performed solely at  MPI. Soyong Shin  performed this work while an intern at the  Max Planck Institute for Intelligent Systems. Marilyn Keller  was supported by the  International Max Planck Research School for Intelligent Systems. Part of her work was performed while an intern at  Stanford. Sergi Pujades ' work was funded by the  ANR   SEMBA  project. This work uses aitviewer for visualization [Kaufmann et al. 2022]. We also thank  Neelay Shah  for the SMPL fits to the MOYO dataset.
It helps declare, discover and interact with Things in a standard way as well as monitor their state. The Thing Description is a definition language, similar to what OpenAPI is for RESTful services. Table  1  represents the efforts established in DD towards the attainment and improvement of the non-functional requirements (NFR) set out in the ISO/IEC 25010 related to the quality parameters of a software product, and compares them against an IoT device and the WoT reference implementation (WoT Scripting API). As an architectural solution based on the WoT, Digital Dice deploys a set of microservices in a distributed containerized environment. This fact allows Digital Dice to become a highly scalable architecture, making this the first step towards attaining the HA requirement. Furthermore, the way that the microservices are deployed allows us to scale the architecture horizontally, increasing the number of instances of each microservice as needed.
Web of Things Web of Things (WoT) describes a set of standards to create flexible, scalable, and open Internet of Things (IoT) ecosystems using web technologies as the application layer. The concept of Digital Dice began as a Web of Things servient system, which is a software stack that implements some of the WoT building blocks. A servient can expose Things, as well as consume them. Digital Dice is what the WoT calls an intermediary servient software, a piece of software capable of doing both, consuming things and exposing them. A Thing Description (TD) describes the metadata and interfaces of Things, which includes the set of properties, actions and events managed by that Thing. Moreover, the properties, actions, and events are what the Thing Description calls InteractionAffordance. The Interactions are simultaneously composed by one or more Forms or ways to access data.
All the Interactions of a Thing are instances of a DataSchema. This DataSchema defines the type of object that the interaction responds with (e. g. , the status of a light has a Boolean as a data schema). Thanks to this, both ourselves and the system can know the type of data that will be returned when requesting an interaction. The @type property can be located in the Thing or in the various Interacti-onAffordances, and they essentially constitute an object with semantic tags (or types). Because it is a semantic value, the parameter can be used for a variety of purposes. For example, in the case of Digital Dice, it allows us to define specific characteristics of a thing custom-made for our Digital Dice, such as when an interaction has a user interface or when a device employs a protocol that the Web of Things does not fully support.
4 Digital dice: core concepts functionality in the best possible way. To achieve our goal, we decided to adopt WoT as a common language for the definition of Digital Dice. Using a standard supported by the W3C ensured the compatibility of Digital Dice with systems and platforms that use the WoT as a definition language, like WebThings. io [14], a platform for monitoring and managing devices over the web, recently liberated by the Mozilla foundation. As an intermediary WoT servient, Digital Dice can consume and expose things. Moreover, it adopts microservices as one of its building blocks. The use of microservices adds complexity to our infrastructure but also offers the possibility of adding replicas to the services that need it. As we mentioned previously, one of the main problems that we have when working directly with physical devices is that they tend to have constrained resources in some cases.
Digital Dice can make use of the following microservices: -Controller (C). This microservice is responsible for communicating with clients or other software components. Furthermore, it orchestrates the requests to the rest of the microservices. -Reflection (R). This microservice is responsible for recording the interactions that occur on the IoT device onto a database, defined by the DBConfig, as well as executing the interactions requested by the user on said device. This microservice establishes a permanent connection with the IoT device it represents. -DataHandler (DH). This microservice is responsible for communication with the underlying database. It records all the interactions requested for a particular device on the database and recovers the data requested by users. This microservice acts as an intermediary and decides when a request can be resolved with the database without establishing a direct connection with the IoT device.
Apart from the microservices established in the metamodel for each DD, the architecture requires a series of common artefacts for their correct operation. First, Digital Dice needs a database. For that, we use MongoDB due to its capability of working with all types of unstructured data and the possibility of making replicas, which allows simpler scalability than other types of databases. It is important to note that DD establishes its own schema when saving the data generated by the IoT device or the requests made by the user. Creating a general schema is necessary since the data required or provided by different IoT devices can be diametrically different. In the best case, these devices will offer data in the form of readable JSON or XML-type schemas, although this is not always the case, since we can also find textual or binary data.
We have now seen all the pieces that make up the architecture of DD. However, it is interesting to see how different types of requests are orchestrated in this architecture to understand its operation better. To do so, in Fig. 3  we depict two different types of requests. On the one hand, the invocation of an action X, and, on the other, the request of a property Y to a Digital Dice. In the first request, the user asks the controller of the DD that represents the device to invoke the action X, while the controller requests the action from DH. DH then registers the aforementioned request on the database, which will be seen by the Reflection of the device that executes the action X on the device. In the second request, the user asks the controller for the value of the property Y, before the controller redirects the request to the DH, which asks the database about this property.