OpenRaiser/Intern-Atlas huggingface dataset:
CAST: Character labeling in Animation using Self-supervision by Tracking
Venue: arXiv cs.CV 2022
Year: 2022

Cartoons and animation domain videos have very different characteristics compared to real-life images and videos. In addition, this domain carries a large variability in styles. Current computer vision and deep-learning solutions often fail on animated content because they were trained on natural images. In this paper we present a method to refine a semantic representation suitable for specific animated content. We first train a neural network on a large-scale set of animation videos and use the mapping to deep features as an embedding space. Next, we use self-supervision to refine the representation for any specific animation style by gathering many examples of animated characters in this style, using a multi-object tracking. These examples are used to define triplets for contrastive loss training. The refined semantic space allows better clustering of animated characters even when they have diverse manifestations. Using this space we can build dictionaries of characters in an animation videos, and define specialized classifiers for specific stylistic content (e.g., characters in a specific animation series) with very little user effort. These classifiers are the basis for automatically labeling characters in animation videos. We present results on a collection of characters in a variety of animation styles.
Smart Service-Oriented Clustering for Dynamic Slice Configuration
Venue: arXiv cs.NI 2022
Year: 2022

The fifth generation (5G) and beyond wireless networks are foreseen to operate in a fully automated manner, in order to fulfill the promise of ultra-short latency, meet the exponentially increasing resource requirements, and offer the quality of experience (QoE) expected from end-users. Among the ingredients involved in such environments, network slicing enables the creation of logical networks tailored to support specific application demands (i.e., service level agreement SLA, quality of service QoS, etc.) on top of physical infrastructure. This creates the need for mechanisms that can collect spatiotemporal information on users'service consumption, and identify meaningful insights and patterns, leveraging machinelearning techniques. In this vein, our paper proposes a framework dubbed"SOCLfor" the Service Oriented CLustering, analysis and profiling of users (i.e., humans, sensors, etc.) when consuming enhanced Mobile BroadBand (eMBB) applications, internet of things (IoT) services, and unmanned aerial vehicles services (UAVs). SOCL relies mainly on the realistic network simulation framework"network slice planne"(NSP), and two clustering methods namely K-means and hierarchical clustering. The obtained results showcase interesting features, highlighting the benefit of the proposed framework.
Efficient Steering Mechanism for Mobile Network-enabled UAVs
Venue: arXiv cs.NI 2022
Year: 2022

The consideration of mobile networks as a communication infrastructure for unmanned aerial vehicles (UAVs) creates a new plethora of emerging services and opportunities. In particular, the availability of different mobile network operators (MNOs) can be exploited by the UAVs to steer connection to the MNO ensuring the best quality of experience (QoE). While the concept of traffic steering is more known at the network side, extending it to the device level would allow meeting the emerging requirements of today's applications. In this vein, an efficient steering solutions that take into account the nature and the characteristics of this new type of communication is highly needed. The authors introduce, in this paper, a mechanism for steering the connection in mobile network-enabled UAVs. The proposed solution considers a realistic communication model that accounts for most of the propagation phenomena experienced by wireless signals. Moreover, given the complexity of the related optimization problem, which is inherent from this realistic model, the authors propose a solution based on coalitional game. The goal is to form UAVs in coalitions around the MNOs, in a way to enhance their QoE. The conducted performance evaluations show the potential of using several MNOs to enhance the QoE for mobile network-enabled UAVs and prove the effectiveness of the proposed solution. Index
Improving Behavioural Cloning with Human-Driven Dynamic Dataset Augmentation
Venue: arXiv cs.AI 2022
Year: 2022

Behavioural cloning has been extensively used to train agents and is recognized as a fast and solid approach to teach general behaviours based on expert trajectories. Such method follows the supervised learning paradigm and it strongly depends on the distribution of the data. In our paper, we show how combining behavioural cloning with human-in-the-loop training solves some of its flaws and provides an agent task-specific corrections to overcome tricky situations while speeding up the training time and lowering the required resources. To do this, we introduce a novel approach that allows an expert to take control of the agent at any moment during a simulation and provide optimal solutions to its problematic situations. Our experiments show that this approach leads to better policies both in terms of quantitative evaluation and in human-likeliness.
Ergonomics Integrated Design Methodology using Parameter Optimization, Computer-Aided Design, and Digital Human Modelling: A Case Study of a Cleaning Equipment
Venue: arXiv cs.HC 2022
Year: 2022

Challenges of enhancing productivity by amplifying efficiency and man-machine compatibility of equipment can be achieved by adopting advanced technologies. This study aims to present and exemplify methodology for incorporating ergonomics pro-actively into the design using computer-aided design and digital human modeling-based analysis. The cleaning equipment is parametrized to detect the critical variables. The relations are then constrained through the 3DSSPP software-based biomechanical and experimental analysis using a prototype. MATLAB and Minitab software is used for optimizing efficiency while satisfying the established constraints. The experiment showed nearly 67%, 120%, and 241% successive improvement in the mechanical advantage in comparison to their immediate predecessors. A significant (6 point) reduction in rapid entire body assessment score has been observed in the final posture while working with the manipulator. 3DSSPP suggested that the joint forces during the actuation of the manipulator were acceptable to 99% of the working population. The study demonstrated the potential of the methodology in revamping the equipment for improved ergonomic design.
SCOTCH: An Efficient Secure Computation Framework for Secure Aggregation
Venue: arXiv cs.CR 2022
Year: 2022

Federated learning enables multiple data owners to jointly train a machine learning model without revealing their private datasets. However, a malicious aggregation server might use the model parameters to derive sensitive information about the training dataset used. To address such leakage, differential privacy and cryptographic techniques have been investigated in prior work, but these often result in large communication overheads or impact model performance. To mitigate this centralization of power, we propose SCOTCH, a decentralized m-party secure-computation framework for federated aggregation that deploys MPC primitives, such as secret sharing. Our protocol is simple, efficient, and provides strict privacy guarantees against curious aggregators or colluding data-owners with minimal communication overheads compared to other existing state-of-the-art privacy-preserving federated learning frameworks. We evaluate our framework by performing extensive experiments on multiple datasets with promising results. SCOTCH can train the standard MLP NN with the training dataset split amongst 3 participating users and 3 aggregating servers with 96.57% accuracy on MNIST, and 98.40% accuracy on the Extended MNIST (digits) dataset, while providing various optimizations.
A Centrality Analysis of the Lightning Network
Venue: arXiv cs.CR 2022
Year: 2022

Payment channel networks (PCNs) such as the Lightning Network offer an appealing solution to the scalability problem faced by many cryptocurrencies operating on a blockchain such as Bitcoin. However, PCNs also inherit the stringent dependability requirements of blockchain. In particular, in order to mitigate liquidity bottlenecks as well as on-path attacks, it is important that payment channel networks maintain a high degree of decentralization. Motivated by this requirement, we conduct an empirical centrality analysis of the popular Lightning Network, and in particular, the betweenness centrality distribution of the routing system. Based on our extensive data set (using several millions of channel update messages), we implemented a TimeMachine tool which enables us to study the network evolution over time. We find that although the network is generally fairly decentralized, a small number of nodes can attract a significant fraction of the transactions, introducing skew. Furthermore, our analysis suggests that over the last two years, the centrality has increased significantly, e.g., the inequality (measured by the Gini index) has increased by more than 10%.
Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulation
Venue: arXiv cs.RO 2022
Year: 2022

Learning to solve precision-based manipulation tasks from visual feedback using Reinforcement Learning (RL) could drastically reduce the engineering efforts required by traditional robot systems. However, performing fine-grained motor control from visual inputs alone is challenging, especially with a static third-person camera as often used in previous work. We propose a setting for robotic manipulation in which the agent receives visual feedback from both a third-person camera and an egocentric camera mounted on the robot's wrist. While the third-person camera is static, the egocentric camera enables the robot to actively control its vision to aid in precise manipulation. To fuse visual information from both cameras effectively, we additionally propose to use Transformers with a cross-view attention mechanism that models spatial attention from one view to another (and vice-versa), and use the learned features as input to an RL policy. Our method improves learning over strong single-view and multi-view baselines, and successfully transfers to a set of challenging manipulation tasks on a real robot with uncalibrated cameras, no access to state information, and a high degree of task variability. In a hammer manipulation task, our method succeeds in 75% of trials versus 38% and 13% for multi-view and single-view baselines, respectively.
On Distributed Lossy Coding of Symmetrically Correlated Gaussian Sources
Venue: arXiv cs.IT 2022
Year: 2022

A distributed lossy compression network with $L$ encoders and a decoder is considered. Each encoder observes a source and sends a compressed version to the decoder. The decoder produces a joint reconstruction of target signals with the mean squared error distortion below a given threshold. It is assumed that the observed sources can be expressed as the sum of target signals and corruptive noises which are independently generated from two symmetric multivariate Gaussian distributions. The minimum compression rate of this network versus the distortion threshold is referred to as the rate-distortion function, for which an explicit lower bound is established by solving a minimization problem. Our lower bound matches the well-known Berger-Tung upper bound for some values of the distortion threshold. The asymptotic gap between the upper and lower bounds is characterized in the large $L$ limit.
Learning-From-Disagreement: A Model Comparison and Visual Analytics Framework
Venue: arXiv cs.LG 2022
Year: 2022

With the fast-growing number of classification models being produced every day, numerous model interpretation and comparison solutions have also been introduced. For example, LIME and SHAP can interpret what input features contribute more to a classifier's output predictions. Different numerical metrics (e.g., accuracy) can be used to easily compare two classifiers. However, few works can interpret the contribution of a data feature to a classifier in comparison with its contribution to another classifier. This comparative interpretation can help to disclose the fundamental difference between two classifiers, select classifiers in different feature conditions, and better ensemble two classifiers. To accomplish it, we propose a learning-from-disagreement (LFD) framework to visually compare two classification models. Specifically, LFD identifies data instances with disagreed predictions from two compared classifiers and trains a discriminator to learn from the disagreed instances. As the two classifiers' training features may not be available, we train the discriminator through a set of meta-features proposed based on certain hypotheses of the classifiers to probe their behaviors. Interpreting the trained discriminator with the SHAP values of different meta-features, we provide actionable insights into the compared classifiers. Also, we introduce multiple metrics to profile the importance of meta-features from different perspectives. With these metrics, one can easily identify meta-features with the most complementary behaviors in two classifiers, and use them to better ensemble the classifiers. We focus on binary classification models in the financial services and advertising industry to demonstrate the efficacy of our proposed framework and visualizations.
Decoupling the Depth and Scope of Graph Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

State-of-the-art Graph Neural Networks (GNNs) have limited scalability with respect to the graph and model sizes. On large graphs, increasing the model depth often means exponential expansion of the scope (i.e., receptive field). Beyond just a few layers, two fundamental challenges emerge: 1. degraded expressivity due to oversmoothing, and 2. expensive computation due to neighborhood explosion. We propose a design principle to decouple the depth and scope of GNNs -- to generate representation of a target entity (i.e., a node or an edge), we first extract a localized subgraph as the bounded-size scope, and then apply a GNN of arbitrary depth on top of the subgraph. A properly extracted subgraph consists of a small number of critical neighbors, while excluding irrelevant ones. The GNN, no matter how deep it is, smooths the local neighborhood into informative representation rather than oversmoothing the global graph into "white noise". Theoretically, decoupling improves the GNN expressive power from the perspectives of graph signal processing (GCN), function approximation (GraphSAGE) and topological learning (GIN). Empirically, on seven graphs (with up to 110M nodes) and six backbone GNN architectures, our design achieves significant accuracy improvement with orders of magnitude reduction in computation and hardware cost.
Pipe Climbing Robot
Venue: arXiv cs.RO 2022
Year: 2022

This paper presents the plan of an in-pipe climbing robot that works utilizing a novel Three-Output Open Differential(3-OOD) component to navigate complex organizations of lines. Customary wheeled/followed in-pipe climbing robots are inclined to slip and haul while navigating in pipe twists. The 3-OOD component helps in accomplishing the original aftereffect of wiping out slip and drag in the robot tracks during movement. The proposed differential understands the practical capacities of the customary two-yield differential, which is accomplished the initial time for a differential with three results. The 3-OOD component precisely tweaks the track rates of the robot in light of the powers applied on each track inside the line organization, by wiping out the requirement for any dynamic control. The recreation of the robot crossing in the line network in various directions and in pipe-twists without slip shows the proposed plan's adequacy
A Practical Approach of Actions for FAIRification Workflows
Venue: arXiv cs.DB 2022
Year: 2022

Since their proposal in 2016, the FAIR principles have been largely discussed by different communities and initiatives involved in the development of infrastructures to enhance support for data findability, accessibility, interoperability, and reuse. One of the challenges in implementing these principles lies in defining a well-delimited process with organized and detailed actions. This paper presents a workflow of actions that is being adopted in the VODAN BR pilot for generating FAIR (meta)data for COVID-19 research. It provides the understanding of each step of the process, establishing their contribution. In this work, we also evaluate potential tools to (semi)automatize (meta)data treatment whenever possible. Although defined for a particular use case, it is expected that this workflow can be applied for other epidemical research and in other domains, benefiting the entire scientific community.
Unsupervised Personalization of an Emotion Recognition System: The Unique Properties of the Externalization of Valence in Speech
Venue: arXiv cs.SD 2022
Year: 2022

The prediction of valence from speech is an important, but challenging problem. The externalization of valence in speech has speaker-dependent cues, which contribute to performances that are often significantly lower than the prediction of other emotional attributes such as arousal and dominance. A practical approach to improve valence prediction from speech is to adapt the models to the target speakers in the test set. Adapting a speech emotion recognition (SER) system to a particular speaker is a hard problem, especially with deep neural networks (DNNs), since it requires optimizing millions of parameters. This study proposes an unsupervised approach to address this problem by searching for speakers in the train set with similar acoustic patterns as the speaker in the test set. Speech samples from the selected speakers are used to create the adaptation set. This approach leverages transfer learning using pre-trained models, which are adapted with these speech samples. We propose three alternative adaptation strategies: unique speaker, oversampling and weighting approaches. These methods differ on the use of the adaptation set in the personalization of the valence models. The results demonstrate that a valence prediction model can be efficiently personalized with these unsupervised approaches, leading to relative improvements as high as 13.52%.
ASL Video Corpora & Sign Bank: Resources Available through the American Sign Language Linguistic Research Project (ASLLRP)
Venue: arXiv cs.CL 2022
Year: 2022

The American Sign Language Linguistic Research Project (ASLLRP) provides Internet access to high-quality ASL video data, generally including front and side views and a close-up of the face. The manual and non-manual components of the signing have been linguistically annotated using SignStream(R). The recently expanded video corpora can be browsed and searched through the Data Access Interface (DAI 2) we have designed; it is possible to carry out complex searches. The data from our corpora can also be downloaded; annotations are available in an XML export format. We have also developed the ASLLRP Sign Bank, which contains almost 6,000 sign entries for lexical signs, with distinct English-based glosses, with a total of 41,830 examples of lexical signs (in addition to about 300 gestures, over 1,000 fingerspelled signs, and 475 classifier examples). The Sign Bank is likewise accessible and searchable on the Internet; it can also be accessed from within SignStream(R) (software to facilitate linguistic annotation and analysis of visual language data) to make annotations more accurate and efficient. Here we describe the available resources. These data have been used for many types of research in linguistics and in computer-based sign language recognition from video; examples of such research are provided in the latter part of this article.
The Role of Facial Expressions and Emotion in ASL
Venue: arXiv cs.CV 2022
Year: 2022

There is little prior work on quantifying the relationships between facial expressions and emotionality in American Sign Language. In this final report, we provide two methods for studying these relationships through probability and prediction. Using a large corpus of natural signing manually annotated with facial features paired with lexical emotion datasets, we find that there exist many relationships between emotionality and the face, and that a simple classifier can predict what someone is saying in terms of broad emotional categories only by looking at the face.
How Do Socio-Demographic Patterns Define Digital Privacy Divide?
Venue: arXiv cs.CY 2022
Year: 2022

Digital privacy has become an essential component of information and communications technology (ICT) systems. There are many existing methods for digital privacy protection, including network security, cryptography, and access control. However, there is still a gap in the digital privacy protection levels available for users. This paper studies the digital privacy divide (DPD) problem in ICT systems. First, we introduce an online DPD study for understanding the DPD problem by collecting responses from 776 ICT users using crowdsourcing task assignments. Second, we propose a factor analysis-based statistical method for generating the DPD index from a set of observable DPD question variables. In particular, the DPD index provides one scaled measure for the DPD gap by exploring the dimensionality of the eight questions in the DPD survey. Third, we introduce a DPD proportional odds model for analyzing the relationship between the DPD status and the socio-demographic patterns of the users. Our results show that the DPD survey meets the internal consistency reliability with rigorous statistical measures, e.g., Cronbach's $α=0.92$. Furthermore, the DPD index is shown to capture the underlying communality of all DPD variables. Finally, the DPD proportional odds model indicates a strong statistical correlation between the DPD status and the age groups of the ICT users. For example, we find that young users (15-32 years) are generally more concerned about their digital privacy than senior ones (33 years and over).
GASCN: Graph Attention Shape Completion Network
Venue: arXiv cs.CV 2022
Year: 2022

Shape completion, the problem of inferring the complete geometry of an object given a partial point cloud, is an important problem in robotics and computer vision. This paper proposes the Graph Attention Shape Completion Network (GASCN), a novel neural network model that solves this problem. This model combines a graph-based model for encoding local point cloud information with an MLP-based architecture for encoding global information. For each completed point, our model infers the normal and extent of the local surface patch which is used to produce dense yet precise shape completions. We report experiments that demonstrate that GASCN outperforms standard shape completion methods on a standard benchmark drawn from the Shapenet dataset.