OpenRaiser/Intern-Atlas huggingface dataset:
Data-Driven 3D Reconstruction of Dressed Humans From Sparse Views
Venue: arXiv cs.CV 2021
Year: 2021

Recently, data-driven single-view reconstruction methods have shown great progress in modeling 3D dressed humans. However, such methods suffer heavily from depth ambiguities and occlusions inherent to single view inputs. In this paper, we tackle this problem by considering a small set of input views and investigate the best strategy to suitably exploit information from these views. We propose a data-driven end-to-end approach that reconstructs an implicit 3D representation of dressed humans from sparse camera views. Specifically, we introduce three key components: first a spatially consistent reconstruction that allows for arbitrary placement of the person in the input views using a perspective camera model; second an attention-based fusion layer that learns to aggregate visual information from several viewpoints; and third a mechanism that encodes local 3D patterns under the multi-view context. In the experiments, we show the proposed approach outperforms the state of the art on standard data both quantitatively and qualitatively. To demonstrate the spatially consistent reconstruction, we apply our approach to dynamic scenes. Additionally, we apply our method on real data acquired with a multi-camera platform and demonstrate our approach can obtain results comparable to multi-view stereo with dramatically less views.
Denial of Wallet -- Defining a Looming Threat to Serverless Computing
Venue: arXiv cs.CR 2021
Year: 2021

Serverless computing is the latest paradigm in cloud computing, offering a framework for the development of event driven, pay-as-you-go functions in a highly scalable environment. While these traits offer a powerful new development paradigm, they have also given rise to a new form of cyber-attack known as Denial of Wallet (forced financial exhaustion). In this work, we define and identify the threat of Denial of Wallet and its potential attack patterns. Also, we demonstrate how this new form of attack can potentially circumvent existing mitigation systems developed for a similar style of attack, Denial of Service. Our goal is twofold. Firstly, we will provide a concise and informative overview of this emerging attack paradigm. Secondly, we propose this paper as a starting point to enable researchers and service providers to create effective mitigation strategies. We include some simulated experiments to highlight the potential financial damage that such attacks can cause and the creation of an isolated test bed for continued safe research on these attacks.
DREAM Lite: Simplifying Robot Assisted Therapy for ASD
Venue: arXiv cs.RO 2021
Year: 2021

Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD). The DREAM project explores how to deliver effective RAT interventions to ASD children within a supervisedautonomy framework for controlling the robotic agent, which could decrease the burden on the clinicians delivering such interventions. In this paper we describe how to use in real life settings the experimental protocols that were developed and extensively tested in the DREAM Project, as well as their deployment and validation in a new ecological study conducted by clinicians.
Signed Distance Function Computation from an Implicit Surface
Venue: arXiv cs.GR 2021
Year: 2021

We describe in this short note a technique to convert an implicit surface into a Signed Distance Function (SDF) while exactly preserving the zero level-set of the implicit. The proposed approach relies on embedding the input implicit in the final layer of a neural network, which is trained to minimize a loss function characterizing the SDF.
Introduction to Big data Technology
Venue: arXiv cs.OH 2021
Year: 2021

Big data is no more "all just hype" but widely applied in nearly all aspects of our business, governments, and organizations with the technology stack of AI. Its influences are far beyond a simple technique innovation but involves all rears in the world. This chapter will first have historical review of big data; followed by discussion of characteristics of big data, i.e. from the 3V's to up 10V's of big data. The chapter then introduces technology stacks for an or-ganization to build a big data application, from infrastruc-ture/platform/ecosystem to constructional units and components. Finally, we provide some big data online resources for reference.
Multiple feature fusion-based video face tracking for IoT big data
Venue: arXiv cs.CV 2021
Year: 2021

With the advancement of IoT and artificial intelligence technologies, and the need for rapid application growth in fields such as security entrance control and financial business trade, facial information processing has become an important means for achieving identity authentication and information security. In this paper, we propose a multi-feature fusion algorithm based on integral histograms and a real-time update tracking particle filtering module. First, edge and colour features are extracted, weighting methods are used to weight the colour histogram and edge features to describe facial features, and fusion of colour and edge features is made adaptive by using fusion coefficients to improve face tracking reliability. Then, the integral histogram is integrated into the particle filtering algorithm to simplify the calculation steps of complex particles. Finally, the tracking window size is adjusted in real time according to the change in the average distance from the particle centre to the edge of the current model and the initial model to reduce the drift problem and achieve stable tracking with significant changes in the target dimension. The results show that the algorithm improves video tracking accuracy, simplifies particle operation complexity, improves the speed, and has good anti-interference ability and robustness.
Split Learning Meets Koopman Theory for Wireless Remote Monitoring and Prediction
Venue: arXiv cs.LG 2021
Year: 2021

Remote state monitoring over wireless is envisaged to play a pivotal role in enabling beyond 5G applications ranging from remote drone control to remote surgery. One key challenge is to identify the system dynamics that is non-linear with a large dimensional state. To obviate this issue, in this article we propose to train an autoencoder whose encoder and decoder are split and stored at a state sensor and its remote observer, respectively. This autoencoder not only decreases the remote monitoring payload size by reducing the state representation dimension, but also learns the system dynamics by lifting it via a Koopman operator, thereby allowing the observer to locally predict future states after training convergence. Numerical results under a non-linear cart-pole environment demonstrate that the proposed split learning of a Koopman autoencoder can locally predict future states, and the prediction accuracy increases with the representation dimension and transmission power.
$\{log\}$: Set Formulas as Programs
Venue: arXiv cs.LO 2021
Year: 2021

$\{log\}$ is a programming language at the intersection of Constraint Logic Programming, set programming and declarative programming. But $\{log\}$ is also a satisfiability solver for a theory of finite sets and finite binary relations. With $\{log\}$ programmers can write abstract programs using all the power of set theory and binary relations. These programs are not very efficient but they are very close to specifications. Then, their correctness is more evident. Furthermore, $\{log\}$ programs are also set formulas. Hence, programmers can use $\{log\}$ again to automatically prove their programs verify non trivial properties. In this paper we show this development methodology by means of several examples.
Governance for Security, Risks, Competition and Cooperation: Mapping the knowledge
Venue: arXiv cs.DL 2021
Year: 2021

The study aims to generate a map of the knowledge based on the research on topics related to governance and security, risks, competition and cooperation for the FDDI (Fudan Development Institute) proceedings publishing project: 'Reflections on Governance: Security and Risks, Competition and Cooperation.' That mapping exercise would enable a broader audience to delve into the current state, and interdisciplinary pathways of the research published worldwide for addressing complex problems of governance. Following this introduction, the second section presents the bibliometric methods used and the results' interpretation. The third section presents the results, followed by the fourth and fifth sections of discussion and conclusion, respectively.
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema
Venue: arXiv cs.CL 2021
Year: 2021

The Winograd Schema (WS) has been proposed as a test for measuring commonsense capabilities of models. Recently, pre-trained language model-based approaches have boosted performance on some WS benchmarks but the source of improvement is still not clear. This paper suggests that the apparent progress on WS may not necessarily reflect progress in commonsense reasoning. To support this claim, we first show that the current evaluation method of WS is sub-optimal and propose a modification that uses twin sentences for evaluation. We also propose two new baselines that indicate the existence of artifacts in WS benchmarks. We then develop a method for evaluating WS-like sentences in a zero-shot setting to account for the commonsense reasoning abilities acquired during the pretraining and observe that popular language models perform randomly in this setting when using our more strict evaluation. We conclude that the observed progress is mostly due to the use of supervision in training WS models, which is not likely to successfully support all the required commonsense reasoning skills and knowledge.
CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning (FL) is an emerging distributed machine learning paradigm that protects privacy and tackles the problem of isolated data islands. At present, there are two main communication strategies of FL: synchronous FL and asynchronous FL. The advantages of synchronous FL are that the model has high precision and fast convergence speed. However, this synchronous communication strategy has the risk that the central server waits too long for the devices, namely, the straggler effect which has a negative impact on some time-critical applications. Asynchronous FL has a natural advantage in mitigating the straggler effect, but there are threats of model quality degradation and server crash. Therefore, we combine the advantages of these two strategies to propose a clustered semi-asynchronous federated learning (CSAFL) framework. We evaluate CSAFL based on four imbalanced federated datasets in a non-IID setting and compare CSAFL to the baseline methods. The experimental results show that CSAFL significantly improves test accuracy by more than +5% on the four datasets compared to TA-FedAvg. In particular, CSAFL improves absolute test accuracy by +34.4% on non-IID FEMNIST compared to TA-FedAvg.
Autoencoder-Based Unequal Error Protection Codes
Venue: arXiv cs.IT 2021
Year: 2021

We present a novel autoencoder-based approach for designing codes that provide unequal error protection (UEP) capabilities. The proposed design is based on a generalization of an autoencoder loss function that accommodates both message-wise and bit-wise UEP scenarios. In both scenarios, the generalized loss function can be adjusted using an associated weight vector to trade off error probabilities corresponding to different importance classes. For message-wise UEP, we compare the proposed autoencoder-based UEP codes with a union of random coset codes. For bit-wise UEP, the proposed codes are compared with UEP rateless spinal codes and the superposition of random Gaussian codes. In all cases, the autoencoder-based codes show superior performance while providing design simplicity and flexibility in trading off error protection among different importance classes.
Controlled abstention neural networks for identifying skillful predictions for regression problems
Venue: arXiv cs.LG 2021
Year: 2021

The earth system is exceedingly complex and often chaotic in nature, making prediction incredibly challenging: we cannot expect to make perfect predictions all of the time. Instead, we look for specific states of the system that lead to more predictable behavior than others, often termed "forecasts of opportunity". When these opportunities are not present, scientists need prediction systems that are capable of saying "I don't know." We introduce a novel loss function, termed "abstention loss", that allows neural networks to identify forecasts of opportunity for regression problems. The abstention loss works by incorporating uncertainty in the network's prediction to identify the more confident samples and abstain (say "I don't know") on the less confident samples. The abstention loss is designed to determine the optimal abstention fraction, or abstain on a user-defined fraction via a PID controller. Unlike many methods for attaching uncertainty to neural network predictions post-training, the abstention loss is applied during training to preferentially learn from the more confident samples. The abstention loss is built upon a standard computer science method. While the standard approach is itself a simple yet powerful tool for incorporating uncertainty in regression problems, we demonstrate that the abstention loss outperforms this more standard method for the synthetic climate use cases explored here. The implementation of proposed loss function is straightforward in most network architectures designed for regression, as it only requires modification of the output layer and loss function.
Learning Evolved Combinatorial Symbols with a Neuro-symbolic Generative Model
Venue: arXiv cs.CL 2021
Year: 2021

Humans have the ability to rapidly understand rich combinatorial concepts from limited data. Here we investigate this ability in the context of auditory signals, which have been evolved in a cultural transmission experiment to study the emergence of combinatorial structure in language. We propose a neuro-symbolic generative model which combines the strengths of previous approaches to concept learning. Our model performs fast inference drawing on neural network methods, while still retaining the interpretability and generalization from limited data seen in structured generative approaches. This model outperforms a purely neural network-based approach on classification as evaluated against both ground truth and human experimental classification preferences, and produces superior reproductions of observed signals as well. Our results demonstrate the power of flexible combined neural-symbolic architectures for human-like generalization in raw perceptual domains and offers a step towards developing precise computational models of inductive biases in language evolution.
Learning to Boost the Efficiency of Modern Code Review
Venue: arXiv cs.SE 2021
Year: 2021

Modern Code Review (MCR) is a standard in all kinds of organizations that develop software. MCR pays for itself through perceived and proven benefits in quality assurance and knowledge transfer. However, the time invest in MCR is generally substantial. The goal of this thesis is to boost the efficiency of MCR by developing AI techniques that can partially replace or assist human reviewers. The envisioned techniques distinguish from existing MCR-related AI models in that we interpret these challenges as graph-learning problems. This should allow us to use state-of-science algorithms from that domain to learn coding and reviewing standards directly from existing projects. The required training data will be mined from online repositories and the experiments will be designed to use standard, quantitative evaluation metrics. This research proposal defines the motivation, research-questions, and solution components for the thesis, and gives an overview of the relevant related work.
Controller Synthesis for Multi-Agent Systems with Intermittent Communication and Metric Temporal Logic Specifications
Venue: arXiv cs.RO 2021
Year: 2021

This paper investigates the controller synthesis problem for a multi-agent system (MAS) with intermittent communication. We adopt a relay-explorer scheme, where a mobile relay agent with absolute position sensors switches among a set of explorers with relative position sensors to provide intermittent state information. We model the MAS as a switched system where the explorers' dynamics can be either fully-actuated or underactuated. The objective of the explorers is to reach approximate consensus to a predetermined goal region. To guarantee the stability of the switched system and the approximate consensus of the explorers, we derive maximum dwell-time conditions to constrain the length of time each explorer goes without state feedback (from the relay agent). Furthermore, the relay agent needs to satisfy practical constraints such as charging its battery and staying in specific regions of interest. Both the maximum dwell-time conditions and these practical constraints can be expressed by metric temporal logic (MTL) specifications. We iteratively compute the optimal control inputs for the relay agent to satisfy the MTL specifications, while guaranteeing stability and approximate consensus of the explorers. We implement the proposed method on a case study with the CoppeliaSim robot simulator.
Identification of mental fatigue in language comprehension tasks based on EEG and deep learning
Venue: arXiv cs.AI 2021
Year: 2021

Mental fatigue increases the risk of operator error in language comprehension tasks. In order to prevent operator performance degradation, we used EEG signals to assess the mental fatigue of operators in human-computer systems. This study presents an experimental design for fatigue detection in language comprehension tasks. We obtained EEG signals from a 14-channel wireless EEG detector in 15 healthy participants. Each participant was given a cognitive test of a language comprehension task, in the form of multiple choice questions, in which pronoun references were selected between nominal and surrogate sentences. In this paper, the 2400 EEG fragments collected are divided into three data sets according to different utilization rates, namely 1200s data set with 50% utilization rate, 1500s data set with 62.5% utilization rate, and 1800s data set with 75% utilization rate. In the aspect of feature extraction, different EEG features were extracted, including time domain features, frequency domain features and entropy features, and the effects of different features and feature combinations on classification accuracy were explored. In terms of classification, we introduced the Convolutional Neural Network (CNN) method as the preferred method, It was compared with Least Squares Support Vector Machines(LSSVM),Support Vector Machines(SVM),Logistic Regression (LR), Random Forest(RF), Naive Bayes (NB), K-Nearest Neighbor (KNN) and Decision Tree(DT).According to the results, the classification accuracy of convolutional neural network (CNN) is higher than that of other classification methods. The classification results show that the classification accuracy of 1200S dataset is higher than the other two datasets. The combination of Frequency and entropy feature and CNN has the highest classification accuracy, which is 85.34%.
I Only Have Eyes for You: The Impact of Masks On Convolutional-Based Facial Expression Recognition
Venue: arXiv cs.CV 2021
Year: 2021

The current COVID-19 pandemic has shown us that we are still facing unpredictable challenges in our society. The necessary constrain on social interactions affected heavily how we envision and prepare the future of social robots and artificial agents in general. Adapting current affective perception models towards constrained perception based on the hard separation between facial perception and affective understanding would help us to provide robust systems. In this paper, we perform an in-depth analysis of how recognizing affect from persons with masks differs from general facial expression perception. We evaluate how the recently proposed FaceChannel adapts towards recognizing facial expressions from persons with masks. In Our analysis, we evaluate different training and fine-tuning schemes to understand better the impact of masked facial expressions. We also perform specific feature-level visualization to demonstrate how the inherent capabilities of the FaceChannel to learn and combine facial features change when in a constrained social interaction scenario.
Hercule: Representing and Reasoning about Norms as a Foundation for Declarative Contracts over Blockchain
Venue: arXiv cs.MA 2021
Year: 2021

Current blockchain approaches for business contracts are based on smart contracts, namely, software programs placed on a blockchain that are automatically executed to realize a contract. However, smart contracts lack flexibility and interfere with the autonomy of the parties concerned. We propose Hercule, an approach for declaratively specifying blockchain applications in a manner that reflects business contracts. Hercule represents a contract via regulatory norms that capture the involved parties' expectations of one another. It computes the states of norms (hence, of contracts) from events in the blockchain. Hercule's novelty and significance lie in that it operationalizes declarative contracts over semistructured databases, the underlying representation for practical blockchain such as Hyperledger Fabric and Ethereum. Specifically, it exploits the map-reduce capabilities of such stores to compute norm states. We demonstrate that our implementation over Hyperledger Fabric can process thousands of events per second, sufficient for many applications.
Accurate 3D Facial Geometry Prediction by Multi-Task, Multi-Modal, and Multi-Representation Landmark Refinement Network
Venue: arXiv cs.CV 2021
Year: 2021

This work focuses on complete 3D facial geometry prediction, including 3D facial alignment via 3D face modeling and face orientation estimation using the proposed multi-task, multi-modal, and multi-representation landmark refinement network (M$^3$-LRN). Our focus is on the important facial attributes, 3D landmarks, and we fully utilize their embedded information to guide 3D facial geometry learning. We first propose a multi-modal and multi-representation feature aggregation for landmark refinement. Next, we are the first to study 3DMM regression from sparse 3D landmarks and utilize multi-representation advantage to attain better geometry prediction. We attain the state of the art from extensive experiments on all tasks of learning 3D facial geometry. We closely validate contributions of each modality and representation. Our results are robust across cropped faces, underwater scenarios, and extreme poses. Specially we adopt only simple and widely used network operations in M$^3$-LRN and attain a near 20\% improvement on face orientation estimation over the current best performance. See our project page here.