OpenRaiser/Intern-Atlas huggingface dataset:
SGToolkit: An Interactive Gesture Authoring Toolkit for Embodied Conversational Agents
Venue: arXiv cs.HC 2021
Year: 2021

Non-verbal behavior is essential for embodied agents like social robots, virtual avatars, and digital humans. Existing behavior authoring approaches including keyframe animation and motion capture are too expensive to use when there are numerous utterances requiring gestures. Automatic generation methods show promising results, but their output quality is not satisfactory yet, and it is hard to modify outputs as a gesture designer wants. We introduce a new gesture generation toolkit, named SGToolkit, which gives a higher quality output than automatic methods and is more efficient than manual authoring. For the toolkit, we propose a neural generative model that synthesizes gestures from speech and accommodates fine-level pose controls and coarse-level style controls from users. The user study with 24 participants showed that the toolkit is favorable over manual authoring, and the generated gestures were also human-like and appropriate to input speech. The SGToolkit is platform agnostic, and the code is available at https://github.com/ai4r/SGToolkit.
Logical Information Cells I
Venue: arXiv cs.AI 2021
Year: 2021

In this study we explore the spontaneous apparition of visible intelligible reasoning in simple artificial networks, and we connect this experimental observation with a notion of semantic information. We start with the reproduction of a DNN model of natural neurons in monkeys, studied by Neromyliotis and Moschovakis in 2017 and 2018, to explain how "motor equivalent neurons", coding only for the action of pointing, are supplemented by other neurons for specifying the actor of the action, the eye E, the hand H, or the eye and the hand together EH. There appear inner neurons performing a logical work, making intermediary proposition, for instance E V EH. Then, we remarked that adding a second hidden layer and choosing a symmetric metric for learning, the activities of the neurons become almost quantized and more informative. Using the work of Carnap and Bar-Hillel 1952, we define a measure of the logical value for collections of such cells. The logical score growths with the depth of the layer, i.e. the information on the output decision increases, which confirms a kind of bottleneck principle. Then we study a bit more complex tasks, a priori involving predicate logic. We compare the logic and the measured weights. This shows, for groups of neurons, a neat correlation between the logical score and the size of the weights. It exhibits a form of sparsity between the layers. The most spectacular result concerns the triples which can conclude for all conditions: when applying their weight matrices to their logical matrix, we recover the classification. This shows that weights precisely perform the proofs.
FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learning
Venue: arXiv cs.LG 2021
Year: 2021

Federated Averaging (FedAvg, also known as Local-SGD) (McMahan et al., 2017) is a classical federated learning algorithm in which clients run multiple local SGD steps before communicating their update to an orchestrating server. We propose a new federated learning algorithm, FedPAGE, able to further reduce the communication complexity by utilizing the recent optimal PAGE method (Li et al., 2021) instead of plain SGD in FedAvg. We show that FedPAGE uses much fewer communication rounds than previous local methods for both federated convex and nonconvex optimization. Concretely, 1) in the convex setting, the number of communication rounds of FedPAGE is $O(\frac{N^{3/4}}{Sε})$, improving the best-known result $O(\frac{N}{Sε})$ of SCAFFOLD (Karimireddy et al.,2020) by a factor of $N^{1/4}$, where $N$ is the total number of clients (usually is very large in federated learning), $S$ is the sampled subset of clients in each communication round, and $ε$ is the target error; 2) in the nonconvex setting, the number of communication rounds of FedPAGE is $O(\frac{\sqrt{N}+S}{Sε^2})$, improving the best-known result $O(\frac{N^{2/3}}{S^{2/3}ε^2})$ of SCAFFOLD (Karimireddy et al.,2020) by a factor of $N^{1/6}S^{1/3}$, if the sampled clients $S\leq \sqrt{N}$. Note that in both settings, the communication cost for each round is the same for both FedPAGE and SCAFFOLD. As a result, FedPAGE achieves new state-of-the-art results in terms of communication complexity for both federated convex and nonconvex optimization.
Multi-Valued Cognitive Maps: Calculations with Linguistic Variables without Using Numbers
Venue: arXiv cs.AI 2021
Year: 2021

A concept of multi-valued cognitive maps is introduced in this paper. The concept expands the fuzzy one. However, all variables and weights are not linearly ordered in the concept, but are only partially-ordered. Such an ap- proach allows us to operate in cognitive maps with partially-ordered linguis- tic variables directly, without vague fuzzification/defuzzification methods. Hence, we may consider more subtle differences in degrees of experts' uncer- tainty, than in the fuzzy case. We prove the convergence of such cognitive maps and give a simple computational example which demonstrates using such a partially-ordered uncertainty degree scale.
Analyzing Effects of The COVID-19 Pandemic on Road Traffic Safety: The Cases of New York City, Los Angeles, and Boston
Venue: arXiv cs.LG 2021
Year: 2021

The COVID-19 pandemic has resulted in significant social and economic impacts throughout the world. In addition to the health consequences, the impacts on traffic behaviors have also been sudden and dramatic. We have analyzed how the road traffic safety of New York City, Los Angeles, and Boston in the U.S. have been impacted by the pandemic and corresponding local government orders and restrictions. To be specific, we have studied the accident hotspots' distributions before and after the outbreak of the pandemic and found that traffic accidents have shifted in both location and time compared to previous years. In addition, we have studied the road network characteristics in those hotspot regions with the hope to understand the underlying cause of the hotspot shifts.
The Effect of the Loss on Generalization: Empirical Study on Synthetic Lung Nodule Data
Venue: arXiv cs.CV 2021
Year: 2021

Convolutional Neural Networks (CNNs) are widely used for image classification in a variety of fields, including medical imaging. While most studies deploy cross-entropy as the loss function in such tasks, a growing number of approaches have turned to a family of contrastive learning-based losses. Even though performance metrics such as accuracy, sensitivity and specificity are regularly used for the evaluation of CNN classifiers, the features that these classifiers actually learn are rarely identified and their effect on the classification performance on out-of-distribution test samples is insufficiently explored. In this paper, motivated by the real-world task of lung nodule classification, we investigate the features that a CNN learns when trained and tested on different distributions of a synthetic dataset with controlled modes of variation. We show that different loss functions lead to different features being learned and consequently affect the generalization ability of the classifier on unseen data. This study provides some important insights into the design of deep learning solutions for medical imaging tasks.
Impact of Dietary Habits and Opinionated Lifestyle during COVID-19 Pandemic : A Case Study on Engineering Students
Venue: arXiv cs.SI 2021
Year: 2021

COVID-19 pandemic has introduced a new lifestyle due to lockdown. The impact was on food habits, working hours, and sleeping patterns. The goal of this study is to detect lifestyle changes caused by confinement during the COVID-19 pandemic, such as dietary habits, physical activities, and to explore changes in the body weight. A structured questionnaire was used in the study to collect anthropometric data; daily consumption of particular foods, water intake, food frequency, and number of meals/day. The data is presented in a graph illustration to show health , lifestyle trends and the friendship among engineering students.
Self-supervised Consensus Representation Learning for Attributed Graph
Venue: arXiv cs.SI 2021
Year: 2021

Attempting to fully exploit the rich information of topological structure and node features for attributed graph, we introduce self-supervised learning mechanism to graph representation learning and propose a novel Self-supervised Consensus Representation Learning (SCRL) framework. In contrast to most existing works that only explore one graph, our proposed SCRL method treats graph from two perspectives: topology graph and feature graph. We argue that their embeddings should share some common information, which could serve as a supervisory signal. Specifically, we construct the feature graph of node features via k-nearest neighbor algorithm. Then graph convolutional network (GCN) encoders extract features from two graphs respectively. Self-supervised loss is designed to maximize the agreement of the embeddings of the same node in the topology graph and the feature graph. Extensive experiments on real citation networks and social networks demonstrate the superiority of our proposed SCRL over the state-of-the-art methods on semi-supervised node classification task. Meanwhile, compared with its main competitors, SCRL is rather efficient.
Contrained Generalization For Data Anonymization - A Systematic Search Based Approach
Venue: arXiv cs.DB 2021
Year: 2021

Data generalization is a powerful technique for sanitizing multi-attribute data for publication. In a multidimensional model, a subset of attributes called the quasi-identifiers (QI) are used to define the space and a generalization scheme corresponds to a partitioning of the data space. The process of sanitization can be modeled as a constrained optimization problem where the information loss metric is to be minimized while ensuring that the privacy criteria are enforced. The privacy requirements translate into constraints on the partitions (bins), like minimum occupancy constraints for k-anonymity, value diversity constraint for l-diversity etc. Most algorithms proposed till date use some greedy search heuristic to search for a locally optimal generalization scheme. The performance of such algorithms degrade rapidly as the constraints are made more complex and numerous. To address this issue, in this paper we develop a complete enumeration based systematic search framework that searches for the globally optimal generalization scheme amongst all feasible candidates. We employ a novel enumeration technique that eliminates duplicates and develop effective pruning heuristics that cut down the solution space in order to make the search tractable. Our scheme is versatile enough to accommodate multiple constraints and information loss functions satisfying a set of generic properties (that are usually satisfied by most metrics proposed in literature). Additionally, our approach allows the user to specify various stopping criteria and can give a bound on the approximation factor achieved by any candidate solution. Finally, we carry out extensive experimentation whose results illustrate the power of our algorithm and its advantage over other competing approaches.
Depth Infused Binaural Audio Generation using Hierarchical Cross-Modal Attention
Venue: arXiv cs.SD 2021
Year: 2021

Binaural audio gives the listener the feeling of being in the recording place and enhances the immersive experience if coupled with AR/VR. But the problem with binaural audio recording is that it requires a specialized setup which is not possible to fabricate within handheld devices as compared to traditional mono audio that can be recorded with a single microphone. In order to overcome this drawback, prior works have tried to uplift the mono recorded audio to binaural audio as a post processing step conditioning on the visual input. But all the prior approaches missed other most important information required for the task, i.e. distance of different sound producing objects from the recording setup. In this work, we argue that the depth map of the scene can act as a proxy for encoding distance information of objects in the scene and show that adding depth features along with image features improves the performance both qualitatively and quantitatively. We propose a novel encoder-decoder architecture, where we use a hierarchical attention mechanism to encode the image and depth feature extracted from individual transformer backbone, with audio features at each layer of the decoder.
SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)
Venue: arXiv cs.CR 2021
Year: 2021

Deep Neural Network (DNN) watermarking is a method for provenance verification of DNN models. Watermarking should be robust against watermark removal attacks that derive a surrogate model that evades provenance verification. Many watermarking schemes that claim robustness have been proposed, but their robustness is only validated in isolation against a relatively small set of attacks. There is no systematic, empirical evaluation of these claims against a common, comprehensive set of removal attacks. This uncertainty about a watermarking scheme's robustness causes difficulty to trust their deployment in practice. In this paper, we evaluate whether recently proposed watermarking schemes that claim robustness are robust against a large set of removal attacks. We survey methods from the literature that (i) are known removal attacks, (ii) derive surrogate models but have not been evaluated as removal attacks, and (iii) novel removal attacks. Weight shifting and smooth retraining are novel removal attacks adapted to the DNN watermarking schemes surveyed in this paper. We propose taxonomies for watermarking schemes and removal attacks. Our empirical evaluation includes an ablation study over sets of parameters for each attack and watermarking scheme on the CIFAR-10 and ImageNet datasets. Surprisingly, none of the surveyed watermarking schemes is robust in practice. We find that schemes fail to withstand adaptive attacks and known methods for deriving surrogate models that have not been evaluated as removal attacks. This points to intrinsic flaws in how robustness is currently evaluated. We show that watermarking schemes need to be evaluated against a more extensive set of removal attacks with a more realistic adversary model. Our source code and a complete dataset of evaluation results are publicly available, which allows to independently verify our conclusions.
Deep Pairwise Learning To Rank For Search Autocomplete
Venue: arXiv cs.IR 2021
Year: 2021

Autocomplete (a.k.a "Query Auto-Completion", "AC") suggests full queries based on a prefix typed by customer. Autocomplete has been a core feature of commercial search engine. In this paper, we propose a novel context-aware neural network based pairwise ranker (DeepPLTR) to improve AC ranking, DeepPLTR leverages contextual and behavioral features to rank queries by minimizing a pairwise loss, based on a fully-connected neural network structure. Compared to LambdaMART ranker, DeepPLTR shows +3.90% MeanReciprocalRank (MRR) lift in offline evaluation, and yielded +0.06% (p < 0.1) Gross Merchandise Value (GMV) lift in an Amazon's online A/B experiment.
Winning the NIST Contest: A scalable and general approach to differentially private synthetic data
Venue: arXiv cs.CR 2021
Year: 2021

We propose a general approach for differentially private synthetic data generation, that consists of three steps: (1) select a collection of low-dimensional marginals, (2) measure those marginals with a noise addition mechanism, and (3) generate synthetic data that preserves the measured marginals well. Central to this approach is Private-PGM, a post-processing method that is used to estimate a high-dimensional data distribution from noisy measurements of its marginals. We present two mechanisms, NIST-MST and MST, that are instances of this general approach. NIST-MST was the winning mechanism in the 2018 NIST differential privacy synthetic data competition, and MST is a new mechanism that can work in more general settings, while still performing comparably to NIST-MST. We believe our general approach should be of broad interest, and can be adopted in future mechanisms for synthetic data generation.
Shannon capacity, Chess, DNA and Umbrellas
Venue: arXiv cs.IT 2021
Year: 2021

A vexing open problem in information theory is to find the Shannon capacity of odd cyclic graphs larger than the pentagon and especially for the heptagon. Lower bounds for the capacity are obtained by solving King chess puzzles. Upper bounds are obtained by solving entanglement problems, that is to find good Lovasz umbrellas, quantum state realizations of the graph. We observe that optimal states are always pure states. The rest is expository. One general interesting question is whether the Shannon capacity is always some n-th root of the independence number of the n'th power of the graph.
Frequency-based tension assessment of an inclined cable with complex boundary conditions using the PSO algorithm
Venue: arXiv cs.AI 2021
Year: 2021

The frequency-based method is the most commonly used method for measuring cable tension. However, the calculation formulas for the conventional frequency-based method are generally based on the ideally hinged or fixed boundary conditions without a comprehensive consideration of the inclination angle, sag-extensibility, and flexural stiffness of cables, leading to a significant error in cable tension identification. This study aimed to propose a frequency-based method of cable tension identification considering the complex boundary conditions at the two ends of cables using the particle swarm optimization (PSO) algorithm. First, the refined stay cable model was established considering the inclination angle, flexural stiffness, and sag-extensibility, as well as the rotational constraint stiffness and lateral support stiffness for the unknown boundaries of cables. The vibration mode equation of the stay cable model was discretized and solved using the finite difference method. Then, a multiparameter identification method based on the PSO algorithm was proposed. This method was able to identify the tension, flexural stiffness, axial stiffness, boundary rotational constraint stiffness, and boundary lateral support stiffness according to the measured multiorder frequencies in a synchronous manner. The feasibility and accuracy of this method were validated through numerical cases. Finally, the proposed approach was applied to the tension identification of the anchor span strands of a suspension bridge (Jindong Bridge) in China. The results of cable tension identification using the proposed method and the existing methods discussed in previous studies were compared with the on-site pressure ring measurement results. The comparison showed that the proposed approach had a high accuracy in cable tension identification.
Parallel algorithms for mining of frequent itemsets
Venue: arXiv cs.DB 2021
Year: 2021

In the recent decade companies started collecting of large amount of data. Without a proper analyse, the data are usually useless. The field of analysing the data is called data mining. Unfortunately, the amount of data is quite large: the data do not fit into main memory and the processing time can become quite huge. Therefore, we need parallel data mining algorithms. One of the popular and important data mining algorithm is the algorithm for generation of so called frequent itemsets. The problem of mining of frequent itemsets can be explained on the following example: customers goes in a store put into theirs baskets some goods; the owner of the store collects the baskets and wants to know the set of goods that are bought together in at least p% of the baskets. Currently, the sequential algorithms for mining of frequent itemsets are quite good in the means of performance. However, the parallel algorithms for mining of frequent itemsets still do not achieve good speedup. In this thesis, we develop a parallel method for mining of frequent itemsets that can be used for an arbitrary depth first search sequential algorithms on a distributed memory parallel computer. Our method achieves speedup of ~ 6 on 10 processors. The method is based on an approximate estimation of processor load from a database sample - however it always computes the set of frequent itemsets from the whole database. In this thesis, we show a theory underlying our method and show the performance of the estimation process.
Group LCD and Group Reversible LCD Codes
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we give a new method for constructing LCD codes. We employ group rings and a well known map that sends group ring elements to a subring of the $n \times n$ matrices to obtain LCD codes. Our construction method guarantees that our LCD codes are also group codes, namely, the codes are ideals in a group ring. We show that with a certain condition on the group ring element $v,$ one can construct non-trivial group LCD codes. Moreover, we also show that by adding more constraints on the group ring element $v,$ one can construct group LCD codes that are reversible. We present many examples of binary group LCD codes of which some are optimal and group reversible LCD codes with different parameters.
Multi-Source Fusion and Automatic Predictor Selection for Zero-Shot Video Object Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Location and appearance are the key cues for video object segmentation. Many sources such as RGB, depth, optical flow and static saliency can provide useful information about the objects. However, existing approaches only utilize the RGB or RGB and optical flow. In this paper, we propose a novel multi-source fusion network for zero-shot video object segmentation. With the help of interoceptive spatial attention module (ISAM), spatial importance of each source is highlighted. Furthermore, we design a feature purification module (FPM) to filter the inter-source incompatible features. By the ISAM and FPM, the multi-source features are effectively fused. In addition, we put forward an automatic predictor selection network (APS) to select the better prediction of either the static saliency predictor or the moving object predictor in order to prevent over-reliance on the failed results caused by low-quality optical flow maps. Extensive experiments on three challenging public benchmarks (i.e. DAVIS$_{16}$, Youtube-Objects and FBMS) show that the proposed model achieves compelling performance against the state-of-the-arts. The source code will be publicly available at \textcolor{red}{\url{https://github.com/Xiaoqi-Zhao-DLUT/Multi-Source-APS-ZVOS}}.
Capture Uncertainties in Deep Neural Networks for Safe Operation of Autonomous Driving Vehicles
Venue: arXiv cs.RO 2021
Year: 2021

Uncertainties in Deep Neural Network (DNN)-based perception and vehicle's motion pose challenges to the development of safe autonomous driving vehicles. In this paper, we propose a safe motion planning framework featuring the quantification and propagation of DNN-based perception uncertainties and motion uncertainties. Contributions of this work are twofold: (1) A Bayesian Deep Neural network model which detects 3D objects and quantitatively captures the associated aleatoric and epistemic uncertainties of DNNs; (2) An uncertainty-aware motion planning algorithm (PU-RRT) that accounts for uncertainties in object detection and ego-vehicle's motion. The proposed approaches are validated via simulated complex scenarios built in CARLA. Experimental results show that the proposed motion planning scheme can cope with uncertainties of DNN-based perception and vehicle motion, and improve the operational safety of autonomous vehicles while still achieving desirable efficiency.