OpenRaiser/Intern-Atlas huggingface dataset:
DeepBlur: A Simple and Effective Method for Natural Image Obfuscation
Venue: arXiv cs.CV 2021
Year: 2021

There is a growing privacy concern due to the popularity of social media and surveillance systems, along with advances in face recognition software. However, established image obfuscation techniques are either vulnerable to re-identification attacks by human or deep learning models, insufficient in preserving image fidelity, or too computationally intensive to be practical. To tackle these issues, we present DeepBlur, a simple yet effective method for image obfuscation by blurring in the latent space of an unconditionally pre-trained generative model that is able to synthesize photo-realistic facial images. We compare it with existing methods by efficiency and image quality, and evaluate against both state-of-the-art deep learning models and industrial products (e.g., Face++, Microsoft face service). Experiments show that our method produces high quality outputs and is the strongest defense for most test cases.
Personalized Entity Resolution with Dynamic Heterogeneous Knowledge Graph Representations
Venue: arXiv cs.CL 2021
Year: 2021

The growing popularity of Virtual Assistants poses new challenges for Entity Resolution, the task of linking mentions in text to their referent entities in a knowledge base. Specifically, in the shopping domain, customers tend to use implicit utterances (e.g., "organic milk") rather than explicit names, leading to a large number of candidate products. Meanwhile, for the same query, different customers may expect different results. For example, with "add milk to my cart", a customer may refer to a certain organic product, while some customers may want to re-order products they regularly purchase. To address these issues, we propose a new framework that leverages personalized features to improve the accuracy of product ranking. We first build a cross-source heterogeneous knowledge graph from customer purchase history and product knowledge graph to jointly learn customer and product embeddings. After that, we incorporate product, customer, and history representations into a neural reranking model to predict which candidate is most likely to be purchased for a specific customer. Experiments show that our model substantially improves the accuracy of the top ranked candidates by 24.6% compared to the state-of-the-art product search model.
Conditional physics informed neural networks
Venue: arXiv cs.LG 2021
Year: 2021

We introduce conditional PINNs (physics informed neural networks) for estimating the solution of classes of eigenvalue problems. The concept of PINNs is expanded to learn not only the solution of one particular differential equation but the solutions to a class of problems. We demonstrate this idea by estimating the coercive field of permanent magnets which depends on the width and strength of local defects. When the neural network incorporates the physics of magnetization reversal, training can be achieved in an unsupervised way. There is no need to generate labeled training data. The presented test cases have been rigorously studied in the past. Thus, a detailed and easy comparison with analytical solutions is made. We show that a single deep neural network can learn the solution of partial differential equations for an entire class of problems.
Proof of the Theory-to-Practice Gap in Deep Learning via Sampling Complexity bounds for Neural Network Approximation Spaces
Venue: arXiv cs.LG 2021
Year: 2021

We study the computational complexity of (deterministic or randomized) algorithms based on point samples for approximating or integrating functions that can be well approximated by neural networks. Such algorithms (most prominently stochastic gradient descent and its variants) are used extensively in the field of deep learning. One of the most important problems in this field concerns the question of whether it is possible to realize theoretically provable neural network approximation rates by such algorithms. We answer this question in the negative by proving hardness results for the problems of approximation and integration on a novel class of neural network approximation spaces. In particular, our results confirm a conjectured and empirically observed theory-to-practice gap in deep learning. We complement our hardness results by showing that approximation rates of a comparable order of convergence are (at least theoretically) achievable.
A Quadratic Programming Approach to Manipulation in Real-Time Using Modular Robots
Venue: arXiv cs.RO 2021
Year: 2021

Motion planning in high-dimensional space is a challenging task. In order to perform dexterous manipulation in an unstructured environment, a robot with many degrees of freedom is usually necessary, which also complicates its motion planning problem. Real-time control brings about more difficulties in which robots have to maintain the stability while moving towards the target. Redundant systems are common in modular robots that consist of multiple modules and are able to transformed into different configurations with respect to different needs. Different from robots with fixed geometry or configurations, the kinematics model of a modular robotic system can alter as the robot reconfigures itself, and developing a generic control and motion planning approach for such systems is difficult, especially when multiple motion goals are coupled. A new manipulation planning framework is developed in this paper. The problem is formulated as a sequential linearly constrained quadratic program (QP) that can be solved efficiently. Some constraints can be incorporated into this QP, including a novel way to approximate environment obstacles. This solution can be used directly for real-time applications or as an off-line planning tool, and it is validated and demonstrated on the CKBot and SMORES-EP modular robot platforms.
A New Dimension in Testimony: Relighting Video with Reflectance Field Exemplars
Venue: arXiv cs.CV 2021
Year: 2021

We present a learning-based method for estimating 4D reflectance field of a person given video footage illuminated under a flat-lit environment of the same subject. For training data, we use one light at a time to illuminate the subject and capture the reflectance field data in a variety of poses and viewpoints. We estimate the lighting environment of the input video footage and use the subject's reflectance field to create synthetic images of the subject illuminated by the input lighting environment. We then train a deep convolutional neural network to regress the reflectance field from the synthetic images. We also use a differentiable renderer to provide feedback for the network by matching the relit images with the input video frames. This semi-supervised training scheme allows the neural network to handle unseen poses in the dataset as well as compensate for the lighting estimation error. We evaluate our method on the video footage of the real Holocaust survivors and show that our method outperforms the state-of-the-art methods in both realism and speed.
Autoencoder-based Representation Learning from Heterogeneous Multivariate Time Series Data of Mechatronic Systems
Venue: arXiv cs.LG 2021
Year: 2021

Sensor and control data of modern mechatronic systems are often available as heterogeneous time series with different sampling rates and value ranges. Suitable classification and regression methods from the field of supervised machine learning already exist for predictive tasks, for example in the context of condition monitoring, but their performance scales strongly with the number of labeled training data. Their provision is often associated with high effort in the form of person-hours or additional sensors. In this paper, we present a method for unsupervised feature extraction using autoencoder networks that specifically addresses the heterogeneous nature of the database and reduces the amount of labeled training data required compared to existing methods. Three public datasets of mechatronic systems from different application domains are used to validate the results.
Localization of Autonomous Vehicles: Proof of Concept for A Computer Vision Approach
Venue: arXiv cs.CV 2021
Year: 2021

This paper introduces a visual-based localization method for autonomous vehicles (AVs) that operate in the absence of any complicated hardware system but a single camera. Visual localization refers to techniques that aim to find the location of an object based on visual information of its surrounding area. The problem of localization has been of interest for many years. However, visual localization is a relatively new subject in the literature of transportation. Moreover, the inevitable application of this type of localization in the context of autonomous vehicles demands special attention from the transportation community to this problem. This study proposes a two-step localization method that requires a database of geotagged images and a camera mounted on a vehicle that can take pictures while the car is moving. The first step which is image retrieval uses SIFT local feature descriptor to find an initial location for the vehicle using image matching. The next step is to utilize the Kalman filter to estimate a more accurate location for the vehicle as it is moving. All stages of the introduced method are implemented as a complete system using different Python libraries. The proposed system is tested on the KITTI dataset and has shown an average accuracy of 2 meters in finding the final location of the vehicle.
VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representations
Venue: arXiv cs.CL 2021
Year: 2021

Word vector embeddings have been shown to contain and amplify biases in data they are extracted from. Consequently, many techniques have been proposed to identify, mitigate, and attenuate these biases in word representations. In this paper, we utilize interactive visualization to increase the interpretability and accessibility of a collection of state-of-the-art debiasing techniques. To aid this, we present Visualization of Embedding Representations for deBiasing system ("VERB"), an open-source web-based visualization tool that helps the users gain a technical understanding and visual intuition of the inner workings of debiasing techniques, with a focus on their geometric properties. In particular, VERB offers easy-to-follow use cases in exploring the effects of these debiasing techniques on the geometry of high-dimensional word vectors. To help understand how various debiasing techniques change the underlying geometry, VERB decomposes each technique into interpretable sequences of primitive transformations and highlights their effect on the word vectors using dimensionality reduction and interactive visual exploration. VERB is designed to target natural language processing (NLP) practitioners who are designing decision-making systems on top of word embeddings, and also researchers working with fairness and ethics of machine learning systems in NLP. It can also serve as a visual medium for education, which helps an NLP novice to understand and mitigate biases in word embeddings.
Growing the Simulation Ecosystem: Introducing Mesa Data to Provide Transparent, Accessible and Extensible Data Pipelines for Simulation Development
Venue: arXiv cs.CY 2021
Year: 2021

The Agent Based Model community has a rich and diverse ecosystem of libraries, platforms, and applications to help modelers develop rigorous simulations. Despite this robust and diverse ecosystem, the complexity of life from microbial communities to the global ecosystem still presents substantial challenges in making reusable code that can optimize the ability of the knowledge-sharing and reproducibility. This research seeks to provide new tools to mitigate some of these challenges by offering a vision of a more holistic ecosystem that takes researchers and practitioners from the data collection through validation, with transparent, accessible, and extensible subcomponents. This proposed approach is demonstrated through two data pipelines (crop yield and synthetic population) that take users from data download through the cleaning and processing until users of have data that can be integrated into an ABM. These pipelines are built to be transparent: by walking users step by step through the process, accessible: by being skill scalable so users can leverage them without code or with code, and extensible by being freely available on the coding sharing repository GitHub to facilitate community development. Reusing code that simulates complex phenomena is a significant challenge but one that must be consistently addressed to help the community move forward. This research seeks to aid that progress by offering potential new tools extended from the already robust ecosystem to help the community collaborate more effectively internally and across disciplines.
IndoFashion : Apparel Classification for Indian Ethnic Clothes
Venue: arXiv cs.CV 2021
Year: 2021

Cloth categorization is an important research problem that is used by e-commerce websites for displaying correct products to the end-users. Indian clothes have a large number of clothing categories both for men and women. The traditional Indian clothes like "Saree" and "Dhoti" are worn very differently from western clothes like t-shirts and jeans. Moreover, the style and patterns of ethnic clothes have a very different distribution from western outfits. Thus the models trained on standard cloth datasets fail miserably on ethnic outfits. To address these challenges, we introduce the first large-scale ethnic dataset of over 106k images with 15 different categories for fine-grained classification of Indian ethnic clothes. We gathered a diverse dataset from a large number of Indian e-commerce websites. We then evaluate several baselines for the cloth classification task on our dataset. In the end, we obtain 88.43% classification accuracy. We hope that our dataset would foster research in the development of several algorithms such as cloth classification, landmark detection, especially for ethnic clothes.
Relay-Reconfigurable Intelligent Surface Cooperation for Energy-Efficient Multiuser Systems
Venue: arXiv cs.IT 2021
Year: 2021

Reconfigurable intelligent surfaces (RIS) have drawn considerable attention recently due to their controllable scattering elements that are able to direct electromagnetic waves into desirable directions. Although RISs share some similarities with relays, the two have fundamental differences impacting their performance. To harness the benefits of both relaying and RISs, a multi-user communication system is proposed in this paper wherein a relay and an RIS cooperate to improve performance in terms of energy efficiency. To utilize the RIS efficiently, the discrete phase shifts of the RIS elements are optimized along with the beamforming matrices at the transmitter and the relay, targeting the minimization of the total transmit power subject to a quality-of-service (QoS) constraint. Then, two suboptimal efficient solutions are proposed for the resulting discrete and non-convex problem, one based on singular value decomposition (SVD) and uplink-downlink duality and the other is based on SVD combined with zero-forcing. Simulations show that the proposed solutions outperform a system with either a relay or an RIS only, especially when both are closer to the users than to the base-station.
Quasi-Newton Quasi-Monte Carlo for variational Bayes
Venue: arXiv cs.LG 2021
Year: 2021

Many machine learning problems optimize an objective that must be measured with noise. The primary method is a first order stochastic gradient descent using one or more Monte Carlo (MC) samples at each step. There are settings where ill-conditioning makes second order methods such as L-BFGS more effective. We study the use of randomized quasi-Monte Carlo (RQMC) sampling for such problems. When MC sampling has a root mean squared error (RMSE) of $O(n^{-1/2})$ then RQMC has an RMSE of $o(n^{-1/2})$ that can be close to $O(n^{-3/2})$ in favorable settings. We prove that improved sampling accuracy translates directly to improved optimization. In our empirical investigations for variational Bayes, using RQMC with stochastic L-BFGS greatly speeds up the optimization, and sometimes finds a better parameter value than MC does.
Darts-Conformer: Towards Efficient Gradient-Based Neural Architecture Search For End-to-End ASR
Venue: arXiv cs.SD 2021
Year: 2021

Neural architecture search (NAS) has been successfully applied to tasks like image classification and language modeling for finding efficient high-performance network architectures. In ASR field especially end-to-end ASR, the related research is still in its infancy. In this work, we focus on applying NAS on the most popular manually designed model: Conformer, and then propose an efficient ASR model searching method that benefits from the natural advantage of differentiable architecture search (Darts) in reducing computational overheads. We fuse Darts mutator and Conformer blocks to form a complete search space, within which a modified architecture called Darts-Conformer cell is found automatically. The entire searching process on AISHELL-1 dataset costs only 0.7 GPU days. Replacing the Conformer encoder by stacking searched cell, we get an end-to-end ASR model (named as Darts-Conformner) that outperforms the Conformer baseline by 4.7\% on the open-source AISHELL-1 dataset. Besides, we verify the transferability of the architecture searched on a small dataset to a larger 2k-hour dataset. To the best of our knowledge, this is the first successful attempt to apply gradient-based architecture search in the attention-based encoder-decoder ASR model.
3D Beam Training in Terahertz Communication: A Quadruple-UPA Architecture
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we propose a quadruple-UPA architecture for realizing the beamforming with wide coverage in terahertz (THz) communication. Considering the severe path loss suffered in the THz wave propagation, the pilot signals of conventional channel estimation strategies may not be effectively detected. In sight of this, we propose a fast 3D beam training strategy to jointly realize the angle estimation and the beamforming. Specifically, we first develop a novel hierarchical codebook that predefines some codewords (including narrow beams and wide beams) stage by stage. The narrow beams are designed as the beamforming solution candidates, whereas the wide beams are designed for reducing the training complexity. Then, we propose a 3D beam training procedure to find the optimal beamforming solution, i.e., the optimal narrow-beam pair, in a fast manner. Numerical results are presented to validate the effectiveness of the proposed scheme.
On Salum's Algorithm for $\mathrm{X3SAT}$
Venue: arXiv cs.CC 2021
Year: 2021

This is a commentary on, and critique of, Latif Salum's paper titled "Tractability of One-in-three $\mathrm{3SAT}$: $\mathrm{P} = \mathrm{NP}$." Salum purports to give a polynomial-time algorithm that solves the $\mathrm{NP}$-complete problem $\mathrm{X3SAT}$, thereby claiming $\mathrm{P} = \mathrm{NP}$. The algorithm, in short, fixes the polarity of a variable, carries out simplifications over the resulting formula to decide whether to keep the value assigned or flip the polarity, and repeats with the remaining variables. One thing this algorithm does not do is backtrack. We give an illustrative counterexample showing why the lack of backtracking makes this algorithm flawed.
Polynomial Anonymous Dynamic Distributed Computing without a Unique Leader
Venue: arXiv cs.DC 2021
Year: 2021

Counting the number of nodes in Anonymous Dynamic Networks is enticing from an algorithmic perspective: an important computation in a restricted platform with promising applications. Starting with Michail, Chatzigiannakis, and Spirakis [19], a flurry of papers sped up the running time guarantees from doubly-exponential to polynomial [16]. There is a common theme across all those works: a distinguished node is assumed to be present, because Counting cannot be solved deterministically without at least one. In the present work we study challenging questions that naturally follow: how to efficiently count with more than one distinguished node, or how to count without any distinguished node. More importantly, what is the minimal information needed about these distinguished nodes and what is the best we can aim for (count precision, stochastic guarantees, etc.) without any. We present negative and positive results to answer these questions. To the best of our knowledge, this is the first work that addresses them.
Deep Interpretable Models of Theory of Mind
Venue: arXiv cs.LG 2021
Year: 2021

When developing AI systems that interact with humans, it is essential to design both a system that can understand humans, and a system that humans can understand. Most deep network based agent-modeling approaches are 1) not interpretable and 2) only model external behavior, ignoring internal mental states, which potentially limits their capability for assistance, interventions, discovering false beliefs, etc. To this end, we develop an interpretable modular neural framework for modeling the intentions of other observed entities. We demonstrate the efficacy of our approach with experiments on data from human participants on a search and rescue task in Minecraft, and show that incorporating interpretability can significantly increase predictive performance under the right conditions.
Parameterized Complexity of Elimination Distance to First-Order Logic Properties
Venue: arXiv cs.LO 2021
Year: 2021

The elimination distance to some target graph property P is a general graph modification parameter introduced by Bulian and Dawar. We initiate the study of elimination distances to graph properties expressible in first-order logic. We delimit the problem's fixed-parameter tractability by identifying sufficient and necessary conditions on the structure of prefixes of first-order logic formulas. Our main result is the following meta-theorem: for every graph property P expressible by a first order-logic formula ϕ\in Σ_3, that is, of the form ϕ=\exists x_1\exists x_2\cdots \exists x_r \forall y_1\forall y_2\cdots \forall y_s \exists z_1\exists z_2\cdots \exists z_t ψ, where ψis a quantifier-free first-order formula, checking whether the elimination distance of a graph to P does not exceed k, is fixed-parameter tractable parameterized by k. Properties of graphs expressible by formulas from Σ_3 include being of bounded degree, excluding a forbidden subgraph, or containing a bounded dominating set. We complement this theorem by showing that such a general statement does not hold for formulas with even slightly more expressive prefix structure: there are formulas ϕ\in Π_3, for which computing elimination distance is W[2]-hard.
FedFace: Collaborative Learning of Face Recognition Model
Venue: arXiv cs.CV 2021
Year: 2021

DNN-based face recognition models require large centrally aggregated face datasets for training. However, due to the growing data privacy concerns and legal restrictions, accessing and sharing face datasets has become exceedingly difficult. We propose FedFace, a federated learning (FL) framework for collaborative learning of face recognition models in a privacy-aware manner. FedFace utilizes the face images available on multiple clients to learn an accurate and generalizable face recognition model where the face images stored at each client are neither shared with other clients nor the central host and each client is a mobile device containing face images pertaining to only the owner of the device (one identity per client). Our experiments show the effectiveness of FedFace in enhancing the verification performance of pre-trained face recognition system on standard face verification benchmarks namely LFW, IJB-A, and IJB-C.