OpenRaiser/Intern-Atlas huggingface dataset:
Joint detection and matching of feature points in multimodal images
Venue: arXiv cs.CV 2021
Year: 2021

In this work, we propose a novel Convolutional Neural Network (CNN) architecture for the joint detection and matching of feature points in images acquired by different sensors using a single forward pass. The resulting feature detector is tightly coupled with the feature descriptor, in contrast to classical approaches (SIFT, etc.), where the detection phase precedes and differs from computing the descriptor. Our approach utilizes two CNN subnetworks, the first being a Siamese CNN and the second, consisting of dual non-weight-sharing CNNs. This allows simultaneous processing and fusion of the joint and disjoint cues in the multimodal image patches. The proposed approach is experimentally shown to outperform contemporary state-of-the-art schemes when applied to multiple datasets of multimodal images. It is also shown to provide repeatable feature points detections across multisensor images, outperforming state-of-the-art detectors. To the best of our knowledge, it is the first unified approach for the detection and matching of such images.
DeepTileBars: Visualizing Term Distribution for Neural Information Retrieval
Venue: arXiv cs.IR 2021
Year: 2021

Most neural Information Retrieval (Neu-IR) models derive query-to-document ranking scores based on term-level matching. Inspired by TileBars, a classical term distribution visualization method, in this paper, we propose a novel Neu-IR model that handles query-to-document matching at the subtopic and higher levels. Our system first splits the documents into topical segments, "visualizes" the matchings between the query and the segments, and then feeds an interaction matrix into a Neu-IR model, DeepTileBars, to obtain the final ranking scores. DeepTileBars models the relevance signals occurring at different granularities in a document's topic hierarchy. It better captures the discourse structure of a document and thus the matching patterns. Although its design and implementation are light-weight, DeepTileBars outperforms other state-of-the-art Neu-IR models on benchmark datasets including the Text REtrieval Conference (TREC) 2010-2012 Web Tracks and LETOR 4.0.
Learning Representations from Product Titles for Modeling Shopping Transactions
Venue: arXiv cs.IR 2021
Year: 2021

Shopping transaction analysis is important for understanding the shopping behaviors of customers. Existing models such as association rules are poor at modeling products that have short purchase histories and cannot be applied to new products (the cold-start problem). In this paper, we propose BASTEXT, an efficient model of shopping baskets and the texts associated with the products (e.g., product titles). The model's goal is to learn the product representations from the textual contents to capture the relationships between the products in the baskets. Given the products already in a basket, a classifier identifies whether a potential product is relevant to the basket based on their vector representations. This relevancy enables us to learn high-quality representations of the products. The experiments demonstrate that BASTEXT can efficiently model millions of baskets and that it outperforms the state-of-the-art methods in the next product recommendation task. We also show that BASTEXT is a strong baseline for keyword-based product search.
Deep Ad-hoc Beamforming
Venue: arXiv cs.SD 2021
Year: 2021

Far-field speech processing is an important and challenging problem. In this paper, we propose \textit{deep ad-hoc beamforming}, a deep-learning-based multichannel speech enhancement framework based on ad-hoc microphone arrays, to address the problem. It contains three novel components. First, it combines \textit{ad-hoc microphone arrays} with deep-learning-based multichannel speech enhancement, which reduces the probability of the occurrence of far-field acoustic environments significantly. Second, it groups the microphones around the speech source to a local microphone array by a supervised channel selection framework based on deep neural networks. Third, it develops a simple time synchronization framework to synchronize the channels that have different time delay. Besides the above novelties and advantages, the proposed model is also trained in a single-channel fashion, so that it can easily employ new development of speech processing techniques. Its test stage is also flexible in incorporating any number of microphones without retraining or modifying the framework. We have developed many implementations of the proposed framework and conducted an extensive experiment in scenarios where the locations of the speech sources are far-field, random, and blind to the microphones. Results on speech enhancement tasks show that our method outperforms its counterpart that works with linear microphone arrays by a considerable margin in both diffuse noise reverberant environments and point source noise reverberant environments. We have also tested the framework with different handcrafted features. Results show that although designing good features lead to high performance, they do not affect the conclusion on the effectiveness of the proposed framework.
Malware Epidemics Effects in a Lanchester Conflict Model
Venue: arXiv cs.CR 2021
Year: 2021

We propose a framework for examining the effects of infections with self-replicating malware on military forces engaged in kinetic combat. The framework uses models, in which kinetic attrition is represented by a Lanchester model coupled with an SIR-like model describing the malware propagation across the forces. Basic knowledge about the expected circumstances restricts the set of scenarios to be analyzed using the model. Remaining uncertainties are taken into account as random variations given by information-theoretic principles. The situation assessment is realized by Monte-Carlo simulations with the risk as a possible assessment measure. An application of the proposed framework to a simple exemplary situation demonstrates its usage in practice. The assumed uncertainties about the considered situation lead to an outcome statistics, which changes corresponding to the improving knowledge about the situation. Large uncertainties may lead to results profoundly different from point estimates. For assuring practicability, the paper provides options to determine the values of important model parameters by measurement. It also discusses how to utilize the assessment results calculated with help of the framework.
Hardware Distortion Correlation Has Negligible Impact on UL Massive MIMO Spectral Efficiency
Venue: arXiv cs.IT 2021
Year: 2021

This paper analyzes how the distortion created by hardware impairments in a multiple-antenna base station affects the uplink spectral efficiency (SE), with focus on Massive MIMO. This distortion is correlated across the antennas, but has been often approximated as uncorrelated to facilitate (tractable) SE analysis. To determine when this approximation is accurate, basic properties of distortion correlation are first uncovered. Then, we separately analyze the distortion correlation caused by third-order non-linearities and by quantization. Finally, we study the SE numerically and show that the distortion correlation can be safely neglected in Massive MIMO when there are sufficiently many users. Under i.i.d. Rayleigh fading and equal signal-to-noise ratios (SNRs), this occurs for more than five transmitting users. Other channel models and SNR variations have only minor impact on the accuracy. We also demonstrate the importance of taking the distortion characteristics into account in the receive combining.
Estimation of Relationship between Stimulation Current and Force Exerted during Isometric Contraction
Venue: arXiv cs.HC 2021
Year: 2021

In this study, we developed a method to estimate the relationship between stimulation current and volatility during isometric contraction. In functional electrical stimulation (FES), joints are driven by applying voltage to muscles. This technology has been used for a long time in the field of rehabilitation, and recently application oriented research has been reported. However, estimation of the relationship between stimulus value and exercise capacity has not been discussed to a great extent. Therefore, in this study, a human muscle model was estimated using the transfer function estimation method with fast Fourier transform. It was found that the relationship between stimulation current and force exerted could be expressed by a first-order lag system. In verification of the force estimate, the ability of the proposed model to estimate the exerted force under steady state response was found to be good.
Improvements on SCORE, Especially for Weak Signals
Venue: arXiv cs.SI 2021
Year: 2021

A network may have weak signals and severe degree heterogeneity, and may be very sparse in one occurrence but very dense in another. SCORE (Jin, 2015) is a recent approach to network community detection. It accommodates severe degree heterogeneity and is adaptive to different levels of sparsity, but its performance for networks with weak signals is unclear. In this paper, we show that in a broad class of network settings where we allow for weak signals, severe degree heterogeneity, and a wide range of network sparsity, SCORE achieves prefect clustering and has the so-called "exponential rate" in Hamming clustering errors. The proof uses the most recent advancement on entry-wise bounds for the leading eigenvectors of the network adjacency matrix. The theoretical analysis assures us that SCORE continues to work well in the weak signal settings, but it does not rule out the possibility that SCORE may be further improved to have better performance in real applications, especially for networks with weak signals. As a second contribution of the paper, we propose SCORE+ as an improved version of SCORE. We investigate SCORE+ with 8 network data sets and found that it outperforms several representative approaches. In particular, for the 6 data sets with relatively strong signals, SCORE+ has similar performance as that of SCORE, but for the 2 data sets (Simmons, Caltech) with possibly weak signals, SCORE+ has much lower error rates. SCORE+ proposes several changes to SCORE. We carefully explain the rationale underlying each of these changes, using a mixture of theoretical and numerical study.
Pacing Equilibrium in First-Price Auction Markets
Venue: arXiv cs.GT 2021
Year: 2021

Mature internet advertising platforms offer high-level campaign management tools to help advertisers run their campaigns, often abstracting away the intricacies of how each ad is placed and focusing on aggregate metrics of interest to advertisers. On such platforms, advertisers often participate in auctions through a proxy bidder, so the standard incentive analyses that are common in the literature do not apply directly. In this paper, we take the perspective of a budget management system that surfaces aggregated incentives -- instead of individual auctions -- and compare first and second price auctions. We show that theory offers surprising endorsement for using a first price auction to sell individual impressions. In particular, first price auctions guarantee uniqueness of the steady-state equilibrium of the budget management system, monotonicity, and other desirable properties, as well as efficient computation through the solution to the well-studied Eisenberg-Gale convex program. Contrary to what one can expect from first price auctions, we show that incentives issues are not a barrier that undermines the system. Using realistic instances generated from data collected at real-world auction platforms, we show that bidders have small regret with respect to their optimal ex-post strategy, and they do not have a big incentive to misreport when they can influence equilibria directly by giving inputs strategically. Finally, budget-constrained bidders, who have significant prevalence in real-world platforms, tend to have smaller regrets. Our computations indicate that bidder budgets, pacing multipliers and regrets all have a positive association in statistical terms.
Algorithmic complexity in computational biology: basics, challenges and limitations
Venue: arXiv cs.CC 2021
Year: 2021

Computational problems can be classified according to their algorithmic complexity, which is defined based on how the resources needed to solve the problem, e.g. the execution time, scale with the problem size. Many problems in computational biology are computationally infeasible in the sense that the exhaustive search for the optimal solution is prohibitive in practical terms. As a consequence, these problems are tackled through heuristics and approximations aiming to overcome the exceeding computational requirements at the cost of providing suboptimal solutions. The importance of defining the computational complexity of computational biology algorithms is a topic rarely surveyed for broad audiences of bioinformaticians and users of bioinformatics tools. However, recognizing the underlying complexity of any algorithm is essential for understanding their potential and limitations. Thus, the aim of this review is to survey the main algorithmic solutions to intractable problems in computational biology, highlighting the importance of High-Performance Computing in this area.
Deep Shape-from-Template: Wide-Baseline, Dense and Fast Registration and Deformable Reconstruction from a Single Image
Venue: arXiv cs.CV 2021
Year: 2021

We present Deep Shape-from-Template (DeepSfT), a novel Deep Neural Network (DNN) method for solving real-time automatic registration and 3D reconstruction of a deformable object viewed in a single monocular image.DeepSfT advances the state-of-the-art in various aspects. Compared to existing DNN SfT methods, it is the first fully convolutional real-time approach that handles an arbitrary object geometry, topology and surface representation. It also does not require ground truth registration with real data and scales well to very complex object models with large numbers of elements. Compared to previous non-DNN SfT methods, it does not involve numerical optimization at run-time, and is a dense, wide-baseline solution that does not demand, and does not suffer from, feature-based matching. It is able to process a single image with significant deformation and viewpoint changes, and handles well the core challenges of occlusions, weak texture and blur. DeepSfT is based on residual encoder-decoder structures and refining blocks. It is trained end-to-end with a novel combination of supervised learning from simulated renderings of the object model and semi-supervised automatic fine-tuning using real data captured with a standard RGB-D camera. The cameras used for fine-tuning and run-time can be different, making DeepSfT practical for real-world use. We show that DeepSfT significantly outperforms state-of-the-art wide-baseline approaches for non-trivial templates, with quantitative and qualitative evaluation.
Driver Behavior Recognition via Interwoven Deep Convolutional Neural Nets with Multi-stream Inputs
Venue: arXiv cs.CV 2021
Year: 2021

Understanding driver activity is vital for in-vehicle systems that aim to reduce the incidence of car accidents rooted in cognitive distraction. Automating real-time behavior recognition while ensuring actions classification with high accuracy is however challenging, given the multitude of circumstances surrounding drivers, the unique traits of individuals, and the computational constraints imposed by in-vehicle embedded platforms. Prior work fails to jointly meet these runtime/accuracy requirements and mostly rely on a single sensing modality, which in turn can be a single point of failure. In this paper, we harness the exceptional feature extraction abilities of deep learning and propose a dedicated Interwoven Deep Convolutional Neural Network (InterCNN) architecture to tackle the problem of accurate classification of driver behaviors in real-time. The proposed solution exploits information from multi-stream inputs, i.e., in-vehicle cameras with different fields of view and optical flows computed based on recorded images, and merges through multiple fusion layers abstract features that it extracts. This builds a tight ensembling system, which significantly improves the robustness of the model. In addition, we introduce a temporal voting scheme based on historical inference instances, to enhance the classification accuracy. Experiments conducted with a dataset that we collect in a mock-up car environment demonstrate that the proposed InterCNN with MobileNet convolutional blocks can classify 9 different behaviors with 73.97% accuracy, and 5 'aggregated' behaviors with 81.66% accuracy. We further show that our architecture is highly computationally efficient, as it performs inferences within 15ms, which satisfies the real-time constraints of intelligent cars. Nevertheless, our InterCNN is robust to lossy input, as the classification remains accurate when two input streams are occluded.
Backdoor Decomposable Monotone Circuits and their Propagation Complete Encodings
Venue: arXiv cs.AI 2021
Year: 2021

We describe a compilation language of backdoor decomposable monotone circuits (BDMCs) which generalizes several concepts appearing in the literature, e.g. DNNFs and backdoor trees. A $\mathcal{C}$-BDMC sentence is a monotone circuit which satisfies decomposability property (such as in DNNF) in which the inputs (or leaves) are associated with CNF encodings from a given base class $\mathcal{C}$. We consider the class of propagation complete (PC) encodings as a base class and we show that PC-BDMCs are polynomially equivalent to PC encodings. Additionally, we use this to determine the properties of PC-BDMCs and PC encodings with respect to the knowledge compilation map including the list of efficient operations on the languages.
Efficient constructions of convex combinations for 2-edge-connected subgraphs on fundamental classes
Venue: arXiv cs.DM 2021
Year: 2021

Finding the exact integrality gap $α$ for the LP relaxation of the 2-edge-connected spanning multigraph problem (2EC) is closely related to the same problem for the Held-Karp relaxation of the metric traveling salesman problem (TSP). While the former problem seems easier than the latter, since it is less constrained, currently the upper bounds on the respective integrality gaps for the two problems are the same. An approach to proving integrality gaps for both of these problems is to consider fundamental classes of extreme points. For 2EC, better bounds on the integrality gap are known for certain important special cases of these fundamental points. For example, for half-integer square points, the integrality gap is between $\frac{6}{5}$ and $\frac{4}{3}$. Our main result is to improve the approximation factor to $\frac{9}{7}$ for 2EC for these points. Our approach is based on constructing convex combinations and our key tool is the top-down coloring framework for tree augmentation, whose flexibility we employ to exploit beneficial properties in both the initial spanning tree and in the input graph. We also show how these tools can be tailored to the closely related problem of uniform covers for which the proofs of the best-known bounds do not yield polynomial-time algorithms. Another key ingredient is to use a rainbow spanning tree decomposition, which allows us to obtain a convex combination of spanning trees with particular properties
Finding a Bounded-Degree Expander Inside a Dense One
Venue: arXiv cs.DC 2021
Year: 2021

It follows from the Marcus-Spielman-Srivastava proof of the Kadison-Singer conjecture that if $G=(V,E)$ is a $Δ$-regular dense expander then there is an edge-induced subgraph $H=(V,E_H)$ of $G$ of constant maximum degree which is also an expander. As with other consequences of the MSS theorem, it is not clear how one would explicitly construct such a subgraph. We show that such a subgraph (although with quantitatively weaker expansion and near-regularity properties than those predicted by MSS) can be constructed with high probability in linear time, via a simple algorithm. Our algorithm allows a distributed implementation that runs in $\mathcal O(\log n)$ rounds and does $\mathcal O(n)$ total work with high probability. The analysis of the algorithm is complicated by the complex dependencies that arise between edges and between choices made in different rounds. We sidestep these difficulties by following the combinatorial approach of counting the number of possible random choices of the algorithm which lead to failure. We do so by a compression argument showing that such random choices can be encoded with a non-trivial compression. Our algorithm bears some similarity to the way agents construct a communication graph in a peer-to-peer network, and, in the bipartite case, to the way agents select servers in blockchain protocols.
Fast UAV Trajectory Optimization using Bilevel Optimization with Analytical Gradients
Venue: arXiv cs.RO 2021
Year: 2021

We present an efficient optimization framework that solves trajectory optimization problems by decoupling state variables from timing variables, thereby decomposing a challenging nonlinear programming (NLP) problem into two easier subproblems. With timing fixed, the state variables can be optimized efficiently using convex optimization, and the timing variables can be optimized in a separate NLP, which forms a bilevel optimization problem. The challenge of obtaining the gradient of the timing variables is solved by sensitivity analysis of parametric NLPs. The exact analytic gradient is computed from the dual solution as a by-product, whereas existing finite-difference techniques require additional optimization. The bilevel optimization framework efficiently optimizes both timing and state variables which is demonstrated on generating trajectories for an unmanned aerial vehicle. Numerical experiments demonstrate that bilevel optimization converges significantly more reliably than a standard NLP solver, and analytical gradients outperform finite differences in terms of computation speed and accuracy. Physical experiments demonstrate its real-time applicability for reactive target tracking tasks.
Geometry-aware Manipulability Learning, Tracking and Transfer
Venue: arXiv cs.RO 2021
Year: 2021

Body posture influences human and robots performance in manipulation tasks, as appropriate poses facilitate motion or force exertion along different axes. In robotics, manipulability ellipsoids arise as a powerful descriptor to analyze, control and design the robot dexterity as a function of the articulatory joint configuration. This descriptor can be designed according to different task requirements, such as tracking a desired position or apply a specific force. In this context, this paper presents a novel \emph{manipulability transfer} framework, a method that allows robots to learn and reproduce manipulability ellipsoids from expert demonstrations. The proposed learning scheme is built on a tensor-based formulation of a Gaussian mixture model that takes into account that manipulability ellipsoids lie on the manifold of symmetric positive definite matrices. Learning is coupled with a geometry-aware tracking controller allowing robots to follow a desired profile of manipulability ellipsoids. Extensive evaluations in simulation with redundant manipulators, a robotic hand and humanoids agents, as well as an experiment with two real dual-arm systems validate the feasibility of the approach.