OpenRaiser/Intern-Atlas huggingface dataset:
Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion
Venue: arXiv cs.CV 2022
Year: 2022

Uncalibrated photometric stereo is proposed to estimate the detailed surface normal from images under varying and unknown lightings. Recently, deep learning brings powerful data priors to this underdetermined problem. This paper presents a new method for deep uncalibrated photometric stereo, which efficiently utilizes the inter-image representation to guide the normal estimation. Previous methods use optimization-based neural inverse rendering or a single size-independent pooling layer to deal with multiple inputs, which are inefficient for utilizing information among input images. Given multi-images under different lighting, we consider the intra-image and inter-image variations highly correlated. Motivated by the correlated variations, we designed an inter-intra image feature fusion module to introduce the inter-image representation into the per-image feature extraction. The extra representation is used to guide the per-image feature extraction and eliminate the ambiguity in normal estimation. We demonstrate the effect of our design on a wide range of samples, especially on dark materials. Our method produces significantly better results than the state-of-the-art methods on both synthetic and real data.
Detecting Algorithmically Generated Domains Using a GCNN-LSTM Hybrid Neural Network
Venue: arXiv cs.CR 2022
Year: 2022

Domain generation algorithm (DGA) is used by botnets to build a stealthy command and control (C&C) communication channel between the C&C server and the bots. A DGA can periodically produce a large number of pseudo-random algorithmically generated domains (AGDs). AGD detection algorithms provide a lightweight, promising solution in response to the existing DGA techniques. In this paper, a GCNN (gated convolutional neural network)-LSTM (long short-term memory) Hybrid Neural Network (GLHNN) for AGD detection is proposed. In GLHNN, GCNN is applied to extract the informative features from domain names on top of LSTM which further processes the feature sequence. GLHNN is experimentally validated using representative AGDs covering six classes of DGAs. GLHNN is compared with the state-of-the-art detection models and demonstrates the best overall detection performance among these tested models.
Data science in public health: building next generation capacity
Venue: arXiv cs.CY 2022
Year: 2022

Rapidly evolving technology, data and analytic landscapes are permeating many fields and professions. In public health, the need for data science skills including data literacy is particularly prominent given both the potential of novel data types and analysis methods to fill gaps in existing public health research and intervention practices, as well as the potential of such data or methods to perpetuate or augment health disparities. Through a review of public health courses and programs at the top 10 U.S. and globally ranked schools of public health, this article summarizes existing educational efforts in public health data science. These existing practices serve to inform efforts for broadening such curricula to further schools and populations. Data science ethics course offerings are also examined in context of assessing how population health principles can be blended into training across levels of data involvement to augment the traditional core of public health curricula. Parallel findings from domestic and international 'outside the classroom' training programs are also synthesized to advance approaches for increasing diversity in public health data science. Based on these program reviews and their synthesis, a four-point formula is distilled for furthering public health data science education efforts, toward development of a critical and inclusive mass of practitioners with fluency to leverage data to advance goals of public health and improve quality of life in the digital age.
A Set-Theoretic Decision Procedure for Quantifier-Free, Decidable Languages Extended with Restricted Quantifiers
Venue: arXiv cs.LO 2022
Year: 2022

Let $\mathcal{L}_{\mathcal{X}}$ be the language of first-order, decidable theory $\mathcal{X}$. Consider the language, $\mathcal{L}_{\mathcal{RQ}}(\mathcal{X})$, that extends $\mathcal{L}_{\mathcal{X}}$ with formulas of the form $\forall x \in A: ϕ$ (restricted universal quantifier, RUQ) and $\exists x \in A: ϕ$ (restricted existential quantifier, REQ), where $A$ is a finite set and $ϕ$ is a formula made of $\mathcal{X}$-formulas, RUQ and REQ. That is, $\mathcal{L}_{\mathcal{RQ}}(\mathcal{X})$ admits nested restricted quantifiers. In this paper we present a decision procedure for $\mathcal{L}_{\mathcal{RQ}}(\mathcal{X})$ based on the decision procedure already defined for the Boolean algebra of finite sets extended with restricted intensional sets ($\mathcal{L}_\mathcal{RIS}$). The implementation of the decision procedure as part of the $\{log\}$ (`setlog') tool is also introduced. The usefulness of the approach is shown through a number of examples drawn from several real-world case studies.
A Successful Integration of the Robotic Technology Kernel (RTK) for a By-Wire Electric Vehicle System with a Mobile App Interface
Venue: arXiv cs.RO 2022
Year: 2022

We were able to complete the full integration of the Robotic Technology Kernel (RTK) into an electric vehicle by-wire system using lidar and GPS sensors. The solution included a mobile application to interface with the RTK-enabled autonomous vehicle. Altogether the system was designed to be modular, using the concepts of message-based software design that is built into the Robot Operating System (ROS), which is at the foundation of RTK. The team worked incrementally to develop working software to demonstrate each milestone on the path to successfully completing the RTK integration for the development of an application called the Vehicle Summoning System (VSS).
MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer
Venue: arXiv cs.CV 2022
Year: 2022

Self-supervised monocular depth estimation is an attractive solution that does not require hard-to-source depth labels for training. Convolutional neural networks (CNNs) have recently achieved great success in this task. However, their limited receptive field constrains existing network architectures to reason only locally, dampening the effectiveness of the self-supervised paradigm. In the light of the recent successes achieved by Vision Transformers (ViTs), we propose MonoViT, a brand-new framework combining the global reasoning enabled by ViT models with the flexibility of self-supervised monocular depth estimation. By combining plain convolutions with Transformer blocks, our model can reason locally and globally, yielding depth prediction at a higher level of detail and accuracy, allowing MonoViT to achieve state-of-the-art performance on the established KITTI dataset. Moreover, MonoViT proves its superior generalization capacities on other datasets such as Make3D and DrivingStereo.
Development of a mobile robot assistant for wind turbines manufacturing
Venue: arXiv cs.RO 2022
Year: 2022

The thrust for increased rating capacity of wind turbines has resulted into larger generators, longer blades, and taller towers. Presently, up to 16 MW wind turbines are being offered by wind turbines manufacturers which is nearly a 60 percent increase in the design capacity over the last five years. Manufacturing of these turbines involves assembling of gigantic sized components. Due to the frequent design changes and the variety of tasks involved, conventional automation is not possible making it a labor-intensive activity. However the handling and assembling of large components are challenging the human capabilities. The article proposes the use of mobile robotic assistants for partial automation of wind turbines manufacturing. The robotic assistant can result into reducing production costs, and better work conditions. The article presents development of a robot assistant for human operators to effectively perform assembly of wind turbines. The case is from a leading wind turbines manufacturer. The developed system is also applicable to other cases of large component manufacturing involving intensive manual effort.
An Unsupervised Learning Approach for Spectrum Allocation in Terahertz Communication Systems
Venue: arXiv cs.LG 2022
Year: 2022

We propose a new spectrum allocation strategy, aided by unsupervised learning, for multiuser terahertz communication systems. In this strategy, adaptive sub-band bandwidth is considered such that the spectrum of interest can be divided into sub-bands with unequal bandwidths. This strategy reduces the variation in molecular absorption loss among the users, leading to the improved data rate performance. We first formulate an optimization problem to determine the optimal sub-band bandwidth and transmit power, and then propose the unsupervised learning-based approach to obtaining the near-optimal solution to this problem. In the proposed approach, we first train a deep neural network (DNN) while utilizing a loss function that is inspired by the Lagrangian of the formulated problem. Then using the trained DNN, we approximate the near-optimal solutions. Numerical results demonstrate that comparing to existing approaches, our proposed unsupervised learning-based approach achieves a higher data rate, especially when the molecular absorption coefficient within the spectrum of interest varies in a highly non-linear manner.
Multi-Stage NMPC for a MAV based Collision Free Navigation under Varying Communication Delays
Venue: arXiv cs.RO 2022
Year: 2022

Time delays in communication networks are one of the main concerns in deploying robots with computation boards on the edge. This article proposes a multi-stage Nonlinear Model Predictive Control (NMPC) that is capable of handling varying network-induced time delays for establishing a control framework being able to guarantee collision-free Micro Aerial Vehicles (MAVs) navigation. This study introduces a novel approach that considers different sampling times by a tree of discretization scenarios contrary to the existing typical multi-stage NMPC where system uncertainties are modeled by a tree of scenarios. Additionally, the proposed method considers adaptive weights for the multi-stage NMPC scenarios based on the probability of time delays in the communication link. As a result of the multi-stage NMPC, the obtained optimal control action is valid for multiple sampling times. Finally, the overall effectiveness of the proposed novel control framework is demonstrated in various tests and different simulation environments.
Jointformer: Single-Frame Lifting Transformer with Error Prediction and Refinement for 3D Human Pose Estimation
Venue: arXiv cs.CV 2022
Year: 2022

Monocular 3D human pose estimation technologies have the potential to greatly increase the availability of human movement data. The best-performing models for single-image 2D-3D lifting use graph convolutional networks (GCNs) that typically require some manual input to define the relationships between different body joints. We propose a novel transformer-based approach that uses the more generalised self-attention mechanism to learn these relationships within a sequence of tokens representing joints. We find that the use of intermediate supervision, as well as residual connections between the stacked encoders benefits performance. We also suggest that using error prediction as part of a multi-task learning framework improves performance by allowing the network to compensate for its confidence level. We perform extensive ablation studies to show that each of our contributions increases performance. Furthermore, we show that our approach outperforms the recent state of the art for single-frame 3D human pose estimation by a large margin. Our code and trained models are made publicly available on Github.
Adaptive incomplete multi-view learning via tensor graph completion
Venue: arXiv cs.LG 2022
Year: 2022

With the advancement of the data acquisition techniques, multi-view learning has become a hot topic. Some multi-view learning methods assume that the multi-view data is complete, which means that all instances are present, but this too ideal. Certain tensor-based methods for handing incomplete multi-view data have emerged and have achieved better result. However, there are still some problems, such as use of traditional tensor norm which makes the computation high and is not able to handle out-of-sample. To solve these two problems, we proposed a new incomplete multi view learning method. A new tensor norm is defined to implement graph tensor data recover. The recovered graphs are then regularized to a consistent low-dimensional representation of the samples. In addition, adaptive weights are equipped to each view to adjust the importance of different views. Compared with the existing methods, our method nor only explores the consistency among views, but also obtains the low-dimensional representation of the new samples by using the learned projection matrix. An efficient algorithm based on inexact augmented Lagrange multiplier (ALM) method are designed to solve the model and convergence is proved. Experimental results on four datasets show the effectiveness of our method.
Label Semantic Knowledge Distillation for Unbiased Scene Graph Generation
Venue: arXiv cs.CV 2022
Year: 2022

The Scene Graph Generation (SGG) task aims to detect all the objects and their pairwise visual relationships in a given image. Although SGG has achieved remarkable progress over the last few years, almost all existing SGG models follow the same training paradigm: they treat both object and predicate classification in SGG as a single-label classification problem, and the ground-truths are one-hot target labels. However, this prevalent training paradigm has overlooked two characteristics of current SGG datasets: 1) For positive samples, some specific subject-object instances may have multiple reasonable predicates. 2) For negative samples, there are numerous missing annotations. Regardless of the two characteristics, SGG models are easy to be confused and make wrong predictions. To this end, we propose a novel model-agnostic Label Semantic Knowledge Distillation (LS-KD) for unbiased SGG. Specifically, LS-KD dynamically generates a soft label for each subject-object instance by fusing a predicted Label Semantic Distribution (LSD) with its original one-hot target label. LSD reflects the correlations between this instance and multiple predicate categories. Meanwhile, we propose two different strategies to predict LSD: iterative self-KD and synchronous self-KD. Extensive ablations and results on three SGG tasks have attested to the superiority and generality of our proposed LS-KD, which can consistently achieve decent trade-off performance between different predicate categories.
Codebook Based Two-Time Scale Resource Allocation Design for IRS-Assisted eMBB-URLLC Systems
Venue: arXiv cs.IT 2022
Year: 2022

This paper investigates the resource allocation algorithm design for wireless systems assisted by large intelligent reflecting surfaces (IRSs) with coexisting enhanced mobile broadband (eMBB) and ultra reliable low-latency communication (URLLC) users. We consider a two-time scale resource allocation scheme, whereby the base station's precoders are optimized in each mini-slot to adapt to newly arriving URLLC traffic, whereas the IRS phase shifts are reconfigured only in each time slot to avoid excessive base station-IRS signaling. To facilitate efficient resource allocation design for large IRSs, we employ a codebook-based optimization framework, where the IRS is divided into several tiles and the phase-shift elements of each tile are selected from a pre-defined codebook. The resource allocation algorithm design is formulated as an optimization problem for the maximization of the average sum data rate of the eMBB users over a time slot while guaranteeing the quality-of-service (QoS) of each URLLC user in each mini-slot. An iterative algorithm based on alternating optimization (AO) is proposed to find a high-quality suboptimal solution. As a case study, the proposed algorithm is applied in an industrial indoor environment modelled via the Quadriga channel simulator. Our simulation results show that the proposed algorithm design enables the coexistence of eMBB and URLLC users and yields large performance gains compared to three baseline schemes. Furthermore, our simulation results reveal that the proposed two-time scale resource allocation design incurs only a small performance loss compared to the case when the IRSs are optimized in each mini-slot.
Garbled EDA: Privacy Preserving Electronic Design Automation
Venue: arXiv cs.CR 2022
Year: 2022

The complexity of modern integrated circuits (ICs) necessitates collaboration between multiple distrusting parties, including thirdparty intellectual property (3PIP) vendors, design houses, CAD/EDA tool vendors, and foundries, which jeopardizes confidentiality and integrity of each party's IP. IP protection standards and the existing techniques proposed by researchers are ad hoc and vulnerable to numerous structural, functional, and/or side-channel attacks. Our framework, Garbled EDA, proposes an alternative direction through formulating the problem in a secure multi-party computation setting, where the privacy of IPs, CAD tools, and process design kits (PDKs) is maintained. As a proof-of-concept, Garbled EDA is evaluated in the context of simulation, where multiple IP description formats (Verilog, C, S) are supported. Our results demonstrate a reasonable logical-resource cost and negligible memory overhead. To further reduce the overhead, we present another efficient implementation methodology, feasible when the resource utilization is a bottleneck, but the communication between two parties is not restricted. Interestingly, this implementation is private and secure even in the presence of malicious adversaries attempting to, e.g., gain access to PDKs or in-house IPs of the CAD tool providers.
Creating Reverse Bilingual Dictionaries
Venue: arXiv cs.CL 2022
Year: 2022

Bilingual dictionaries are expensive resources and not many are available when one of the languages is resource-poor. In this paper, we propose algorithms for creation of new reverse bilingual dictionaries from existing bilingual dictionaries in which English is one of the two languages. Our algorithms exploit the similarity between word-concept pairs using the English Wordnet to produce reverse dictionary entries. Since our algorithms rely on available bilingual dictionaries, they are applicable to any bilingual dictionary as long as one of the two languages has Wordnet type lexical ontology.
Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graph
Venue: arXiv cs.CL 2022
Year: 2022

The generalizability to new databases is of vital importance to Text-to-SQL systems which aim to parse human utterances into SQL statements. Existing works achieve this goal by leveraging the exact matching method to identify the lexical matching between the question words and the schema items. However, these methods fail in other challenging scenarios, such as the synonym substitution in which the surface form differs between the corresponding question words and schema items. In this paper, we propose a framework named ISESL-SQL to iteratively build a semantic enhanced schema-linking graph between question tokens and database schemas. First, we extract a schema linking graph from PLMs through a probing procedure in an unsupervised manner. Then the schema linking graph is further optimized during the training process through a deep graph learning method. Meanwhile, we also design an auxiliary task called graph regularization to improve the schema information mentioned in the schema-linking graph. Extensive experiments on three benchmarks demonstrate that ISESL-SQL could consistently outperform the baselines and further investigations show its generalizability and robustness.
Bridging the Gap: Commonality and Differences between Online and Offline COVID-19 Data
Venue: arXiv cs.SI 2022
Year: 2022

With the onset of the COVID-19 pandemic, news outlets and social media have become central tools for disseminating and consuming information. Because of their ease of access, users seek COVID-19-related information from online social media (i.e., online news) and news outlets (i.e., offline news). Online and offline news are often connected, sharing common topics while each has unique, different topics. A gap between these two news sources can lead to misinformation propagation. For instance, according to the Guardian, most COVID-19 misinformation comes from users on social media. Without fact-checking social media news, misinformation can lead to health threats. In this paper, we focus on the novel problem of bridging the gap between online and offline data by monitoring their common and distinct topics generated over time. We employ Twitter (online) and local news (offline) data for a time span of two years. Using online matrix factorization, we analyze and study online and offline COVID-19-related data differences and commonalities. We design experiments to show how online and offline data are linked together and what trends they follow.
Depth Quality-Inspired Feature Manipulation for Efficient RGB-D and Video Salient Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Recently CNN-based RGB-D salient object detection (SOD) has obtained significant improvement on detection accuracy. However, existing models often fail to perform well in terms of efficiency and accuracy simultaneously. This hinders their potential applications on mobile devices as well as many real-world problems. To bridge the accuracy gap between lightweight and large models for RGB-D SOD, in this paper, an efficient module that can greatly improve the accuracy but adds little computation is proposed. Inspired by the fact that depth quality is a key factor influencing the accuracy, we propose an efficient depth quality-inspired feature manipulation (DQFM) process, which can dynamically filter depth features according to depth quality. The proposed DQFM resorts to the alignment of low-level RGB and depth features, as well as holistic attention of the depth stream to explicitly control and enhance cross-modal fusion. We embed DQFM to obtain an efficient lightweight RGB-D SOD model called DFM-Net, where we in addition design a tailored depth backbone and a two-stage decoder as basic parts. Extensive experimental results on nine RGB-D datasets demonstrate that our DFM-Net outperforms recent efficient models, running at about 20 FPS on CPU with only 8.5Mb model size, and meanwhile being 2.9/2.4 times faster and 6.7/3.1 times smaller than the latest best models A2dele and MobileSal. It also maintains state-of-the-art accuracy when even compared to non-efficient models. Interestingly, further statistics and analyses verify the ability of DQFM in distinguishing depth maps of various qualities without any quality labels. Last but not least, we further apply DFM-Net to deal with video SOD (VSOD), achieving comparable performance against recent efficient models while being 3/2.3 times faster/smaller than the prior best in this field. Our code is available at https://github.com/zwbx/DFM-Net.