OpenRaiser/Intern-Atlas huggingface dataset:
DAMix: A Density-Aware Mixup Augmentation for Single Image Dehazing under Domain Shift
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning-based methods have achieved considerable success on single image dehazing in recent years. However, these methods are often subject to performance degradation when domain shifts are confronted. Specifically, haze density gaps exist among the existing datasets, often resulting in poor performance when these methods are tested across datasets. To address this issue, we propose a density-aware mixup augmentation (DAMix). DAMix generates samples in an attempt to minimize the Wasserstein distance with the hazy images in the target domain. These DAMix-ed samples not only mitigate domain gaps but are also proven to comply with the atmospheric scattering model. Thus, DAMix achieves comprehensive improvements on domain adaptation. Furthermore, we show that DAMix is helpful with respect to data efficiency. Specifically, a network trained with half of the source dataset using DAMix can achieve even better adaptivity than that trained with the whole source dataset but without DAMix.
Extracting and Inferring Personal Attributes from Dialogue
Venue: arXiv cs.CL 2022
Year: 2022

Personal attributes represent structured information about a person, such as their hobbies, pets, family, likes and dislikes. We introduce the tasks of extracting and inferring personal attributes from human-human dialogue, and analyze the linguistic demands of these tasks. To meet these challenges, we introduce a simple and extensible model that combines an autoregressive language model utilizing constrained attribute generation with a discriminative reranker. Our model outperforms strong baselines on extracting personal attributes as well as inferring personal attributes that are not contained verbatim in utterances and instead requires commonsense reasoning and lexical inferences, which occur frequently in everyday conversation. Finally, we demonstrate the benefit of incorporating personal attributes in social chit-chat and task-oriented dialogue settings.
Anti-collision Technologies for Unmanned Aerial Vehicles: Recent Advances and Future Trends
Venue: arXiv cs.RO 2022
Year: 2022

Unmanned aerial vehicles (UAVs) are widely applied in civil applications, such as disaster relief, agriculture and cargo transportation, etc. With the massive number of UAV flight activities, the anti-collision technologies aiming to avoid the collisions between UAVs and other objects have attracted much attention. The anti-collision technologies are of vital importance to guarantee the survivability and safety of UAVs. In this article, a comprehensive survey on UAV anti-collision technologies is presented. We firstly introduce laws and regulations on UAV safety which prevent collision at the policy level. Then, the process of anti-collision technologies are reviewed from three aspects, i.e., obstacle sensing, collision prediction, and collision avoidance. We provide detailed survey and comparison of the methods of each aspect and analyze their pros and cons. Besides, the future trends on UAV anti-collision technologies are presented from the perspective of fast obstacle sensing and fast wireless networking. Finally, we summarize this article.
Nonlinear MPC for Quadrotor Fault-Tolerant Control
Venue: arXiv cs.RO 2022
Year: 2022

The mechanical simplicity, hover capabilities, and high agility of quadrotors lead to a fast adaption in the industry for inspection, exploration, and urban aerial mobility. On the other hand, the unstable and underactuated dynamics of quadrotors render them highly susceptible to system faults, especially rotor failures. In this work, we propose a fault-tolerant controller using nonlinear model predictive control (NMPC) to stabilize and control a quadrotor subjected to the complete failure of a single rotor. Differently from existing works, which either rely on linear assumptions or resort to cascaded structures neglecting input constraints in the outer-loop, our method leverages full nonlinear dynamics of the damaged quadrotor and considers the thrust constraint of each rotor. Hence, this method could effectively perform upset recovery from extreme initial conditions. Extensive simulations and real-world experiments are conducted for validation, which demonstrates that the proposed NMPC method can effectively recover the damaged quadrotor even if the failure occurs during aggressive maneuvers, such as flipping and tracking agile trajectories.
ML4ML: Automated Invariance Testing for Machine Learning Models
Venue: arXiv cs.LG 2022
Year: 2022

In machine learning (ML) workflows, determining the invariance qualities of an ML model is a common testing procedure. Traditionally, invariance qualities are evaluated using simple formula-based scores, e.g., accuracy. In this paper, we show that testing the invariance qualities of ML models may result in complex visual patterns that cannot be classified using simple formulas. In order to test ML models by analyzing such visual patterns automatically using other ML models, we propose a systematic framework that is applicable to a variety of invariance qualities. We demonstrate the effectiveness and feasibility of the framework by developing ML4ML models (assessors) for determining rotation-, brightness-, and size-variances of a collection of neural networks. Our testing results show that the trained ML4ML assessors can perform such analytical tasks with sufficient accuracy.
An empirical study of question discussions on Stack Overflow
Venue: arXiv cs.SE 2022
Year: 2022

Stack Overflow provides a means for developers to exchange knowledge. While much previous research on Stack Overflow has focused on questions and answers (Q&A), recent work has shown that discussions in comments also contain rich information. On Stack Overflow, discussions through comments and chat rooms can be tied to questions or answers. In this paper, we conduct an empirical study that focuses on the nature of question discussions. We observe that: (1) Question discussions occur at all phases of the Q&A process, with most beginning before the first answer is received. (2) Both askers and answerers actively participate in question discussions; the likelihood of their participation increases as the number of comments increases. (3) There is a strong correlation between the number of question comments and the question answering time (i.e., more discussed questions receive answers more slowly); also, questions with a small number of comments are likely to be answered more quickly than questions with no discussion. Our findings suggest that question discussions contain a rich trove of data that is integral to the Q&A processes on Stack Overflow. We further suggest how future research can leverage the information in question discussions, along with the commonly studied Q&A information.
Lyapunov-Net: A Deep Neural Network Architecture for Lyapunov Function Approximation
Venue: arXiv cs.LG 2022
Year: 2022

We develop a versatile deep neural network architecture, called Lyapunov-Net, to approximate Lyapunov functions of dynamical systems in high dimensions. Lyapunov-Net guarantees positive definiteness, and thus it can be easily trained to satisfy the negative orbital derivative condition, which only renders a single term in the empirical risk function in practice. This significantly reduces the number of hyper-parameters compared to existing methods. We also provide theoretical justifications on the approximation power of Lyapunov-Net and its complexity bounds. We demonstrate the efficiency of the proposed method on nonlinear dynamical systems involving up to 30-dimensional state spaces, and show that the proposed approach significantly outperforms the state-of-the-art methods.
Random Dilated Shapelet Transform: A New Approach for Time Series Shapelets
Venue: arXiv cs.CV 2022
Year: 2022

Shapelet-based algorithms are widely used for time series classification because of their ease of interpretation, but they are currently outperformed by recent state-of-the-art approaches. We present a new formulation of time series shapelets including the notion of dilation, and we introduce a new shapelet feature to enhance their discriminative power for classification. Experiments performed on 112 datasets show that our method improves on the state-of-the-art shapelet algorithm, and achieves comparable accuracy to recent state-of-the-art approaches, without sacrificing neither scalability, nor interpretability.
Template-free Prompt Tuning for Few-shot NER
Venue: arXiv cs.CL 2022
Year: 2022

Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words. However, when applied to token-level labeling tasks such as NER, it would be time-consuming to enumerate the template queries over all potential entity spans. In this work, we propose a more elegant method to reformulate NER tasks as LM problems without any templates. Specifically, we discard the template construction process while maintaining the word prediction paradigm of pre-training models to predict a class-related pivot word (or label word) at the entity position. Meanwhile, we also explore principled ways to automatically search for appropriate label words that the pre-trained models can easily adapt to. While avoiding complicated template-based process, the proposed LM objective also reduces the gap between different objectives used in pre-training and fine-tuning, thus it can better benefit the few-shot performance. Experimental results demonstrate the effectiveness of the proposed method over bert-tagger and template-based method under few-shot setting. Moreover, the decoding speed of the proposed method is up to 1930.12 times faster than the template-based method.
Convergence of Deep Convolutional Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Convergence of deep neural networks as the depth of the networks tends to infinity is fundamental in building the mathematical foundation for deep learning. In a previous study, we investigated this question for deep ReLU networks with a fixed width. This does not cover the important convolutional neural networks where the widths are increasing from layer to layer. For this reason, we first study convergence of general ReLU networks with increasing widths and then apply the results obtained to deep convolutional neural networks. It turns out the convergence reduces to convergence of infinite products of matrices with increasing sizes, which has not been considered in the literature. We establish sufficient conditions for convergence of such infinite products of matrices. Based on the conditions, we present sufficient conditions for piecewise convergence of general deep ReLU networks with increasing widths, and as well as pointwise convergence of deep ReLU convolutional neural networks.
Intelligent Reflecting Surface Aided Wireless Networks: From Single-Reflection to Multi-Reflection Design and Optimization
Venue: arXiv cs.IT 2022
Year: 2022

Intelligent reflecting surface (IRS) has emerged as a promising technique for wireless communication networks. By dynamically tuning the reflection amplitudes/phase shifts of a large number of passive elements, IRS enables flexible wireless channel control and configuration, and thereby enhances the wireless signal transmission rate and reliability significantly. Despite the vast literature on designing and optimizing assorted IRS-aided wireless systems, prior works have mainly focused on enhancing wireless links with single signal reflection only by one or multiple IRSs, which may be insufficient to boost the wireless link capacity under some harsh propagation conditions (e.g., indoor environment with dense blockages/obstructions). This issue can be tackled by employing two or more IRSs to assist each wireless link and jointly exploiting their single as well as multiple signal reflections over them. However, the resultant double-/multi-IRS aided wireless systems face more complex design issues as well as new practical challenges for implementation as compared to the conventional single-IRS counterpart, in terms of IRS reflection optimization, channel acquisition, as well as IRS deployment and association/selection. As such, a new paradigm for designing multi-IRS cooperative passive beamforming and joint active/passive beam routing arises which calls for innovative design approaches and optimization methods. In this paper, we give a tutorial overview of multi-IRS aided wireless networks, with an emphasis on addressing the new challenges due to multi-IRS signal reflection and routing. Moreover, we point out important directions worthy of research and investigation in the future.
A deep dive into the accuracy of IP Geolocation Databases and its impact on online advertising
Venue: arXiv cs.CY 2022
Year: 2022

The quest for every time more personalized Internet experience relies on the enriched contextual information about each user. Online advertising also follows this approach. Among the context information that advertising stakeholders leverage, location information is certainly one of them. However, when this information is not directly available from the end users, advertising stakeholders infer it using geolocation databases, matching IP addresses to a position on earth. The accuracy of this approach has often been questioned in the past: however, the reality check on an advertising DSP shows that this technique accounts for a large fraction of the served advertisements. In this paper, we revisit the work in the field, that is mostly from almost one decade ago, through the lenses of big data. More specifically, we, i) benchmark two commercial Internet geolocation databases, evaluate the quality of their information using a ground truth database of user positions containing more than 2 billion samples, ii) analyze the internals of these databases, devising a theoretical upper bound for the quality of the Internet geolocation approach, and iii) we run an empirical study that unveils the monetary impact of this technology by considering the costs associated with a real-world ad impressions dataset. We show that when factoring cost in, IP geolocation technology may be, under certain campaign characteristics, a better alternative than GPS from an economic point of view, despite its inferior performance.
IRMAC: Interpretable Refined Motifs in Binary Classification for Smart Grid Applications
Venue: arXiv cs.LG 2022
Year: 2022

Modern power systems are experiencing the challenge of high uncertainty with the increasing penetration of renewable energy resources and the electrification of heating systems. In this paradigm shift, understanding electricity users' demand is of utmost value to retailers, aggregators, and policymakers. However, behind-the-meter (BTM) equipment and appliances at the household level are unknown to the other stakeholders mainly due to privacy concerns and tight regulations. In this paper, we seek to identify residential consumers based on their BTM equipment, mainly rooftop photovoltaic (PV) systems and electric heating, using imported/purchased energy data from utility meters. To solve this problem with an interpretable, fast, secure, and maintainable solution, we propose an integrated method called Interpretable Refined Motifs And binary Classification (IRMAC). The proposed method comprises a novel shape-based pattern extraction technique, called Refined Motif (RM) discovery, and a single-neuron classifier. The first part extracts a sub-pattern from the long time series considering the frequency of occurrences, average dissimilarity, and time dynamics while emphasising specific times with annotated distances. The second part identifies users' types with linear complexity while preserving the transparency of the algorithms. With the real data from Australia and Denmark, the proposed method is tested and verified in identifying PV owners and electrical heating system users.
Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation
Venue: arXiv cs.RO 2022
Year: 2022

We tackle real-world long-horizon robot manipulation tasks through skill discovery. We present a bottom-up approach to learning a library of reusable skills from unsegmented demonstrations and use these skills to synthesize prolonged robot behaviors. Our method starts with constructing a hierarchical task structure from each demonstration through agglomerative clustering. From the task structures of multi-task demonstrations, we identify skills based on the recurring patterns and train goal-conditioned sensorimotor policies with hierarchical imitation learning. Finally, we train a meta controller to compose these skills to solve long-horizon manipulation tasks. The entire model can be trained on a small set of human demonstrations collected within 30 minutes without further annotations, making it amendable to real-world deployment. We systematically evaluated our method in simulation environments and on a real robot. Our method has shown superior performance over state-of-the-art imitation learning methods in multi-stage manipulation tasks. Furthermore, skills discovered from multi-task demonstrations boost the average task success by $8\%$ compared to those discovered from individual tasks.
RAFT: A Real-World Few-Shot Text Classification Benchmark
Venue: arXiv cs.CL 2022
Year: 2022

Large pre-trained language models have shown promise for few-shot learning, completing text-based tasks given only a few task-specific examples. Will models soon solve classification tasks that have so far been reserved for human research assistants? Existing benchmarks are not designed to measure progress in applied settings, and so don't directly answer this question. The RAFT benchmark (Real-world Annotated Few-shot Tasks) focuses on naturally occurring tasks and uses an evaluation setup that mirrors deployment. Baseline evaluations on RAFT reveal areas current techniques struggle with: reasoning over long texts and tasks with many classes. Human baselines show that some classification tasks are difficult for non-expert humans, reflecting that real-world value sometimes depends on domain expertise. Yet even non-expert human baseline F1 scores exceed GPT-3 by an average of 0.11. The RAFT datasets and leaderboard will track which model improvements translate into real-world benefits at https://raft.elicit.org .
A simulation sandbox to compare fixed-route, semi-flexible-transit, and on-demand microtransit system designs
Venue: arXiv cs.CY 2022
Year: 2022

With advances in emerging technologies, options for operating public transit services have broadened from conventional fixed-route service through semi-flexible service to on-demand microtransit. Nevertheless, guidelines for deciding between these services remain limited in the real implementation. An open-source simulation sandbox is developed that can compare state-of-the-practice methods for evaluating between the different types of public transit operations. For the case of the semi-flexible service, the Mobility Allowance Shuttle Transit (MAST) system is extended to include passenger deviations. A case study demonstrates the sandbox to evaluate and existing B63 bus route in Brooklyn, NY and compares its performance with the four other system designs spanning across the three service types for three different demand scenarios.
Quadratic-Curve-Lifted Reed-Solomon Codes
Venue: arXiv cs.IT 2022
Year: 2022

Lifted codes are a class of evaluation codes attracting more attention due to good locality and intermediate availability. In this work we introduce and study quadratic-curve-lifted Reed-Solomon (QC-LRS) codes, where the codeword symbols whose coordinates are on a quadratic curve form a codeword of a Reed-Solomon code. We first develop a necessary and sufficient condition on the monomials which form a basis the code. Based on the condition, we give upper and lower bounds on the dimension and show that the asymptotic rate of a QC-LRS code over $\mathbb{F}_q$ with local redundancy $r$ is $1-Θ(q/r)^{-0.2284}$. Moreover, we provide analytical results on the minimum distance of this class of codes and compare QC-LRS codes with lifted Reed-Solomon codes by simulations in terms of the local recovery capability against erasures. For short lengths, QC-LRS codes have better performance in local recovery for erasures than LRS codes of the same dimension.
Automatic Discovery and Description of Human Planning Strategies
Venue: arXiv cs.AI 2022
Year: 2022

Scientific discovery concerns finding patterns in data and creating insightful hypotheses that explain these patterns. Traditionally, this process required human ingenuity, but with the galloping advances in artificial intelligence (AI) it becomes feasible to automate some parts of scientific discovery. In this work we leverage AI for strategy discovery for understanding human planning. In the state-of-the-art methods data about the process of human planning is often used to group similar behaviors together and formulate verbal descriptions of the strategies which might underlie those groups. Here, we automate these two steps. Our method utilizes a new algorithm, called Human-Interpret, that performs imitation learning to describe sequences of planning operations in terms of a procedural formula and then translates that formula to natural language. We test our method on a benchmark data set that researchers have previously scrutinized manually. We find that the descriptions of human planning strategies obtained automatically are about as understandable as human-generated descriptions. They also cover a substantial proportion of of relevant types of human planning strategies that had been discovered manually. Our method saves scientists' time and effort as all the reasoning about human planning is done automatically. This might make it feasible to more rapidly scale up the search for yet undiscovered cognitive strategies to many new decision environments, populations, tasks, and domains. Given these results, we believe that the presented work may accelerate scientific discovery in psychology, and due to its generality, extend to problems from other fields.
SlovakBERT: Slovak Masked Language Model
Venue: arXiv cs.CL 2022
Year: 2022

We introduce a new Slovak masked language model called SlovakBERT. This is to our best knowledge the first paper discussing Slovak transformers-based language models. We evaluate our model on several NLP tasks and achieve state-of-the-art results. This evaluation is likewise the first attempt to establish a benchmark for Slovak language models. We publish the masked language model, as well as the fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.
Automating Internet of Things Network Traffic Collection with Robotic Arm Interactions
Venue: arXiv cs.NI 2022
Year: 2022

Consumer Internet of things research often involves collecting network traffic sent or received by IoT devices. These data are typically collected via crowdsourcing or while researchers manually interact with IoT devices in a laboratory setting. However, manual interactions and crowdsourcing are often tedious, expensive, inaccurate, or do not provide comprehensive coverage of possible IoT device behaviors. We present a new method for generating IoT network traffic using a robotic arm to automate user interactions with devices. This eliminates manual button pressing and enables permutation-based interaction sequences that rigorously explore the range of possible device behaviors. We test this approach with an Arduino-controlled robotic arm, a smart speaker, and a smart thermostat, using machine learning to demonstrate that collected network traffic contains information about device interactions that could be useful for network, security, or privacy analyses. We also provide source code and documentation allowing researchers to easily automate IoT device interactions and network traffic collection in future studies.