OpenRaiser/Intern-Atlas huggingface dataset:
Embracing Ambiguity: Improving Similarity-oriented Tasks with Contextual Synonym Knowledge
Venue: arXiv cs.CL 2022
Year: 2022

Contextual synonym knowledge is crucial for those similarity-oriented tasks whose core challenge lies in capturing semantic similarity between entities in their contexts, such as entity linking and entity matching. However, most Pre-trained Language Models (PLMs) lack synonym knowledge due to inherent limitations of their pre-training objectives such as masked language modeling (MLM). Existing works which inject synonym knowledge into PLMs often suffer from two severe problems: (i) Neglecting the ambiguity of synonyms, and (ii) Undermining semantic understanding of original PLMs, which is caused by inconsistency between the exact semantic similarity of the synonyms and the broad conceptual relevance learned from the original corpus. To address these issues, we propose PICSO, a flexible framework that supports the injection of contextual synonym knowledge from multiple domains into PLMs via a novel entity-aware Adapter which focuses on the semantics of the entities (synonyms) in the contexts. Meanwhile, PICSO stores the synonym knowledge in additional parameters of the Adapter structure, which prevents it from corrupting the semantic understanding of the original PLM. Extensive experiments demonstrate that PICSO can dramatically outperform the original PLMs and the other knowledge and synonym injection models on four different similarity-oriented tasks. In addition, experiments on GLUE prove that PICSO also benefits general natural language understanding tasks. Codes and data will be public.
Who Tracks Who? A Surveillance Capitalist Examination of Commercial Bluetooth Tracking Networks
Venue: arXiv cs.SI 2022
Year: 2022

Object and person tracking networks powered by Bluetooth and mobile devices have become increasingly popular for purposes of public safety and individual concerns. This essay examines popular commercial tracking networks and their campaigns from Apple, Samsung and Tile with reference to surveillance capitalism and digital privacy, discovering the hidden assets commodified through said networks, and their potential of turning users into unregulated digital labour while leaving individual privacy at risk.
Vibration Free Flexible Object Handling with a Robot Manipulator Using Learning Control
Venue: arXiv cs.RO 2022
Year: 2022

Many industries extensively use flexible materials. Effective approaches for handling flexible objects with a robot manipulator must address residual vibrations. Existing solutions rely on complex models, use additional instrumentation for sensing the vibrations, or do not exploit the repetitive nature of most industrial tasks. This paper develops an iterative learning control approach that jointly learns model parameters and residual dynamics using only the interoceptive sensors of the robot. The learned model is subsequently utilized to design optimal (PTP) trajectories that accounts for residual vibration, nonlinear kinematics of the manipulator and joint limits. We experimentally show that the proposed approach reduces the residual vibrations by an order of magnitude compared with optimal vibration suppression using the analytical model and threefold compared with the available state-of-the-art method. These results demonstrate that effective handling of a flexible object does not require neither complex models nor additional instrumentation.
R2-MLP: Round-Roll MLP for Multi-View 3D Object Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Recently, vision architectures based exclusively on multi-layer perceptrons (MLPs) have gained much attention in the computer vision community. MLP-like models achieve competitive performance on a single 2D image classification with less inductive bias without hand-crafted convolution layers. In this work, we explore the effectiveness of MLP-based architecture for the view-based 3D object recognition task. We present an MLP-based architecture termed as Round-Roll MLP (R$^2$-MLP). It extends the spatial-shift MLP backbone by considering the communications between patches from different views. R$^2$-MLP rolls part of the channels along the view dimension and promotes information exchange between neighboring views. We benchmark MLP results on ModelNet10 and ModelNet40 datasets with ablations in various aspects. The experimental results show that, with a conceptually simple structure, our R$^2$-MLP achieves competitive performance compared with existing state-of-the-art methods.
Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness
Venue: arXiv cs.CL 2022
Year: 2022

Data-driven predictive solutions predominant in commercial applications tend to suffer from biases and stereotypes, which raises equity concerns. Prediction models may discover, use, or amplify spurious correlations based on gender or other protected personal characteristics, thus discriminating against marginalized groups. Mitigating gender bias has become an important research focus in natural language processing (NLP) and is an area where annotated corpora are available. Data augmentation reduces gender bias by adding counterfactual examples to the training dataset. In this work, we show that some of the examples in the augmented dataset can be not important or even harmful for fairness. We hence propose a general method for pruning both the factual and counterfactual examples to maximize the model's fairness as measured by the demographic parity, equality of opportunity, and equality of odds. The fairness achieved by our method surpasses that of data augmentation on three text classification datasets, using no more than half of the examples in the augmented dataset. Our experiments are conducted using models of varying sizes and pre-training settings.
Structure-Encoding Auxiliary Tasks for Improved Visual Representation in Vision-and-Language Navigation
Venue: arXiv cs.CV 2022
Year: 2022

In Vision-and-Language Navigation (VLN), researchers typically take an image encoder pre-trained on ImageNet without fine-tuning on the environments that the agent will be trained or tested on. However, the distribution shift between the training images from ImageNet and the views in the navigation environments may render the ImageNet pre-trained image encoder suboptimal. Therefore, in this paper, we design a set of structure-encoding auxiliary tasks (SEA) that leverage the data in the navigation environments to pre-train and improve the image encoder. Specifically, we design and customize (1) 3D jigsaw, (2) traversability prediction, and (3) instance classification to pre-train the image encoder. Through rigorous ablations, our SEA pre-trained features are shown to better encode structural information of the scenes, which ImageNet pre-trained features fail to properly encode but is crucial for the target navigation task. The SEA pre-trained features can be easily plugged into existing VLN agents without any tuning. For example, on Test-Unseen environments, the VLN agents combined with our SEA pre-trained features achieve absolute success rate improvement of 12% for Speaker-Follower, 5% for Env-Dropout, and 4% for AuxRN.
Optimal quaternary linear codes with one-dimensional Hermitian hull and the related EAQECCs
Venue: arXiv cs.IT 2022
Year: 2022

Linear codes with small hulls over finite fields have been extensively studied due to their practical applications in computational complexity and information protection. In this paper, we develop a general method to determine the exact value of $D_4^H(n,k,1)$ for $n\leq 12$ or $k\in \{1,2,3,n-1,n-2,n-3\}$, where $D_4^H(n,k,1)$ denotes the largest minimum distance among all quaternary linear $[n,k]$ codes with one-dimensional Hermitian hull. As a consequence, we solve a conjecture proposed by Mankean and Jitman on the largest minimum distance of a quaternary linear code with one-dimensional Hermitian hull. As an application, we construct some binary entanglement-assisted quantum error-correcting codes (EAQECCs) from quaternary linear codes with one-dimensional Hermitian hull. Some of these EAQECCs are optimal codes, and some of them are better than previously known ones.
Unifying Vision-Language Representation Space with Single-tower Transformer
Venue: arXiv cs.LG 2022
Year: 2022

Contrastive learning is a form of distance learning that aims to learn invariant features from two related representations. In this paper, we explore the bold hypothesis that an image and its caption can be simply regarded as two different views of the underlying mutual information, and train a model to learn a unified vision-language representation space that encodes both modalities at once in a modality-agnostic manner. We first identify difficulties in learning a generic one-tower model for vision-language pretraining (VLP), and propose OneR as a simple yet effective framework for our goal. We discover intriguing properties that distinguish OneR from the previous works that learn modality-specific representation spaces such as zero-shot object localization, text-guided visual reasoning and multi-modal retrieval, and present analyses to provide insights into this new form of multi-modal representation learning. Thorough evaluations demonstrate the potential of a unified modality-agnostic VLP framework.
A min-max theorem for the minimum fleet-size problem
Venue: arXiv cs.DM 2022
Year: 2022

A retrospective fleet-sizing problem can be solved via bipartite matching, where a maximum cardinality matching corresponds to the minimum number of vehicles needed to cover all trips. We prove a min-max theorem on this minimum fleet-size problem: the maximum number of pairwise incompatible trips is equal to the minimum fleet size needed.
An Empirical Study of Package Management Issues via Stack Overflow
Venue: arXiv cs.SE 2022
Year: 2022

The package manager (PM) is crucial to most technology stacks, acting as a broker to ensure that a verified dependency package is correctly installed, configured, or removed from an application. Diversity in technology stacks has led to dozens of PMs with various features. While our recent study indicates that package management features of PM are related to end-user experiences, it is unclear what those issues are and what information is required to resolve them. In this paper, we have investigated PM issues faced by end-users through an empirical study of content on Stack Overflow (SO). We carried out a qualitative analysis of 1,131 questions and their accepted answer posts for three popular PMs (i.e., Maven, npm, and NuGet) to identify issue types, underlying causes, and their resolutions. Our results confirm that end-users struggle with PM tool usage (approximately 64-72%). We observe that most issues are raised by end-users due to lack of instructions and errors messages from PM tools. In terms of issue resolution, we find that external link sharing is the most common practice to resolve PM issues. Additionally, we observe that links pointing to useful resources (i.e., official documentation websites, tutorials, etc.) are most frequently shared, indicating the potential for tool support and the ability to provide relevant information for PM end-users.
A Novel Uncalibrated Visual Servoing Controller Baesd on Model-Free Adaptive Control Method with Neural Network
Venue: arXiv cs.RO 2022
Year: 2022

Nowadays, with the continuous expansion of application scenarios of robotic arms, there are more and more scenarios where nonspecialist come into contact with robotic arms. However, in terms of robotic arm visual servoing, traditional Position-based Visual Servoing (PBVS) requires a lot of calibration work, which is challenging for the nonspecialist to cope with. To cope with this situation, Uncalibrated Image-Based Visual Servoing (UIBVS) frees people from tedious calibration work. This work applied a model-free adaptive control (MFAC) method which means that the parameters of controller are updated in real time, bringing better ability of suppression changes of system and environment. An artificial intelligent neural network is applied in designs of controller and estimator for hand-eye relationship. The neural network is updated with the knowledge of the system input and output information in MFAC method. Inspired by "predictive model" and "receding-horizon" in Model Predictive Control (MPC) method and introducing similar structures into our algorithm, we realizes the uncalibrated visual servoing for both stationary targets and moving trajectories. Simulated experiments with a robotic manipulator will be carried out to validate the proposed algorithm.
300 GHz Channel Measurement and Characterization in the Atrium of a Building
Venue: arXiv cs.IT 2022
Year: 2022

With abundant bandwidth resource, the Terahertz band (0.1~THz to 10~THz) is envisioned as a key technology to realize ultra-high data rates in the 6G and beyond mobile communication systems. However, moving to the THz band, existing channel models dedicated for microwave or millimeter-wave bands are ineffective. To fill this research gap, extensive channel measurement campaigns and characterizations are necessary. In this paper, using a frequency-domain Vector Network Analyzer (VNA)-based sounder, a measurement campaign is conducted in the outdoor atrium of a building in 306-321 GHz band. The measured data are further processed to obtain the channel transfer functions (CTFs), parameters of multipath components (MPCs), as well as clustering results. Based on the MPC parameters, the channel characteristics, such as path loss, shadow fading, K-factor, etc., are calculated and analyzed. The extracted channel characteristics and numerology are helpful to study channel modeling and guide system design for THz communications.
VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models
Venue: arXiv cs.CV 2022
Year: 2022

Diffusion models have shown impressive results in text-to-image synthesis. Using massive datasets of captioned images, diffusion models learn to generate raster images of highly diverse objects and scenes. However, designers frequently use vector representations of images like Scalable Vector Graphics (SVGs) for digital icons or art. Vector graphics can be scaled to any size, and are compact. We show that a text-conditioned diffusion model trained on pixel representations of images can be used to generate SVG-exportable vector graphics. We do so without access to large datasets of captioned SVGs. By optimizing a differentiable vector graphics rasterizer, our method, VectorFusion, distills abstract semantic knowledge out of a pretrained diffusion model. Inspired by recent text-to-3D work, we learn an SVG consistent with a caption using Score Distillation Sampling. To accelerate generation and improve fidelity, VectorFusion also initializes from an image sample. Experiments show greater quality than prior work, and demonstrate a range of styles including pixel art and sketches. See our project webpage at https://ajayj.com/vectorfusion .
An Implicit Parametric Morphable Dental Model
Venue: arXiv cs.CV 2022
Year: 2022

3D Morphable models of the human body capture variations among subjects and are useful in reconstruction and editing applications. Current dental models use an explicit mesh scene representation and model only the teeth, ignoring the gum. In this work, we present the first parametric 3D morphable dental model for both teeth and gum. Our model uses an implicit scene representation and is learned from rigidly aligned scans. It is based on a component-wise representation for each tooth and the gum, together with a learnable latent code for each of such components. It also learns a template shape thus enabling several applications such as segmentation, interpolation, and tooth replacement. Our reconstruction quality is on par with the most advanced global implicit representations while enabling novel applications. Project page: https://vcai.mpi-inf.mpg.de/projects/DMM/
Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanations
Venue: arXiv cs.CV 2022
Year: 2022

Applying traditional post-hoc attribution methods to segmentation or object detection predictors offers only limited insights, as the obtained feature attribution maps at input level typically resemble the models' predicted segmentation mask or bounding box. In this work, we address the need for more informative explanations for these predictors by proposing the post-hoc eXplainable Artificial Intelligence method L-CRP to generate explanations that automatically identify and visualize relevant concepts learned, recognized and used by the model during inference as well as precisely locate them in input space. Our method therefore goes beyond singular input-level attribution maps and, as an approach based on the recently published Concept Relevance Propagation technique, is efficiently applicable to state-of-the-art black-box architectures in segmentation and object detection, such as DeepLabV3+ and YOLOv6, among others. We verify the faithfulness of our proposed technique by quantitatively comparing different concept attribution methods, and discuss the effect on explanation complexity on popular datasets such as CityScapes, Pascal VOC and MS COCO 2017. The ability to precisely locate and communicate concepts is used to reveal and verify the use of background features, thereby highlighting possible biases of the model.
Revisiting a Privacy-Preserving Location-based Service Protocol using Edge Computing
Venue: arXiv cs.CR 2022
Year: 2022

Location-based services are getting more popular day by day. Finding nearby stores, proximity-based marketing, on-road service assistance, etc., are some of the services that use location-based services. In location-based services, user information like user identity, user query, and location must be protected. Ma et al. (INFOCOM-BigSecurity 2019) proposed a privacy-preserving location-based service using Somewhat Homomorphic Encryption (SHE). Their protocol uses edge nodes that compute on SHE encrypted location data and determines the $k$-nearest points of interest contained in the Location-based Server (LBS) without revealing the original user coordinates to LBS, hence, ensuring privacy of users locations. In this work, we show that the above protocol by Ma et al. has a critical flaw. In particular, we show that their secure comparison protocol has a correctness issue in that it will not lead to correct comparison. A major consequence of this flaw is that straightforward approaches to fix this issue will make their protocol insecure. Namely, the LBS will be able to recover the actual locations of the users in each and every query.
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation
Venue: arXiv cs.SE 2022
Year: 2022

We introduce DS-1000, a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas. Compared to prior works, DS-1000 incorporates three core features. First, our problems reflect diverse, realistic, and practical use cases since we collected them from StackOverflow. Second, our automatic evaluation is highly specific (reliable) -- across all Codex-002-predicted solutions that our evaluation accept, only 1.8% of them are incorrect; we achieve this with multi-criteria metrics, checking both functional correctness by running test cases and surface-form constraints by restricting API usages or keywords. Finally, we proactively defend against memorization by slightly modifying our problems to be different from the original StackOverflow source; consequently, models cannot answer them correctly by memorizing the solutions from pre-training. The current best public system (Codex-002) achieves 43.3% accuracy, leaving ample room for improvement. We release our benchmark at https://ds1000-code-gen.github.io.
Differentiable Physics-based Greenhouse Simulation
Venue: arXiv cs.LG 2022
Year: 2022

We present a differentiable greenhouse simulation model based on physical processes whose parameters can be obtained by training from real data. The physics-based simulation model is fully interpretable and is able to do state prediction for both climate and crop dynamics in the greenhouse over very a long time horizon. The model works by constructing a system of linear differential equations and solving them to obtain the next state. We propose a procedure to solve the differential equations, handle the problem of missing unobservable states in the data, and train the model efficiently. Our experiment shows the procedure is effective. The model improves significantly after training and can simulate a greenhouse that grows cucumbers accurately.
Machine Learning enabled models for YouTube Ranking Mechanism and Views Prediction
Venue: arXiv cs.IR 2022
Year: 2022

With the continuous increase of internet usage in todays time, everyone is influenced by this source of the power of technology. Due to this, the rise of applications and games Is unstoppable. A major percentage of our population uses these applications for multiple purposes. These range from education, communication, news, entertainment, and many more. Out of this, the application that is making sure that the world stays in touch with each other and with current affairs is social media. Social media applications have seen a boom in the last 10 years with the introduction of smartphones and the internet being available at affordable prices. Applications like Twitch and Youtube are some of the best platforms for producing content and expressing their talent as well. It is the goal of every content creator to post the best and most reliable content so that they can gain recognition. It is important to know the methods of achieving popularity easily, which is what this paper proposes to bring to the spotlight. There should be certain parameters based on which the reach of content could be multiplied by a good factor. The proposed research work aims to identify and estimate the reach, popularity, and views of a YouTube video by using certain features using machine learning and AI techniques. A ranking system would also be used keeping the trending videos in consideration. This would eventually help the content creator know how authentic their content is and healthy competition to make better content before uploading the video on the platform will be ensured.
Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers
Venue: arXiv cs.CL 2022
Year: 2022

Large-scale transformer models have become the de-facto architectures for various machine learning applications, e.g., CV and NLP. However, those large models also introduce prohibitive training costs. To mitigate this issue, we propose a novel random and layerwise token dropping method (random-LTD), which skips the computation of a subset of the input tokens at all middle layers. Particularly, random-LTD achieves considerable speedups and comparable accuracy as the standard training baseline. Compared to other token dropping methods, random-LTD does not require (1) any importance score-based metrics, (2) any special token treatment (e.g., [CLS]), and (3) many layers in full sequence length training except the first and the last layers. Besides, a new LayerToken learning rate schedule is proposed for pretraining problems that resolve the heavy tuning requirement for our proposed training mechanism. Finally, we demonstrate that random-LTD can be applied to broader applications, including GPT and BERT pretraining as well as ViT and GPT finetuning tasks. Our results show that random-LTD can save about 33.3% theoretical compute cost and 25.6% wall-clock training time while achieving similar zero-shot evaluations on GPT-31.3B as compared to baseline.