OpenRaiser/Intern-Atlas huggingface dataset:
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings
Venue: arXiv cs.CL 2023
Year: 2023

Adding interpretability to word embeddings represents an area of active research in text representation. Recent work has explored thepotential of embedding words via so-called polar dimensions (e.g. good vs. bad, correct vs. wrong). Examples of such recent approaches include SemAxis, POLAR, FrameAxis, and BiImp. Although these approaches provide interpretable dimensions for words, they have not been designed to deal with polysemy, i.e. they can not easily distinguish between different senses of words. To address this limitation, we present SensePOLAR, an extension of the original POLAR framework that enables word-sense aware interpretability for pre-trained contextual word embeddings. The resulting interpretable word embeddings achieve a level of performance that is comparable to original contextual word embeddings across a variety of natural language processing tasks including the GLUE and SQuAD benchmarks. Our work removes a fundamental limitation of existing approaches by offering users sense aware interpretations for contextual word embeddings.
Fast Persistent Homology Computation for Functions on $\mathbb{R}$
Venue: arXiv cs.CG 2023
Year: 2023

0-dimensional persistent homology is known, from a computational point of view, as the easy case. Indeed, given a list of $n$ edges in non-decreasing order of filtration value, one only needs a union-find data structure to keep track of the connected components and we get the persistence diagram in time $O(nα(n))$. The running time is thus usually dominated by sorting the edges in $Θ(n\log(n))$. A little-known fact is that, in the particularly simple case of studying the sublevel sets of a piecewise-linear function on $\mathbb{R}$ or $\mathbb{S}^1$, persistence can actually be computed in linear time. This note presents a simple algorithm that achieves this complexity and an extension to image persistence. An implementation is available in Gudhi.
LSDM: Long-Short Diffeomorphic Motion for Weakly-Supervised Ultrasound Landmark Tracking
Venue: arXiv cs.CV 2023
Year: 2023

Accurate tracking of an anatomical landmark over time has been of high interests for disease assessment such as minimally invasive surgery and tumor radiation therapy. Ultrasound imaging is a promising modality benefiting from low-cost and real-time acquisition. However, generating a precise landmark tracklet is very challenging, as attempts can be easily distorted by different interference such as landmark deformation, visual ambiguity and partial observation. In this paper, we propose a long-short diffeomorphic motion network, which is a multi-task framework with a learnable deformation prior to search for the plausible deformation of landmark. Specifically, we design a novel diffeomorphism representation in both long and short temporal domains for delineating motion margins and reducing long-term cumulative tracking errors. To further mitigate local anatomical ambiguity, we propose an expectation maximisation motion alignment module to iteratively optimize both long and short deformation, aligning to the same directional and spatial representation. The proposed multi-task system can be trained in a weakly-supervised manner, which only requires few landmark annotations for tracking and zero annotation for long-short deformation learning. We conduct extensive experiments on two ultrasound landmark tracking datasets. Experimental results show that our proposed method can achieve better or competitive landmark tracking performance compared with other state-of-the-art tracking methods, with a strong generalization capability across different scanner types and different ultrasound modalities.
Artificial Intelligence Generated Coins for Size Comparison
Venue: arXiv cs.CV 2023
Year: 2023

Authors of scientific articles use coins in photographs as a size reference for objects. For this purpose, coins are placed next to objects when taking the photo. In this letter we propose a novel method that uses artificial intelligence (AI) generated images of coins to provide a size reference in photos. The newest generation is able to quickly generate realistic high-quality images from textual descriptions. With the proposed method no physical coin is required while taking photos. Coins can be added to photos that contain none. Furthermore, we show how the coin motif can be matched to the object.
Inferring Needless Write Memory Accesses on Ethereum Bytecode (Extended Version)
Venue: arXiv cs.PL 2023
Year: 2023

Efficiency is a fundamental property of any type of program, but it is even more so in the context of the programs executing on the blockchain (known as smart contracts). This is because optimizing smart contracts has direct consequences on reducing the costs of deploying and executing the contracts, as there are fees to pay related to their bytes-size and to their resource consumption (called gas). Optimizing memory usage is considered a challenging problem that, among other things, requires a precise inference of the memory locations being accessed. This is also the case for the Ethereum Virtual Machine (EVM) bytecode generated by the most-widely used compiler, \texttt{solc}, whose rather unconventional and low-level memory usage challenges automated reasoning. This paper presents a static analysis, developed at the level of the EVM bytecode generated by \texttt{solc}, that infers write memory accesses that are needless and thus can be safely removed. The application of our implementation on more than 19,000 real smart contracts has detected about 6,200 needless write accesses in less than 4 hours. Interestingly, many of these writes were involved in memory usage patterns generated by \texttt{solc} that can be greatly optimized by removing entire blocks of bytecodes. To the best of our knowledge, existing optimization tools cannot infer such needless write accesses, and hence cannot detect these inefficiencies that affect both the deployment and the execution costs of Ethereum smart contracts.
RAD-Sim: Rapid Architecture Exploration for Novel Reconfigurable Acceleration Devices
Venue: arXiv cs.AR 2023
Year: 2023

With the continued growth in field-programmable gate array (FPGA) capacity and their incorporation into new environments such as datacenters, we have witnessed the introduction of a new class of reconfigurable acceleration devices (RADs) that go beyond conventional FPGA architectures. These devices combine a reconfigurable fabric with coarse-grained domain-specialized accelerator blocks all connected via a high-performance packet-switched network-on-chip (NoC) for efficient system-wide communication. However, we lack the tools necessary to efficiently explore the huge design space for RADs, study the complex interactions between their different components and evaluate various combinations of design choices. In this work, we develop RAD-Sim, a cycle-level architecture simulator that allows rapid application-driven exploration of the design space of novel RADs. To showcase the capabilities of RADSim, we map and simulate a state-of-the-art deep learning (DL) inference overlay on a RAD instance incorporating an FPGA fabric and a complex of hard matrix-vector multiplication engines, communicating over a system-wide NoC. Through this example, we show how RAD-Sim can help architects quantify the effect of changing specific architecture parameters on end-to-end application performance.
A Programming Model for GPU Load Balancing
Venue: arXiv cs.DC 2023
Year: 2023

We propose a GPU fine-grained load-balancing abstraction that decouples load balancing from work processing and aims to support both static and dynamic schedules with a programmable interface to implement new load-balancing schedules. Prior to our work, the only way to unleash the GPU's potential on irregular problems has been to workload-balance through application-specific, tightly coupled load-balancing techniques. With our open-source framework for load-balancing, we hope to improve programmers' productivity when developing irregular-parallel algorithms on the GPU, and also improve the overall performance characteristics for such applications by allowing a quick path to experimentation with a variety of existing load-balancing techniques. Consequently, we also hope that by separating the concerns of load-balancing from work processing within our abstraction, managing and extending existing code to future architectures becomes easier.
1st Place Solution for ECCV 2022 OOD-CV Challenge Object Detection Track
Venue: arXiv cs.CV 2023
Year: 2023

OOD-CV challenge is an out-of-distribution generalization task. To solve this problem in object detection track, we propose a simple yet effective Generalize-then-Adapt (G&A) framework, which is composed of a two-stage domain generalization part and a one-stage domain adaptation part. The domain generalization part is implemented by a Supervised Model Pretraining stage using source data for model warm-up and a Weakly Semi-Supervised Model Pretraining stage using both source data with box-level label and auxiliary data (ImageNet-1K) with image-level label for performance boosting. The domain adaptation part is implemented as a Source-Free Domain Adaptation paradigm, which only uses the pre-trained model and the unlabeled target data to further optimize in a self-supervised training manner. The proposed G&A framework help us achieve the first place on the object detection leaderboard of the OOD-CV challenge. Code will be released in https://github.com/hikvision-research/OOD-CV.
Data-centric AI: Perspectives and Challenges
Venue: arXiv cs.AI 2023
Year: 2023

The role of data in building AI systems has recently been significantly magnified by the emerging concept of data-centric AI (DCAI), which advocates a fundamental shift from model advancements to ensuring data quality and reliability. Although our community has continuously invested efforts into enhancing data in different aspects, they are often isolated initiatives on specific tasks. To facilitate the collective initiative in our community and push forward DCAI, we draw a big picture and bring together three general missions: training data development, inference data development, and data maintenance. We provide a top-level discussion on representative DCAI tasks and share perspectives. Finally, we list open challenges. More resources are summarized at https://github.com/daochenzha/data-centric-AI
LZR: Identifying Unexpected Internet Services
Venue: arXiv cs.CR 2023
Year: 2023

Internet-wide scanning is a commonly used research technique that has helped uncover real-world attacks, find cryptographic weaknesses, and understand both operator and miscreant behavior. Studies that employ scanning have largely assumed that services are hosted on their IANA-assigned ports, overlooking the study of services on unusual ports. In this work, we investigate where Internet services are deployed in practice and evaluate the security posture of services on unexpected ports. We show protocol deployment is more diffuse than previously believed and that protocols run on many additional ports beyond their primary IANA-assigned port. For example, only 3% of HTTP and 6% of TLS services run on ports 80 and 443, respectively. Services on non-standard ports are more likely to be insecure, which results in studies dramatically underestimating the security posture of Internet hosts. Building on our observations, we introduce LZR ("Laser"), a system that identifies 99% of identifiable unexpected services in five handshakes and dramatically reduces the time needed to perform application-layer scans on ports with few responsive expected services (e.g., 5500% speedup on 27017/MongoDB). We conclude with recommendations for future studies.
Grant-Free Random Access of IoT devices in Massive MIMO with Partial CSI
Venue: arXiv cs.IT 2023
Year: 2023

The number of wireless devices is drastically increasing, resulting in many devices contending for radio resources. In this work, we present an algorithm to detect active devices for unsourced random access, i.e., the devices are uncoordinated. The devices use a unique, but non-orthogonal preamble, known to the network, prior to sending the payload data. They do not employ any carrier sensing technique and blindly transmit the preamble and data. To detect the active users, we exploit partial channel state information (CSI), which could have been obtained through a previous channel estimate. For static devices, e.g., Internet of Things nodes, it is shown that CSI is less time-variant than assumed in many theoretical works. The presented iterative algorithm uses a maximum likelihood approach to estimate both the activity and a potential phase offset of each known device. The convergence of the proposed algorithm is evaluated. The performance in terms of probability of miss detection and false alarm is assessed for different qualities of partial CSI and different signal-to-noise ratio.
Semantic Segmentation via Pixel-to-Center Similarity Calculation
Venue: arXiv cs.CV 2023
Year: 2023

Since the fully convolutional network has achieved great success in semantic segmentation, lots of works have been proposed focusing on extracting discriminative pixel feature representations. However, we observe that existing methods still suffer from two typical challenges, i.e. (i) large intra-class feature variation in different scenes, (ii) small inter-class feature distinction in the same scene. In this paper, we first rethink semantic segmentation from a perspective of similarity between pixels and class centers. Each weight vector of the segmentation head represents its corresponding semantic class in the whole dataset, which can be regarded as the embedding of the class center. Thus, the pixel-wise classification amounts to computing similarity in the final feature space between pixels and the class centers. Under this novel view, we propose a Class Center Similarity layer (CCS layer) to address the above-mentioned challenges by generating adaptive class centers conditioned on different scenes and supervising the similarities between class centers. It utilizes a Adaptive Class Center Module (ACCM) to generate class centers conditioned on each scene, which adapt the large intra-class variation between different scenes. Specially designed loss functions are introduced to control both inter-class and intra-class distances based on predicted center-to-center and pixel-to-center similarity, respectively. Finally, the CCS layer outputs the processed pixel-to-center similarity as the segmentation prediction. Extensive experiments demonstrate that our model performs favourably against the state-of-the-art CNN-based methods.
What you see is (not) what you get: A VR Framework for Correcting Robot Errors
Venue: arXiv cs.RO 2023
Year: 2023

Many solutions tailored for intuitive visualization or teleoperation of virtual, augmented and mixed (VAM) reality systems are not robust to robot failures, such as the inability to detect and recognize objects in the environment or planning unsafe trajectories. In this paper, we present a novel virtual reality (VR) framework where users can (i) recognize when the robot has failed to detect a real-world object, (ii) correct the error in VR, (iii) modify proposed object trajectories and, (iv) implement behaviors on a real-world robot. Finally, we propose a user study aimed at testing the efficacy of our framework. Project materials can be found in the OSF repository.
Heterogeneous Beliefs and Multi-Population Learning in Network Games
Venue: arXiv cs.MA 2023
Year: 2023

The effect of population heterogeneity in multi-agent learning is practically relevant but remains far from being well-understood. Motivated by this, we introduce a model of multi-population learning that allows for heterogeneous beliefs within each population and where agents respond to their beliefs via smooth fictitious play (SFP).We show that the system state -- a probability distribution over beliefs -- evolves according to a system of partial differential equations akin to the continuity equations that commonly desccribe transport phenomena in physical systems. We establish the convergence of SFP to Quantal Response Equilibria in different classes of games capturing both network competition as well as network coordination. We also prove that the beliefs will eventually homogenize in all network games. Although the initial belief heterogeneity disappears in the limit, we show that it plays a crucial role for equilibrium selection in the case of coordination games as it helps select highly desirable equilibria. Contrary, in the case of network competition, the resulting limit behavior is independent of the initialization of beliefs, even when the underlying game has many distinct Nash equilibria.
Graph Laplacian for Semi-Supervised Learning
Venue: arXiv cs.CV 2023
Year: 2023

Semi-supervised learning is highly useful in common scenarios where labeled data is scarce but unlabeled data is abundant. The graph (or nonlocal) Laplacian is a fundamental smoothing operator for solving various learning tasks. For unsupervised clustering, a spectral embedding is often used, based on graph-Laplacian eigenvectors. For semi-supervised problems, the common approach is to solve a constrained optimization problem, regularized by a Dirichlet energy, based on the graph-Laplacian. However, as supervision decreases, Dirichlet optimization becomes suboptimal. We therefore would like to obtain a smooth transition between unsupervised clustering and low-supervised graph-based classification. In this paper, we propose a new type of graph-Laplacian which is adapted for Semi-Supervised Learning (SSL) problems. It is based on both density and contrastive measures and allows the encoding of the labeled data directly in the operator. Thus, we can perform successfully semi-supervised learning using spectral clustering. The benefits of our approach are illustrated for several SSL problems.
Shape2SAS -- a web application to simulate small-angle scattering data and pair distance distributions from user-defined shapes
Venue: arXiv cs.GR 2023
Year: 2023

Shape2SAS is a web application that allows researchers and students to build intuition and understanding of small-angle scattering. It is available at https://somo.chem.utk.edu/shape2sas. The user defines a model of arbitrary shape by combining geometrical subunits, and Shape2SAS then calculates and displays the scattering intensity, the pair distance distribution as well as a visualization of the user-defined shape. Simulated data with realistic noise are also generated. We demonstrate how Shape2SAS can calculate and display the different scattering patterns for various geometrical shapes, such as spheres and cylinders. We also demonstrate how the effect of structure factors can be visualized. Finally, we show how multi-contrast particles can readily be generated, and how the calculated scattering may be used to validate and visualize analytical models generated in analysis software for fitting small-angle scattering data.
Modeling the evolution of temporal knowledge graphs with uncertainty
Venue: arXiv cs.LG 2023
Year: 2023

Forecasting future events is a fundamental challenge for temporal knowledge graphs (tKG). As in real life predicting a mean function is most of the time not sufficient, but the question remains how confident can we be about our prediction? Thus, in this work, we will introduce a novel graph neural network architecture (WGP-NN) employing (weighted) Gaussian processes (GP) to jointly model the temporal evolution of the occurrence probability of events and their time-dependent uncertainty. Especially we employ Gaussian processes to model the uncertainty of future links by their ability to predict predictive variance. This is in contrast to existing works, which are only able to express uncertainties in the learned entity representations. Moreover, WGP-NN can model parameter-free complex temporal and structural dynamics of tKGs in continuous time. We further demonstrate the model's state-of-the-art performance on two real-world benchmark datasets.
Unsupervised Driving Event Discovery Based on Vehicle CAN-data
Venue: arXiv cs.LG 2023
Year: 2023

The data collected from a vehicle's Controller Area Network (CAN) can quickly exceed human analysis or annotation capabilities when considering fleets of vehicles, which stresses the importance of unsupervised machine learning methods. This work presents a simultaneous clustering and segmentation approach for vehicle CAN-data that identifies common driving events in an unsupervised manner. The approach builds on self-supervised learning (SSL) for multivariate time series to distinguish different driving events in the learned latent space. We evaluate our approach with a dataset of real Tesla Model 3 vehicle CAN-data and a two-hour driving session that we annotated with different driving events. With our approach, we evaluate the applicability of recent time series-related contrastive and generative SSL techniques to learn representations that distinguish driving events. Compared to state-of-the-art (SOTA) generative SSL methods for driving event discovery, we find that contrastive learning approaches reach similar performance.
Channel Training and Estimation for Reconfigurable Intelligent Surfaces: Exposition of Principles, Approaches, and Open Problems
Venue: arXiv cs.IT 2023
Year: 2023

Reconfigurable intelligent surfaces (RIS) are passive controllable arrays of small reflectors that direct electromagnetic energy towards or away from the target nodes, thereby allowing better management of signals and interference in a wireless network. The RIS has the potential for significantly improving the performance of wireless networks. Unfortunately, RIS also multiplies the number of Channel State Information (CSI) coefficients between the transmitter and receiver, which magnifies the challenges in estimating and communicating the channel state information. Furthermore, the simplicity and cost-effectiveness of the passive RIS also implies that the incoming links are not locally estimated at the RIS, and fresh pilots are not inserted into outgoing RIS links. This introduces new challenges for training and estimation of channel state information. The rapid growth of the literature on CSI acquisition in RIS-aided systems has been accompanied by variations in the underlying assumptions, models, and notation, which can obscure the similarities and differences of various techniques, and their relative merits. This paper presents a comprehensive exposition of principles and approaches in RIS channel estimation. The basic ideas underlying each class of techniques are reduced to their simplest form under a unified model and notation, and various approaches within each class are discussed. Several open problems in this area are identified and highlighted.
SynMotor: A Benchmark Suite for Object Attribute Regression and Multi-task Learning
Venue: arXiv cs.CV 2023
Year: 2023

In this paper, we develop a novel benchmark suite including both a 2D synthetic image dataset and a 3D synthetic point cloud dataset. Our work is a sub-task in the framework of a remanufacturing project, in which small electric motors are used as fundamental objects. Apart from the given detection, classification, and segmentation annotations, the key objects also have multiple learnable attributes with ground truth provided. This benchmark can be used for computer vision tasks including 2D/3D detection, classification, segmentation, and multi-attribute learning. It is worth mentioning that most attributes of the motors are quantified as continuously variable rather than binary, which makes our benchmark well-suited for the less explored regression tasks. In addition, appropriate evaluation metrics are adopted or developed for each task and promising baseline results are provided. We hope this benchmark can stimulate more research efforts on the sub-domain of object attribute learning and multi-task learning in the future.