OpenRaiser/Intern-Atlas huggingface dataset:
Large-scale photonic natural language processing
Venue: arXiv cs.ET 2022
Year: 2022

Modern machine learning applications require huge artificial networks demanding in computational power and memory. Light-based platforms promise ultra-fast and energy-efficient hardware, which may help in realizing next-generation data processing devices. However, current photonic networks are limited by the number of input-output nodes that can be processed in a single shot. This restricted network capacity prevents their application to relevant large-scale problems such as natural language processing. Here, we realize a photonic processor with a capacity exceeding $1.5 \times 10^{10}$ optical nodes, more than one order of magnitude larger than any previous implementation, which enables photonic large-scale text encoding and classification. By exploiting the full three-dimensional structure of the optical field propagating in free space, we overcome the interpolation threshold and reach the over-parametrized region of machine learning, a condition that allows high-performance natural language processing with a minimal fraction of training points. Our results provide a novel solution to scale-up light-driven computing and open the route to photonic language processing.
Configuration Balancing for Stochastic Requests
Venue: arXiv cs.DS 2022
Year: 2022

The configuration balancing problem with stochastic requests generalizes many well-studied resource allocation problems such as load balancing and virtual circuit routing. In it, we have $m$ resources and $n$ requests. Each request has multiple possible configurations, each of which increases the load of each resource by some amount. The goal is to select one configuration for each request to minimize the makespan: the load of the most-loaded resource. In our work, we focus on a stochastic setting, where we only know the distribution for how each configuration increases the resource loads, learning the realized value only after a configuration is chosen. We develop both offline and online algorithms for configuration balancing with stochastic requests. When the requests are known offline, we give a non-adaptive policy for configuration balancing with stochastic requests that $O(\frac{\log m}{\log \log m})$-approximates the optimal adaptive policy. In particular, this closes the adaptivity gap for this problem as there is an asymptotically matching lower bound even for the very special case of load balancing on identical machines. When requests arrive online in a list, we give a non-adaptive policy that is $O(\log m)$ competitive. Again, this result is asymptotically tight due to information-theoretic lower bounds for very special cases (e.g., for load balancing on unrelated machines). Finally, we show how to leverage adaptivity in the special case of load balancing on related machines to obtain a constant-factor approximation offline and an $O(\log \log m)$-approximation online. A crucial technical ingredient in all of our results is a new structural characterization of the optimal adaptive policy that allows us to limit the correlations between its decisions.
MPIX Stream: An Explicit Solution to Hybrid MPI+X Programming
Venue: arXiv cs.DC 2022
Year: 2022

The hybrid MPI+X programming paradigm, where X refers to threads or GPUs, has gained prominence in the high-performance computing arena. This corresponds to a trend of system architectures growing more heterogeneous. The current MPI standard only specifies the compatibility levels between MPI and threading runtimes. No MPI concept or interface exists for applications to pass thread context or GPU stream context to MPI implementations explicitly. This lack has made performance optimization complicated in some cases and impossible in other cases. We propose a new concept in MPI, called MPIX stream, to represent the general serial execution context that exists in X runtimes. MPIX streams can be directly mapped to threads or GPU execution streams. Passing thread context into MPI allows implementations to precisely map the execution contexts to network endpoints. Passing GPU execution context into MPI allows implementations to directly operate on GPU streams, lowering the CPU/GPU synchronization cost.
Compressible Topological Vector Spaces
Venue: arXiv cs.IT 2022
Year: 2022

The optimum subspace decomposition of the infinite-dimensional compressible random processes in the locally convex Hausdorff space has been propose and its dimension has been measured. We conduct topological analysis of finite- and infinite-dimensional compressible vector spaces. We prove that if there are a sufficient number of separating points in compressible topological vector space, the Banach Limit coincides the minimal linear functional. Then, optimum orthogonal decomposition of the compressible topological vector space can be formulated as a limit for which minimal linear functional occurs. It has been shown that the separable space, also referred to as an optimum subspace, is the subset of the compressible vector space that contains its extreme points. The inseparable subspace is characterized using a novel absorbing null space property through Minkowski functional and Lebesgue measure. We prove that the absorbing null space is a connected subspace of the locally convex space. We purpose reflexive homomorphism that establishes a relation between signal space and double dual space. It has been shown that the Banach limit can be determined by the compact convergence of the Cauchy nets on the left and right hands sides of a reflexive homomorphism. Finally, we propose to measure the optimum subspace dimension using the Frechet distance metric. To apply the Frechet distance to the compressible topological vector space, the Kothe sequence sampled from a continuous function has been proposed. Briefly, the proposed approach measures the sufficient number of separating points of the finite- and infinite-dimensional compressible vector space for the given undersampled operator with respect to Hahn-Banach and Daniell-Kolmogorov theorems. The numerical analysis have been presented for finite- and infinite-dimensional signals.
Recursive Session Logical Relations
Venue: arXiv cs.PL 2022
Year: 2022

Program equivalence is the fulcrum for reasoning about and proving properties of programs. For noninterference, for example, program equivalence up to the secrecy level of an observer is shown. A powerful enabler for such proofs are logical relations. Logical relations only recently were adopted for session types--but exclusively for terminating languages. This paper scales logical relations to recursive session types. It develops a logical relation for progress-sensitive noninterference for linear session types, tackling the challenges non-termination and concurrency pose. The contributions include secrecy-polymorphic processes and the logical relation with metatheory. A distinguishing feature is the choice of "step index" of the logical relation, allowing for a natural proof of transitivity and soundness.
Resolving inconsistencies of runtime configuration changes through change propagation and adjustments
Venue: arXiv cs.DC 2022
Year: 2022

A system configuration may be modified at runtime to adapt the system to changes in its environment or for fine-tuning. For instance, a system administrator may change a few entities/attributes in the configuration to improve error recovery and system availability. However, these changes focusing on fine-tuning availability may violate some global system constraints captured in the configuration and therefore break configuration consistency, system properties and operations. This is generally due to the partialness of the changes performed by the administrator who is not aware of all the entities/attributes being in relations with the modified entities/attributes. In this paper, we propose an approach for completing such partial sets of changes at runtime to resolve inconsistencies arising from those partial changes. This adjustment approach consists of the characterization of related entities/attributes and their modification to re-establish the configuration consistency. We achieve this by propagating the changes in the configuration according to the system constraints following the possible impacts of the configuration entities on each other. We aim at minimizing the complementary modifications to control the side-effects of the change propagation as we target the domain of highly available systems.
Enjoy the Ride Consciously with CAWA: Context-Aware Advisory Warnings for Automated Driving
Venue: arXiv cs.HC 2022
Year: 2022

In conditionally automated driving, drivers decoupled from driving while immersed in non-driving-related tasks (NDRTs) could potentially either miss the system-initiated takeover request (TOR) or a sudden TOR may startle them. To better prepare drivers for a safer takeover in an emergency, we propose novel context-aware advisory warnings (CAWA) for automated driving to gently inform drivers. This will help them stay vigilant while engaging in NDRTs. The key innovation is that CAWA adapts warning modalities according to the context of NDRTs. We conducted a user study to investigate the effectiveness of CAWA. The study results show that CAWA has statistically significant effects on safer takeover behavior, improved driver situational awareness, less attention demand, and more positive user feedback, compared with uniformly distributed speech-based warnings across all NDRTs.
Minimum Coverage Instrumentation
Venue: arXiv cs.DS 2022
Year: 2022

Modern compilers leverage block coverage profile data to carry out downstream profile-guided optimizations to improve the runtime performance and the size of a binary. Given a control-flow graph $G=(V, E)$ of a function in the binary, where nodes in $V$ correspond to basic blocks (sequences of instructions that are always executed sequentially) and edges in $E$ represent jumps in the control flow, the goal is to know for each block $u \in V$ whether $u$ was executed during a session. To this end, extra instrumentation code that records when a block is executed needs to be added to the binary. This extra code creates a time and space overhead, which one would like to minimize as much as possible. Motivated by this application, we study the Minimum Coverage Instrumentation problem, where the goal is to find a minimum size subset of blocks to instrument such that the coverage of the remaining blocks in the graph can be inferred from the coverage status of the instrumented subset. Our main result is an algorithm to find an optimal instrumentation strategy and to carry out the inference in $O(|E|)$ time. We also study variants of this basic problem in which we are interested in learning the coverage of edges instead of the nodes, or when we are only allowed to instrument edges instead of the nodes.
Finite Sample Identification of Bilinear Dynamical Systems
Venue: arXiv cs.LG 2022
Year: 2022

Bilinear dynamical systems are ubiquitous in many different domains and they can also be used to approximate more general control-affine systems. This motivates the problem of learning bilinear systems from a single trajectory of the system's states and inputs. Under a mild marginal mean-square stability assumption, we identify how much data is needed to estimate the unknown bilinear system up to a desired accuracy with high probability. Our sample complexity and statistical error rates are optimal in terms of the trajectory length, the dimensionality of the system and the input size. Our proof technique relies on an application of martingale small-ball condition. This enables us to correctly capture the properties of the problem, specifically our error rates do not deteriorate with increasing instability. Finally, we show that numerical experiments are well-aligned with our theoretical results.
PercentMatch: Percentile-based Dynamic Thresholding for Multi-Label Semi-Supervised Classification
Venue: arXiv cs.CV 2022
Year: 2022

While much of recent study in semi-supervised learning (SSL) has achieved strong performance on single-label classification problems, an equally important yet underexplored problem is how to leverage the advantage of unlabeled data in multi-label classification tasks. To extend the success of SSL to multi-label classification, we first analyze with illustrative examples to get some intuition about the extra challenges exist in multi-label classification. Based on the analysis, we then propose PercentMatch, a percentile-based threshold adjusting scheme, to dynamically alter the score thresholds of positive and negative pseudo-labels for each class during the training, as well as dynamic unlabeled loss weights that further reduces noise from early-stage unlabeled predictions. Without loss of simplicity, we achieve strong performance on Pascal VOC2007 and MS-COCO datasets when compared to recent SSL methods.
Gridless 3D Recovery of Image Sources from Room Impulse Responses
Venue: arXiv cs.SD 2022
Year: 2022

Given a sound field generated by a sparse distribution of impulse image sources, can the continuous 3D positions and amplitudes of these sources be recovered from discrete, bandlimited measurements of the field at a finite set of locations, e.g., a multichannel room impulse response? Borrowing from recent advances in super-resolution imaging, it is shown that this nonlinear, non-convex inverse problem can be efficiently relaxed into a convex linear inverse problem over the space of Radon measures in R3. The linear operator introduced here stems from the fundamental solution of the free-field inhomogenous wave equation combined with the receivers' responses. An adaptation of the Sliding Frank-Wolfe algorithm is proposed to numerically solve the problem off-the-grid, i.e., in continuous 3D space. Simulated experiments show that the approach achieves near-exact recovery of hundreds of image sources using an arbitrarily placed compact 32-channel spherical microphone array in random rectangular rooms. The impact of noise, sampling rate and array diameter on these results is also examined.
Super-model ecosystem: A domain-adaptation perspective
Venue: arXiv cs.LG 2022
Year: 2022

This paper attempts to establish the theoretical foundation for the emerging super-model paradigm via domain adaptation, where one first trains a very large-scale model, {\it i.e.}, super model (or foundation model in some other papers), on a large amount of data and then adapts it to various specific domains. Super-model paradigms help reduce computational and data cost and carbon emission, which is critical to AI industry, especially enormous small and medium-sized enterprises. We model the super-model paradigm as a two-stage diffusion process: (1) in the pre-training stage, the model parameter diffuses from random initials and converges to a steady distribution; and (2) in the fine-tuning stage, the model parameter is transported to another steady distribution. Both training stages can be mathematically modeled by the Uhlenbeck-Ornstein process which converges to two Maxwell-Boltzmann distributions, respectively, each of which characterizes the corresponding convergent model. An $\mathcal O(1/\sqrt{N})$ generalization bound is then established via PAC-Bayesian framework. The theory finds that the generalization error of the fine-tuning stage is dominant in domain adaptation. In addition, our theory suggests that the generalization is determined by a new measure that characterizes the domain discrepancy between the source domain and target domain, based on the covariance matrices and the shift of the converged local minimum.
Large-scale Multi-granular Concept Extraction Based on Machine Reading Comprehension
Venue: arXiv cs.IR 2022
Year: 2022

The concepts in knowledge graphs (KGs) enable machines to understand natural language, and thus play an indispensable role in many applications. However, existing KGs have the poor coverage of concepts, especially fine-grained concepts. In order to supply existing KGs with more fine-grained and new concepts, we propose a novel concept extraction framework, namely MRC-CE, to extract large-scale multi-granular concepts from the descriptive texts of entities. Specifically, MRC-CE is built with a machine reading comprehension model based on BERT, which can extract more fine-grained concepts with a pointer network. Furthermore, a random forest and rule-based pruning are also adopted to enhance MRC-CE's precision and recall simultaneously. Our experiments evaluated upon multilingual KGs, i.e., English Probase and Chinese CN-DBpedia, justify MRC-CE's superiority over the state-of-the-art extraction models in KG completion. Particularly, after running MRC-CE for each entity in CN-DBpedia, more than 7,053,900 new concepts (instanceOf relations) are supplied into the KG. The code and datasets have been released at https://github.com/fcihraeipnusnacwh/MRC-CE
Semantics and Non-Fungible Tokens for Copyright Management on the Metaverse and Beyond
Venue: arXiv cs.CR 2022
Year: 2022

Recent initiatives related to the Metaverse focus on better visualisation, like augmented or virtual reality, but also persistent digital objects. To guarantee real ownership of these digital objects, open systems based on public blockchains and Non-Fungible Tokens (NFTs) are emerging together with a nascent decentralized and open creator economy. To manage this emerging economy in a more organised way, and fight the so common NFT plagiarism, we propose CopyrightLY, a decentralized application for authorship and copyright management. It provides means to claim content authorship, including supporting evidence. Content and metadata are stored in decentralized storage and registered on the blockchain. A token is used to curate these claims, and potential complaints, by staking it on them. Staking is incentivized by the fact that the token is minted using a bonding curve. The tokenomics include the resolution of complaints and enabling the monetization of curated claims. Monetization is achieved through licensing NFTs with metadata enhanced by semantic technologies. Semantic data makes explicit the reuse conditions transferred with the token while keeping the connection to the underlying copyright claims to improve the trustability of the NFTs. Moreover, the semantic metadata is flexible enough to enable licensing not just in the real world. Licenses can refer to reuses in specific locations in a metaverse, thus facilitating the emergence of creative economies in them.
Exploring the Role of Electro-Tactile and Kinesthetic Feedback in Telemanipulation Task
Venue: arXiv cs.RO 2022
Year: 2022

Teleoperation of robotic systems for precise and delicate object grasping requires high-fidelity haptic feedback to obtain comprehensive real-time information about the grasp. In such cases, the most common approach is to use kinesthetic feedback. However, a single contact point information is insufficient to detect the dynamically changing shape of soft objects. This paper proposes a novel telemanipulation system that provides kinesthetic and cutaneous stimuli to the user's hand to achieve accurate liquid dispensing by dexterously manipulating the deformable object (i.e., pipette). The experimental results revealed that the proposed approach to provide the user with multimodal haptic feedback considerably improves the quality of dosing with a remote pipette. Compared with pure visual feedback, the relative dosing error decreased by 66\% and task execution time decreased by 18\% when users manipulated the deformable pipette with a multimodal haptic interface in combination with visual feedback. The proposed technology can be potentially implemented in delicate dosing procedures during the antibody tests for COVID-19, chemical experiments, operation with organic materials, and telesurgery.
Survey on Architectural Attacks: A Unified Classification and Attack Model
Venue: arXiv cs.CR 2022
Year: 2022

According to the World Economic Forum, cyber attacks are considered as one of the most important sources of risk to companies and institutions worldwide. Attacks can target the network, software, and/or hardware. During the past years, much knowledge has been developed to understand and mitigate cyberattacks. However, new threats have appeared in recent years regarding software attacks that exploit hardware vulnerabilities. We define these attacks as architectural attacks. Today, both industry and academy have only limited comprehension of architectural attacks, which represents a critical issue for the design of future systems. To this end, this work proposes a new taxonomy, a new attack model, and a complete survey of existing architectural attacks. As a result, our study provides the tools to understand the Architectural Attacks deeply and start building better designs as well as protection mechanisms.
A Circular Window-based Cascade Transformer for Online Action Detection
Venue: arXiv cs.CV 2022
Year: 2022

Online action detection aims at the accurate action prediction of the current frame based on long historical observations. Meanwhile, it demands real-time inference on online streaming videos. In this paper, we advocate a novel and efficient principle for online action detection. It merely updates the latest and oldest historical representations in one window but reuses the intermediate ones, which have been already computed. Based on this principle, we introduce a window-based cascade Transformer with a circular historical queue, where it conducts multi-stage attentions and cascade refinement on each window. We also explore the association between online action detection and its counterpart offline action segmentation as an auxiliary task. We find that such an extra supervision helps discriminative history clustering and acts as feature augmentation for better training the classifier and cascade refinement. Our proposed method achieves the state-of-the-art performances on three challenging datasets THUMOS'14, TVSeries, and HDD. Codes will be available after acceptance.
Efficient and Interpretable Neural Models for Entity Tracking
Venue: arXiv cs.CL 2022
Year: 2022

What would it take for a natural language model to understand a novel, such as The Lord of the Rings? Among other things, such a model must be able to: (a) identify and record new characters (entities) and their attributes as they are introduced in the text, and (b) identify subsequent references to the characters previously introduced and update their attributes. This problem of entity tracking is essential for language understanding, and thus, useful for a wide array of downstream applications in NLP such as question-answering, summarization. In this thesis, we focus on two key problems in relation to facilitating the use of entity tracking models: (i) scaling entity tracking models to long documents, such as a novel, and (ii) integrating entity tracking into language models. Applying language technologies to long documents has garnered interest recently, but computational constraints are a significant bottleneck in scaling up current methods. In this thesis, we argue that computationally efficient entity tracking models can be developed by representing entities with rich, fixed-dimensional vector representations derived from pretrained language models, and by exploiting the ephemeral nature of entities. We also argue for the integration of entity tracking into language models as it will allow for: (i) wider application given the current ubiquitous use of pretrained language models in NLP applications, and (ii) easier adoption since it is much easier to swap in a new pretrained language model than to integrate a separate standalone entity tracking model.