OpenRaiser/Intern-Atlas huggingface dataset:
A Theory of Abstraction in Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Reinforcement learning defines the problem facing agents that learn to make good decisions through action and observation alone. To be effective problem solvers, such agents must efficiently explore vast worlds, assign credit from delayed feedback, and generalize to new experiences, all while making use of limited data, computational resources, and perceptual bandwidth. Abstraction is essential to all of these endeavors. Through abstraction, agents can form concise models of their environment that support the many practices required of a rational, adaptive decision maker. In this dissertation, I present a theory of abstraction in reinforcement learning. I first offer three desiderata for functions that carry out the process of abstraction: they should 1) preserve representation of near-optimal behavior, 2) be learned and constructed efficiently, and 3) lower planning or learning time. I then present a suite of new algorithms and analysis that clarify how agents can learn to abstract according to these desiderata. Collectively, these results provide a partial path toward the discovery and use of abstraction that minimizes the complexity of effective reinforcement learning.
Web3: A Decentralized Societal Infrastructure for Identity, Trust, Money, and Data
Venue: arXiv cs.DC 2022
Year: 2022

A movement for a more transparent and decentralized Internet is globally attracting more attention. People are becoming more privacy-aware of their online identities and data. The Internet is constantly evolving. Web2 focused on companies that provide services in exchange for personal user data. Web3 commits to user-centricity using decentralization and zero-server architectures. The current digital society demands a global change to empower citizens and take back control. Citizens are locked into big-tech for personal data storage and their for-profit digital identity. Protection of data has proven to be essential, especially due to increased home Internet traffic during the COVID pandemic. Citizens do not possess their own travel documents. The European Commission aims to transition this governmental property towards self-sovereign identity, introducing many new opportunities. Citizens are locked into banks with non-portable IBAN accounts and unsustainable legacy banking infrastructures. Migration to all-digital low-fraud infrastructures and healthier competitive ecosystems is essential. The overall challenge is to return the power to citizens and users again. The transition to a more decentralized Internet is the first crucial step in the realization of user-centricity. This thesis presents the first exploratory study that integrates governmental-issued travel documents into a (decentralized) societal infrastructure. These self-sovereign identities form the authentic base to a private and secure transfer of money and data, and can effectively provide trust in authenticity that is currently missing in online conversations. A fully operational zero-server infrastructure that incorporates all our requirements has been developed for Android using the P2P network overlay IPv8, and a personalized blockchain called TrustChain...
Short-term passenger flow prediction for multi-traffic modes: A Transformer and residual network based multi-task learning method
Venue: arXiv cs.LG 2022
Year: 2022

With the prevailing of mobility as a service (MaaS), it becomes increasingly important to manage multi-traffic modes simultaneously and cooperatively. As an important component of MaaS, short-term passenger flow prediction for multi-traffic modes has thus been brought into focus. It is a challenging problem because the spatiotemporal features of multi-traffic modes are critically complex. Moreover, the passenger flows of multi-traffic modes differentiate and fluctuate significantly. To solve these problems, this paper proposes a multitask learning-based model, called Res-Transformer, for short-term inflow prediction of multi-traffic modes (subway, taxi, and bus). Each traffic mode is treated as a single task in the model. The Res-Transformer consists of two parts: (1) several modified Transformer layers comprising the conv-Transformer layer and the multi-head attention mechanism, which helps to extract the spatial and temporal features of multi-traffic modes, (2) the structure of residual network is utilized to obtain the correlations of different traffic modes and prevent gradient vanishing, gradient explosion, and overfitting. The Res-Transformer model is evaluated on two large-scale real-world datasets from Beijing, China. One is the region of a traffic hub and the other is the region of a residential area. Experiments are conducted to compare the performance of the proposed model with several baseline models. Results prove the effectiveness and robustness of the proposed method. This paper can give critical insights into the short-term inflow prediction for multi-traffic modes.
se-Shweshwe Inspired Fashion Generation
Venue: arXiv cs.CV 2022
Year: 2022

Fashion is one of the ways in which we show ourselves to the world. It is a reflection of our personal decisions and one of the ways in which people distinguish and represent themselves. In this paper, we focus on the fashion design process and expand computer vision for fashion beyond its current focus on western fashion. We discuss the history of Southern African se-Shweshwe fabric fashion, the collection of a se-Shweshwe dataset, and the application of sketch-to-design image generation for affordable fashion-design. The application to fashion raises both technical questions of training with small amounts of data, and also important questions for computer vision beyond fairness, in particular ethical considerations on creating and employing fashion datasets, and how computer vision supports cultural representation and might avoid algorithmic cultural appropriation.
Memory Planning for Deep Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

We study memory allocation patterns in DNNs during inference, in the context of large-scale systems. We observe that such memory allocation patterns, in the context of multi-threading, are subject to high latencies, due to \texttt{mutex} contention in the system memory allocator. Latencies incurred due to such \texttt{mutex} contention produce undesirable bottlenecks in user-facing services. Thus, we propose a "memorization" based technique, \texttt{MemoMalloc}, for optimizing overall latency, with only moderate increases in peak memory usage. Specifically, our technique consists of a runtime component, which captures all allocations and uniquely associates them with their high-level source operation, and a static analysis component, which constructs an efficient allocation "plan". We present an implementation of \texttt{MemoMalloc} in the PyTorch deep learning framework and evaluate memory consumption and execution performance on a wide range of DNN architectures. We find that \texttt{MemoMalloc} outperforms state-of-the-art general purpose memory allocators, with respect to DNN inference latency, by as much as 40\%.
Render-in-the-loop aerial robotics simulator: Case Study on Yield Estimation in Indoor Agriculture
Venue: arXiv cs.RO 2022
Year: 2022

Inspired by recent promising results in sim-to-real transfer in deep learning we built a realistic simulation environment combining a Robot Operating System (ROS)-compatible physics simulator (Gazebo) with Cycles, the realistic production rendering engine from Blender. The proposed simulator pipeline allows us to simulate near-realistic RGB-D images. To showcase the capabilities of the simulator pipeline we propose a case study that focuses on indoor robotic farming. We developed a solution for sweet pepper yield estimation task. Our approach to yield estimation starts with aerial robotics control and trajectory planning, combined with deep learning-based pepper detection, and a clustering approach for counting fruit. The results of this case study show that we can combine real time dynamic simulation with near realistic rendering capabilities to simulate complex robotic systems.
Mode Recognition by Shape Morphing for Maxwell's Eigenvalue Problem
Venue: arXiv cs.CE 2022
Year: 2022

In electrical engineering, for example during the design of superconducting radio-frequency cavities, eigenmodes must be identified based on their field patterns. This allows to understand the working principle, optimize the performance of a device and distinguish desired from parasitic modes. For cavities with simple shapes, the eigenmodes are easily classified according to the number of nodes and antinodes in each direction as is obvious from analytical formulae. For cavities with complicated shapes, the eigenmodes are determined numerically. Thereby, the classification is cumbersome, if not impossible. In this paper, we propose a new recognition method by morphing the cavity geometry to a pillbox and tracking its eigenmodes during the deformation.
DeepNet: Scaling Transformers to 1,000 Layers
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, we propose a simple yet effective method to stabilize extremely deep Transformers. Specifically, we introduce a new normalization function (DeepNorm) to modify the residual connection in Transformer, accompanying with theoretically derived initialization. In-depth theoretical analysis shows that model updates can be bounded in a stable way. The proposed method combines the best of two worlds, i.e., good performance of Post-LN and stable training of Pre-LN, making DeepNorm a preferred alternative. We successfully scale Transformers up to 1,000 layers (i.e., 2,500 attention and feed-forward network sublayers) without difficulty, which is one order of magnitude deeper than previous deep Transformers. Remarkably, on a multilingual benchmark with 7,482 translation directions, our 200-layer model with 3.2B parameters significantly outperforms the 48-layer state-of-the-art model with 12B parameters by 5 BLEU points, which indicates a promising scaling direction.
Learning Intermediate Representations using Graph Neural Networks for NUMA and Prefetchers Optimization
Venue: arXiv cs.DC 2022
Year: 2022

There is a large space of NUMA and hardware prefetcher configurations that can significantly impact the performance of an application. Previous studies have demonstrated how a model can automatically select configurations based on the dynamic properties of the code to achieve speedups. This paper demonstrates how the static Intermediate Representation (IR) of the code can guide NUMA/prefetcher optimizations without the prohibitive cost of performance profiling. We propose a method to create a comprehensive dataset that includes a diverse set of intermediate representations along with optimum configurations. We then apply a graph neural network model in order to validate this dataset. We show that our static intermediate representation based model achieves 80% of the performance gains provided by expensive dynamic performance profiling based strategies. We further develop a hybrid model that uses both static and dynamic information. Our hybrid model achieves the same gains as the dynamic models but at a reduced cost by only profiling 30% of the programs.
Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale
Venue: arXiv cs.CL 2022
Year: 2022

We introduce Transformer Grammars (TGs), a novel class of Transformer language models that combine (i) the expressive power, scalability, and strong performance of Transformers and (ii) recursive syntactic compositions, which here are implemented through a special attention mask and deterministic transformation of the linearized tree. We find that TGs outperform various strong baselines on sentence-level language modeling perplexity, as well as on multiple syntax-sensitive language modeling evaluation metrics. Additionally, we find that the recursive syntactic composition bottleneck which represents each sentence as a single vector harms perplexity on document-level language modeling, providing evidence that a different kind of memory mechanism -- one that is independent of composed syntactic representations -- plays an important role in current successful models of long text.
Relaxed virtual memory in Armv8-A (extended version)
Venue: arXiv cs.AR 2022
Year: 2022

Virtual memory is an essential mechanism for enforcing security boundaries, but its relaxed-memory concurrency semantics has not previously been investigated in detail. The concurrent systems code managing virtual memory has been left on an entirely informal basis, and OS and hypervisor verification has had to make major simplifying assumptions. We explore the design space for relaxed virtual memory semantics in the Armv8-A architecture, to support future system-software verification. We identify many design questions, in discussion with Arm; develop a test suite, including use cases from the pKVM production hypervisor under development by Google; delimit the design space with axiomatic-style concurrency models; prove that under simple stable configurations our architectural model collapses to previous "user" models; develop tooling to compute allowed behaviours in the model integrated with the full Armv8-A ISA semantics; and develop a hardware test harness. This lays out some of the main issues in relaxed virtual memory bringing these security-critical systems phenomena into the domain of programming-language semantics and verification with foundational architecture semantics. This document is an extended version of a paper in ESOP 2022, with additional explanation and examples in the main body, and appendices detailing our litmus tests, models, proofs, and test results.
Subspace-Based Pilot Decontamination in User-Centric Scalable Cell-Free Wireless Networks
Venue: arXiv cs.IT 2022
Year: 2022

We consider a cell-free wireless system operated in Time Division Duplex (TDD) mode with user-centric clusters of remote radio units (RUs). Since the uplink pilot dimensions per channel coherence slot is limited, co-pilot users might incur mutual pilot contamination. In the current literature, it is assumed that the long-term statistical knowledge of all user channels is available. This enables Minimum Mean-Square Error channel estimation or simplified dominant subspace projection, which achieves significant pilot decontamination under certain assumptions on the channel covariance matrices. However, estimating the channel covariance matrix or even just its dominant subspace at all RUs forming a user cluster is not an easy task. In fact, if not properly designed, a piloting scheme for such long-term statistics estimation will also be subject to the contamination problem. In this paper, we propose a new channel subspace estimation scheme explicitly designed for cell-free wireless networks. Our scheme is based on 1) a sounding reference signal (SRS) using latin squares wideband frequency hopping, and 2) a subspace estimation method based on robust Principal Component Analysis (R-PCA). The SRS hopping scheme ensures that for any user and any RU participating in its cluster, only a few pilot measurements will contain strong co-pilot interference. These few heavily contaminated measurements are (implicitly) eliminated by R-PCA, which is designed to regularize the estimation and discount the ``outlier'' measurements. Our simulation results show that the proposed scheme achieves almost perfect subspace knowledge, which in turns yields system performance very close to that with ideal channel state information, thus essentially solving the problem of pilot contamination in cell-free user-centric TDD wireless networks.
Task-grasping from human demonstration
Venue: arXiv cs.RO 2022
Year: 2022

A challenge in robot grasping is to achieve task-grasping which is to select a grasp that is advantageous to the success of tasks before and after grasps. One of the frameworks to address this difficulty is Learning-from-Observation (LfO), which obtains various hints from human demonstrations. This paper solves three issues in the grasping skills in the LfO framework: 1) how to functionally mimic human-demonstrated grasps to robots with limited grasp capability, 2) how to coordinate grasp skills with reaching body mimicking, 3) how to robustly perform grasps under object pose and shape uncertainty. A deep reinforcement learning using contact-web based rewards and domain randomization of approach directions is proposed to achieve such robust mimicked grasping skills. Experiment results show that the trained grasping skills can be applied in an LfO system and executed on a real robot. In addition, it is shown that the trained skill is robust to errors in the object pose and to the uncertainty of the object shape and can be combined with various reach-coordination.
Knock, knock. Who's there? -- Identifying football player jersey numbers with synthetic data
Venue: arXiv cs.CV 2022
Year: 2022

Automatic player identification is an essential and complex task in sports video analysis. Different strategies have been devised over the years, but identification based on jersey numbers is one of the most common approaches given its versatility and relative simplicity. However, automatic detection of jersey numbers is still challenging due to changing camera angles, low video resolution, small object size in wide-range shots and transient changes in the player's posture and movement. In this paper we present a novel approach for jersey number identification in a small, highly imbalanced dataset from the Seattle Seahawks practice videos. Our results indicate that simple models can achieve an acceptable performance on the jersey number detection task and that synthetic data can improve the performance dramatically (accuracy increase of ~9% overall, ~18% on low frequency numbers) making our approach achieve state of the art results.
There is a Time and Place for Reasoning Beyond the Image
Venue: arXiv cs.CV 2022
Year: 2022

Images are often more significant than only the pixels to human eyes, as we can infer, associate, and reason with contextual information from other sources to establish a more complete picture. For example, in Figure 1, we can find a way to identify the news articles related to the picture through segment-wise understandings of the signs, the buildings, the crowds, and more. This reasoning could provide the time and place the image was taken, which will help us in subsequent tasks, such as automatic storyline construction, correction of image source in intended effect photographs, and upper-stream processing such as image clustering for certain location or time. In this work, we formulate this problem and introduce TARA: a dataset with 16k images with their associated news, time, and location, automatically extracted from New York Times, and an additional 61k examples as distant supervision from WIT. On top of the extractions, we present a crowdsourced subset in which we believe it is possible to find the images' spatio-temporal information for evaluation purpose. We show that there exists a $70\%$ gap between a state-of-the-art joint model and human performance, which is slightly filled by our proposed model that uses segment-wise reasoning, motivating higher-level vision-language joint models that can conduct open-ended reasoning with world knowledge. The data and code are publicly available at https://github.com/zeyofu/TARA.
Instance-aware multi-object self-supervision for monocular depth prediction
Venue: arXiv cs.CV 2022
Year: 2022

This paper proposes a self-supervised monocular image-to-depth prediction framework that is trained with an end-to-end photometric loss that handles not only 6-DOF camera motion but also 6-DOF moving object instances. Self-supervision is performed by warping the images across a video sequence using depth and scene motion including object instances. One novelty of the proposed method is the use of the multi-head attention of the transformer network that matches moving objects across time and models their interaction and dynamics. This enables accurate and robust pose estimation for each object instance. Most image-to-depth predication frameworks make the assumption of rigid scenes, which largely degrades their performance with respect to dynamic objects. Only a few SOTA papers have accounted for dynamic objects. The proposed method is shown to outperform these methods on standard benchmarks and the impact of the dynamic motion on these benchmarks is exposed. Furthermore, the proposed image-to-depth prediction framework is also shown to be competitive with SOTA video-to-depth prediction frameworks.
Computerization of Clinical Pathways: A Literature Review and Directions for Future Research
Venue: arXiv cs.AI 2022
Year: 2022

Clinical Pathways (CP) are medical management plans developed to standardize patient treatment activities, optimize resource usage, reduce expenses, and improve the quality of healthcare services. Most CPs currently in use are paper-based documents (i.e., not computerized). CP computerization has been an active research topic since the inception of CP use in hospitals. This literature review research aims to examine studies that focused on CP computerization and offers recommendations for future research in this important research area. Some critical research suggestions include centralizing computerized CPs in Healthcare Information Systems (HIS), CP term standardization using international medical terminology systems, developing a global CP-specific digital coding system, creating a unified CP meta-ontology, developing independent Clinical Pathway Management Systems (CPMS), and supporting CPMSs with machine learning sub-systems.
GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we study the problem of learning Graph Neural Networks (GNNs) with Differential Privacy (DP). We propose a novel differentially private GNN based on Aggregation Perturbation (GAP), which adds stochastic noise to the GNN's aggregation function to statistically obfuscate the presence of a single edge (edge-level privacy) or a single node and all its adjacent edges (node-level privacy). Tailored to the specifics of private learning, GAP's new architecture is composed of three separate modules: (i) the encoder module, where we learn private node embeddings without relying on the edge information; (ii) the aggregation module, where we compute noisy aggregated node embeddings based on the graph structure; and (iii) the classification module, where we train a neural network on the private aggregations for node classification without further querying the graph edges. GAP's major advantage over previous approaches is that it can benefit from multi-hop neighborhood aggregations, and guarantees both edge-level and node-level DP not only for training, but also at inference with no additional costs beyond the training's privacy budget. We analyze GAP's formal privacy guarantees using Rényi DP and conduct empirical experiments over three real-world graph datasets. We demonstrate that GAP offers significantly better accuracy-privacy trade-offs than state-of-the-art DP-GNN approaches and naive MLP-based baselines. Our code is publicly available at https://github.com/sisaman/GAP.