OpenRaiser/Intern-Atlas huggingface dataset:
Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training
Venue: arXiv cs.CV 2021
Year: 2021

Translating e-commercial product descriptions, a.k.a product-oriented machine translation (PMT), is essential to serve e-shoppers all over the world. However, due to the domain specialty, the PMT task is more challenging than traditional machine translation problems. Firstly, there are many specialized jargons in the product description, which are ambiguous to translate without the product image. Secondly, product descriptions are related to the image in more complicated ways than standard image descriptions, involving various visual aspects such as objects, shapes, colors or even subjective styles. Moreover, existing PMT datasets are small in scale to support the research. In this paper, we first construct a large-scale bilingual product description dataset called Fashion-MMT, which contains over 114k noisy and 40k manually cleaned description translations with multiple product images. To effectively learn semantic alignments among product images and bilingual texts in translation, we design a unified product-oriented cross-modal cross-lingual model (\upoc~) for pre-training and fine-tuning. Experiments on the Fashion-MMT and Multi30k datasets show that our model significantly outperforms the state-of-the-art models even pre-trained on the same dataset. It is also shown to benefit more from large-scale noisy data to improve the translation quality. We will release the dataset and codes at https://github.com/syuqings/Fashion-MMT.
Semantic Scene Segmentation for Robotics Applications
Venue: arXiv cs.CV 2021
Year: 2021

Semantic scene segmentation plays a critical role in a wide range of robotics applications, e.g., autonomous navigation. These applications are accompanied by specific computational restrictions, e.g., operation on low-power GPUs, at sufficient speed, and also for high-resolution input. Existing state-of-the-art segmentation models provide evaluation results under different setups and mainly considering high-power GPUs. In this paper, we investigate the behavior of the most successful semantic scene segmentation models, in terms of deployment (inference) speed, under various setups (GPUs, input sizes, etc.) in the context of robotics applications. The target of this work is to provide a comparative study of current state-of-the-art segmentation models so as to select the most compliant with the robotics applications requirements.
Lizard: A Large-Scale Dataset for Colonic Nuclear Instance Segmentation and Classification
Venue: arXiv cs.CV 2021
Year: 2021

The development of deep segmentation models for computational pathology (CPath) can help foster the investigation of interpretable morphological biomarkers. Yet, there is a major bottleneck in the success of such approaches because supervised deep learning models require an abundance of accurately labelled data. This issue is exacerbated in the field of CPath because the generation of detailed annotations usually demands the input of a pathologist to be able to distinguish between different tissue constructs and nuclei. Manually labelling nuclei may not be a feasible approach for collecting large-scale annotated datasets, especially when a single image region can contain thousands of different cells. However, solely relying on automatic generation of annotations will limit the accuracy and reliability of ground truth. Therefore, to help overcome the above challenges, we propose a multi-stage annotation pipeline to enable the collection of large-scale datasets for histology image analysis, with pathologist-in-the-loop refinement steps. Using this pipeline, we generate the largest known nuclear instance segmentation and classification dataset, containing nearly half a million labelled nuclei in H&E stained colon tissue. We have released the dataset and encourage the research community to utilise it to drive forward the development of downstream cell-based models in CPath.
Revizor: A Data-Driven Approach to Automate Frequent Code Changes Based on Graph Matching
Venue: arXiv cs.SE 2021
Year: 2021

Many code changes that developers make in their projects are repeated and constitute recurrent change patterns. It is of interest to collect such patterns from the version history of open-source repositories and suggest the most useful of them as quick fixes. In this paper, we present Revizor - a tool aimed to build custom plugins for PyCharm, a popular Python IDE. A Revizor-based plugin can take change patterns and highlight potential places for their application in the developer's code editor. If the developer accepts the quick fix, the plugin automatically performs the edit. Our approach uses a graph-based representation of code changes, which allows it to support complex distributed code patterns. Experienced developers have also rated the usability and the performance of such Revizor-based plugin positively. The source code of the tool and test plugin prototype are available on GitHub: https://github.com/JetBrains-Research/revizor. A demonstration video with a short tool description can be found on YouTube: https://youtu.be/5eLs14nco7E.
RefactorInsight: Enhancing IDE Representation of Changes in Git with Refactorings Information
Venue: arXiv cs.SE 2021
Year: 2021

Inspection of code changes is a time-consuming task that constitutes a big part of everyday work of software engineers. Existing IDEs provide little information about the semantics of code changes within the file editor view. Therefore developers have to track changes across multiple files, which is a hard task with large codebases. In this paper, we present RefactorInsight, a plugin for IntelliJ IDEA that introduces a smart diff for code changes in Java and Kotlin where refactorings are auto-folded and provided with their description, thus allowing users to focus on changes that modify the code behavior like bug fixes and new features. RefactorInsight supports three usage scenarios: viewing smart diffs with auto-folded refactorings and hints, inspecting refactorings in pull requests and in any specific commit in the project change history, and exploring the refactoring history of methods and classes. The evaluation shows that commit processing time is acceptable: on median it is less than 0.2 seconds, which delay does not disrupt developers' IDE workflows. RefactorInsight is available at https://github.com/JetBrains-Research/RefactorInsight. The demonstration video is available at https://youtu.be/-6L2AKQ66nA.
RepliComment: Identifying Clones in Code Comments
Venue: arXiv cs.SE 2021
Year: 2021

Code comments are the primary means to document implementation and facilitate program comprehension. Thus, their quality should be a primary concern to improve program maintenance. While much effort has been dedicated to detecting bad smells, such as clones in code, little work has focused on comments. In this paper we present our solution to detect clones in comments that developers should fix. RepliComment can automatically analyze Java projects and report instances of copy-and-paste errors in comments, and can point developers to which comments should be fixed. Moreover, it can report when clones are signs of poorly written comments. Developers should fix these instances too in order to improve the quality of the code documentation. Our evaluation of 10 well-known open source Java projects identified over 11K instances of comment clones, and over 1,300 of them are potentially critical. We improve on our own previous work, which could only find 36 issues in the same dataset. Our manual inspection of 412 issues reported by RepliComment reveals that it achieves a precision of 79% in reporting critical comment clones. The manual inspection of 200 additional comment clones that RepliComment filters out as being legitimate, could not evince any false negative.
Multi-Stage Threat Modelling and Security Monitoring in 5GCN
Venue: arXiv cs.CR 2021
Year: 2021

The fifth generation of mobile networks (5G) promises a range of new capabilities including higher data rates and more connected users. To support the new capabilities and use cases the 5G Core Network (5GCN) will be dynamic and reconfigurable in nature to deal with demand. It is these improvements which also introduce issues for traditional security monitoring methods and techniques which need to adapt to the new network architecture. The increased data volumes and dynamic network architecture mean an approach is required to focus security monitoring resources where it is most needed and react to network changes in real time. When considering multi-stage threat scenarios a coordinated, centralised approach to security monitoring is required for the early detection of attacks which may affect different parts of the network. In this chapter we identify potential solutions for overcoming these challenges which begins by identifying the threats to the 5G networks to determine suitable security monitoring placement in the 5GCN.
Pagurus: Eliminating Cold Startup in Serverless Computing with Inter-Action Container Sharing
Venue: arXiv cs.DC 2021
Year: 2021

Serverless computing provides fine-grain resource sharing between Cloud tenants through containers. Each function invocation (action) runs in an individual container. When there is not an already started container for a user function, a new container has to be created for it. However, the long cold startup time of a container results in the long response latency of the action. Our investigation shows that the containers for some user actions share most of the software packages. If an action that requires a new container can ``borrow'' a similar warm container from other actions, the long cold startup can be eliminated. Based on the above finding, we propose Pagurus, a runtime container management system for eliminating the cold startup in serverless computing. Pagurus is comprised of an inter-action container scheduler and an intra-action container scheduler for each action. The inter-action container scheduler schedules shared containers among actions. The intra-action container scheduler deals with the management of the container lifecycle. Our experimental results show that Pagurus effectively eliminates the time-consuming container cold startup. An action may start to run in 10ms with Pagurus, even if there is not warm container for it.
FogROS: An Adaptive Framework for Automating Fog Robotics Deployment
Venue: arXiv cs.RO 2021
Year: 2021

As many robot automation applications increasingly rely on multi-core processing or deep-learning models, cloud computing is becoming an attractive and economically viable resource for systems that do not contain high computing power onboard. Despite its immense computing capacity, it is often underused by the robotics and automation community due to lack of expertise in cloud computing and cloud-based infrastructure. Fog Robotics balances computing and data between cloud edge devices. We propose a software framework, FogROS, as an extension of the Robot Operating System (ROS), the de-facto standard for creating robot automation applications and components. It allows researchers to deploy components of their software to the cloud with minimal effort, and correspondingly gain access to additional computing cores, GPUs, FPGAs, and TPUs, as well as predeployed software made available by other researchers. FogROS allows a researcher to specify which components of their software will be deployed to the cloud and to what type of computing hardware. We evaluate FogROS on 3 examples: (1) simultaneous localization and mapping (ORB-SLAM2), (2) Dexterity Network (Dex-Net) GPU-based grasp planning, and (3) multi-core motion planning using a 96-core cloud-based server. In all three examples, a component is deployed to the cloud and accelerated with a small change in system launch configuration, while incurring additional latency of 1.2 s, 0.6 s, and 0.5 s due to network communication, the computation speed is improved by 2.6x, 6.0x and 34.2x, respectively. Code, videos, and supplementary material can be found at https://github.com/BerkeleyAutomation/FogROS.
CDCGen: Cross-Domain Conditional Generation via Normalizing Flows and Adversarial Training
Venue: arXiv cs.CV 2021
Year: 2021

How to generate conditional synthetic data for a domain without utilizing information about its labels/attributes? Our work presents a solution to the above question. We propose a transfer learning-based framework utilizing normalizing flows, coupled with both maximum-likelihood and adversarial training. We model a source domain (labels available) and a target domain (labels unavailable) with individual normalizing flows, and perform domain alignment to a common latent space using adversarial discriminators. Due to the invertible property of flow models, the mapping has exact cycle consistency. We also learn the joint distribution of the data samples and attributes in the source domain by employing an encoder to map attributes to the latent space via adversarial training. During the synthesis phase, given any combination of attributes, our method can generate synthetic samples conditioned on them in the target domain. Empirical studies confirm the effectiveness of our method on benchmarked datasets. We envision our method to be particularly useful for synthetic data generation in label-scarce systems by generating non-trivial augmentations via attribute transformations. These synthetic samples will introduce more entropy into the label-scarce domain than their geometric and photometric transformation counterparts, helpful for robust downstream tasks.
Machine Learning for Sensor Transducer Conversion Routines
Venue: arXiv cs.LG 2021
Year: 2021

Sensors with digital outputs require software conversion routines to transform the unitless analogue-to-digital converter samples to physical quantities with correct units. These conversion routines are computationally complex given the limited computational resources of low-power embedded systems. This article presents a set of machine learning methods to learn new, less-complex conversion routines that do not sacrifice accuracy for the BME680 environmental sensor. We present a Pareto analysis of the tradeoff between accuracy and computational overhead for the models and models that reduce the computational overhead of the existing industry-standard conversion routines for temperature, pressure, and humidity by 62%, 71 %, and 18 % respectively. The corresponding RMS errors are 0.0114 degrees C, 0.0280 KPa, and 0.0337 %. These results show that machine learning methods for learning conversion routines can produce conversion routines with reduced computational overhead which maintain good accuracy.
Unsupervised Reservoir Computing for Solving Ordinary Differential Equations
Venue: arXiv cs.LG 2021
Year: 2021

There is a wave of interest in using unsupervised neural networks for solving differential equations. The existing methods are based on feed-forward networks, {while} recurrent neural network differential equation solvers have not yet been reported. We introduce an unsupervised reservoir computing (RC), an echo-state recurrent neural network capable of discovering approximate solutions that satisfy ordinary differential equations (ODEs). We suggest an approach to calculate time derivatives of recurrent neural network outputs without using backpropagation. The internal weights of an RC are fixed, while only a linear output layer is trained, yielding efficient training. However, RC performance strongly depends on finding the optimal hyper-parameters, which is a computationally expensive process. We use Bayesian optimization to efficiently discover optimal sets in a high-dimensional hyper-parameter space and numerically show that one set is robust and can be used to solve an ODE for different initial conditions and time ranges. A closed-form formula for the optimal output weights is derived to solve first order linear equations in a backpropagation-free learning process. We extend the RC approach by solving nonlinear system of ODEs using a hybrid optimization method consisting of gradient descent and Bayesian optimization. Evaluation of linear and nonlinear systems of equations demonstrates the efficiency of the RC ODE solver.
Model-based Decision Making with Imagination for Autonomous Parking
Venue: arXiv cs.RO 2021
Year: 2021

Autonomous parking technology is a key concept within autonomous driving research. This paper will propose an imaginative autonomous parking algorithm to solve issues concerned with parking. The proposed algorithm consists of three parts: an imaginative model for anticipating results before parking, an improved rapid-exploring random tree (RRT) for planning a feasible trajectory from a given start point to a parking lot, and a path smoothing module for optimizing the efficiency of parking tasks. Our algorithm is based on a real kinematic vehicle model; which makes it more suitable for algorithm application on real autonomous cars. Furthermore, due to the introduction of the imagination mechanism, the processing speed of our algorithm is ten times faster than that of traditional methods, permitting the realization of real-time planning simultaneously. In order to evaluate the algorithm's effectiveness, we have compared our algorithm with traditional RRT, within three different parking scenarios. Ultimately, results show that our algorithm is more stable than traditional RRT and performs better in terms of efficiency and quality.
Visualizing JIT Compiler Graphs
Venue: arXiv cs.PL 2021
Year: 2021

Just-in-time (JIT) compilers are used by many modern programming systems in order to improve performance. Bugs in JIT compilers provide exploitable security vulnerabilities and debugging them is difficult as they are large, complex, and dynamic. Current debugging and visualization tools deal with static code and are not suitable in this domain. We describe a new approach for simplifying the large and complex intermediate representation, generated by a JIT compiler and visualize it with a metro map metaphor to aid developers in debugging.
A Riemannian Framework for Analysis of Human Body Surface
Venue: arXiv cs.CV 2021
Year: 2021

We propose a novel framework for comparing 3D human shapes under the change of shape and pose. This problem is challenging since 3D human shapes vary significantly across subjects and body postures. We solve this problem by using a Riemannian approach. Our core contribution is the mapping of the human body surface to the space of metrics and normals. We equip this space with a family of Riemannian metrics, called Ebin (or DeWitt) metrics. We treat a human body surface as a point in a "shape space" equipped with a family of Riemannian metrics. The family of metrics is invariant under rigid motions and reparametrizations; hence it induces a metric on the "shape space" of surfaces. Using the alignment of human bodies with a given template, we show that this family of metrics allows us to distinguish the changes in shape and pose. The proposed framework has several advantages. First, we define a family of metrics with desired invariance properties for the comparison of human shape. Second, we present an efficient framework to compute geodesic paths between human shape given the chosen metric. Third, this framework provides some basic tools for statistical shape analysis of human body surfaces. Finally, we demonstrate the utility of the proposed framework in pose and shape retrieval of human body.
Bandwidth Allocation and Service Differentiation in D2D Wireless Networks
Venue: arXiv cs.IT 2021
Year: 2021

Inspired by a new feature in 5G NR called bandwidth part (BWP), this paper presents a bandwidth allocation (BA) model that allows one to adapt the bandwidth allocated to users depending on their data rate needs. Specifically, in adaptive BA, a wide bandwidth is divided into chunks of smaller bandwidths and the number of bandwidth chunks allocated to a user depends on its needs or type. Although BWP in 5G NR mandates allocation of a set of contiguous bandwidth chunks, our BA model also allows other assumptions on chunk allocation such as the allocation of any set of bandwidth chunks, as in, e.g., LTE resource allocation, where chunks are selected uniformly at random. The BA model studied here is probabilistic in that the user locations are assumed to form a realization of a Poisson point process and each user decides independently to be of a certain type with some probability. This model allows one to quantify spectrum sharing and service differentiation in this context, namely to predict what performance a user gets depending on its type as well as the overall performance. This is based on exact representations of key performance metrics for each user type, namely its success probability, the meta distribution of its signal-to-interference ratio, and its Shannon throughput. We show that, surprisingly, the higher traffic variability stemming from adaptive BA is beneficial: when comparing two networks using adaptive BA and having the same mean signal and the same mean interference powers, the network with higher traffic variability performs better for all these performance metrics. With respect to Shannon throughput, we observe that our BA model is roughly egalitarian per Hertz and leads to a linear service differentiation in aggregated throughput value.
A Class of Trees Having Near-Best Balance
Venue: arXiv cs.DM 2021
Year: 2021

Full binary trees naturally represent commutative non-associative products. There are many important examples of these products: finite-precision floating-point addition and NAND gates, among others. Balance in such a tree is highly desirable for efficiency in calculation. The best balance is attained with a divide-and-conquer approach. However, this may not be the optimal solution, since the success of many calculations is dependent on the grouping and ordering of the calculation, for reasons ranging from the avoidance of rounding error, to calculating with varying precision, to the placement of calculation within a heterogeneous system. We introduce a new class of computational trees having near-best balance in terms of the Colless index from mathematical phylogenetics. These trees are easily constructed from the binary decomposition of the number of terms in the problem. They also permit much more flexibility than the optimally balanced divide-and-conquer trees. This gives needed freedom in the grouping and ordering of calculation, and allows intelligent efficiency trade-offs.
Improving Object Detection and Attribute Recognition by Feature Entanglement Reduction
Venue: arXiv cs.CV 2021
Year: 2021

We explore object detection with two attributes: color and material. The task aims to simultaneously detect objects and infer their color and material. A straight-forward approach is to add attribute heads at the very end of a usual object detection pipeline. However, we observe that the two goals are in conflict: Object detection should be attribute-independent and attributes be largely object-independent. Features computed by a standard detection network entangle the category and attribute features; we disentangle them by the use of a two-stream model where the category and attribute features are computed independently but the classification heads share Regions of Interest (RoIs). Compared with a traditional single-stream model, our model shows significant improvements over VG-20, a subset of Visual Genome, on both supervised and attribute transfer tasks.