OpenRaiser/Intern-Atlas huggingface dataset:
HoW-3D: Holistic 3D Wireframe Perception from a Single Image
Venue: arXiv cs.CV 2022
Year: 2022

This paper studies the problem of holistic 3D wireframe perception (HoW-3D), a new task of perceiving both the visible 3D wireframes and the invisible ones from single-view 2D images. As the non-front surfaces of an object cannot be directly observed in a single view, estimating the non-line-of-sight (NLOS) geometries in HoW-3D is a fundamentally challenging problem and remains open in computer vision. We study the problem of HoW-3D by proposing an ABC-HoW benchmark, which is created on top of CAD models sourced from the ABC-dataset with 12k single-view images and the corresponding holistic 3D wireframe models. With our large-scale ABC-HoW benchmark available, we present a novel Deep Spatial Gestalt (DSG) model to learn the visible junctions and line segments as the basis and then infer the NLOS 3D structures from the visible cues by following the Gestalt principles of human vision systems. In our experiments, we demonstrate that our DSG model performs very well in inferring the holistic 3D wireframes from single-view images. Compared with the strong baseline methods, our DSG model outperforms the previous wireframe detectors in detecting the invisible line geometry in single-view images and is even very competitive with prior arts that take high-fidelity PointCloud as inputs on reconstructing 3D wireframes.
Automatic Controlling Fish Feeding Machine using Feature Extraction of Nutriment and Ripple Behavior
Venue: arXiv cs.CV 2022
Year: 2022

Controlling fish feeding machine is challenging problem because experienced fishermen can adequately control based on assumption. To build robust method for reasonable application, we propose automatic controlling fish feeding machine based on computer vision using combination of counting nutriments and estimating ripple behavior using regression and textural feature, respectively. To count number of nutriments, we apply object detection and tracking methods to acknowledge the nutriments moving to sea surface. Recently, object tracking is active research and challenging problem in computer vision. Unfortunately, the robust tracking method for multiple small objects with dense and complex relationships is unsolved problem in aquaculture field with more appearance creatures. Based on the number of nutriments and ripple behavior, we can control fish feeding machine which consistently performs well in real environment. Proposed method presents the agreement for automatic controlling fish feeding by the activation graphs and textural feature of ripple behavior. Our tracking method can precisely track the nutriments in next frame comparing with other methods. Based on computational time, proposed method reaches 3.86 fps while other methods spend lower than 1.93 fps. Quantitative evaluation can promise that proposed method is valuable for aquaculture fish farm with widely applied to real environment.
Acceleration of Subspace Learning Machine via Particle Swarm Optimization and Parallel Processing
Venue: arXiv cs.LG 2022
Year: 2022

Built upon the decision tree (DT) classification and regression idea, the subspace learning machine (SLM) has been recently proposed to offer higher performance in general classification and regression tasks. Its performance improvement is reached at the expense of higher computational complexity. In this work, we investigate two ways to accelerate SLM. First, we adopt the particle swarm optimization (PSO) algorithm to speed up the search of a discriminant dimension that is expressed as a linear combination of current dimensions. The search of optimal weights in the linear combination is computationally heavy. It is accomplished by probabilistic search in original SLM. The acceleration of SLM by PSO requires 10-20 times fewer iterations. Second, we leverage parallel processing in the SLM implementation. Experimental results show that the accelerated SLM method achieves a speed up factor of 577 in training time while maintaining comparable classification/regression performance of original SLM.
RIS-Aided Communications over Dirty MAC: Capacity Region and Outage Probability
Venue: arXiv cs.IT 2022
Year: 2022

We characterize the capacity region of a two-user multiple access channel (MAC) in a reconfigurable intelligent surface (RIS)-aided communication system with side information (SI) at the transmitters. We consider two uplink communication scenarios: (i) a double dirty MAC where the interferences are known non-causally to both users; and (ii) a single dirty MAC model where only one of the users knows the interference and the other one is not aware of the interference. Considering that each of the users is assisted by a different RIS with Mi elements, we derive closed-form expressions for the capacity region and the outage probability (OP) for both system models. Results show that the use of RISs is beneficial to extend the capacity region and improve the OP in both scenarios.
A Drift Handling Approach for Self-Adaptive ML Software in Scalable Industrial Processes
Venue: arXiv cs.SE 2022
Year: 2022

Most industrial processes in real-world manufacturing applications are characterized by the scalability property, which requires an automated strategy to self-adapt machine learning (ML) software systems to the new conditions. In this paper, we investigate an Electroslag Remelting (ESR) use case process from the Uddeholms AB steel company. The use case involves predicting the minimum pressure value for a vacuum pumping event. Taking into account the long time required to collect new records and efficiently integrate the new machines with the built ML software system. Additionally, to accommodate the changes and satisfy the non-functional requirement of the software system, namely adaptability, we propose an automated and adaptive approach based on a drift handling technique called importance weighting. The aim is to address the problem of adding a new furnace to production and enable the adaptability attribute of the ML software. The overall results demonstrate the improvements in ML software performance achieved by implementing the proposed approach over the classical non-adaptive approach.
Analysis of impact of emotions on target speech extraction and speech separation
Venue: arXiv cs.SD 2022
Year: 2022

Recently, the performance of blind speech separation (BSS) and target speech extraction (TSE) has greatly progressed. Most works, however, focus on relatively well-controlled conditions using, e.g., read speech. The performance may degrade in more realistic situations. One of the factors causing such degradation may be intrinsic speaker variability, such as emotions, occurring commonly in realistic speech. In this paper, we investigate the influence of emotions on TSE and BSS. We create a new test dataset of emotional mixtures for the evaluation of TSE and BSS. This dataset combines LibriSpeech and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). Through controlled experiments, we can analyze the impact of different emotions on the performance of BSS and TSE. We observe that BSS is relatively robust to emotions, while TSE, which requires identifying and extracting the speech of a target speaker, is much more sensitive to emotions. On comparative speaker verification experiments we show that identifying the target speaker may be particularly challenging when dealing with emotional speech. Using our findings, we outline potential future directions that could improve the robustness of BSS and TSE systems toward emotional speech.
Compressing Pre-trained Models of Code into 3 MB
Venue: arXiv cs.SE 2022
Year: 2022

Although large pre-trained models of code have delivered significant advancements in various code processing tasks, there is an impediment to the wide and fluent adoption of these powerful models in software developers' daily workflow: these large models consume hundreds of megabytes of memory and run slowly on personal devices, which causes problems in model deployment and greatly degrades the user experience. It motivates us to propose Compressor, a novel approach that can compress the pre-trained models of code into extremely small models with negligible performance sacrifice. Our proposed method formulates the design of tiny models as simplifying the pre-trained model architecture: searching for a significantly smaller model that follows an architectural design similar to the original pre-trained model. Compressor proposes a genetic algorithm (GA)-based strategy to guide the simplification process. Prior studies found that a model with higher computational cost tends to be more powerful. Inspired by this insight, the GA algorithm is designed to maximize a model's Giga floating-point operations (GFLOPs), an indicator of the model computational cost, to satisfy the constraint of the target model size. Then, we use the knowledge distillation technique to train the small model: unlabelled data is fed into the large model and the outputs are used as labels to train the small model. We evaluate Compressor with two state-of-the-art pre-trained models, i.e., CodeBERT and GraphCodeBERT, on two important tasks, i.e., vulnerability prediction and clone detection. We use our method to compress pre-trained models to a size (3 MB), which is 160$\times$ smaller than the original size. The results show that compressed CodeBERT and GraphCodeBERT are 4.31$\times$ and 4.15$\times$ faster than the original model at inference, respectively. More importantly, ...
Genetic algorithms for the resource-constrained project scheduling problem in aircraft heavy maintenance
Venue: arXiv cs.NE 2022
Year: 2022

Due to complex sets of interrelated activities in aircraft heavy maintenance (AHM), many airlines have to deal with substantial aircraft maintenance downtime. The scheduling problem in AHM is regarded as an NP-hard problem. Using exact algorithms can be time-consuming or even infeasible. This article proposes genetic algorithms for solving the resource-constrained project scheduling problem (RCPSP) in AHM. The objective of the study was to minimise the makespan of the maintenance plan. The proposed algorithms applied five heuristic dispatching rules to generate an initial population based on activity list formation. Resource allocation methods for RCPSPearliest start time (EST) and workgroup and earliest start time (WEST)-were used to evaluate the fitness value. The elitist and roulette wheel methods were applied in the selection process. The sequences of the activity lists were then iteratively improved by crossover and mutation operations. The results show that the proposed algorithms perform efficiently compared to the existing solutions in terms of computational time and resource allocation.
Determining HEDP Foams' Quality with Multi-View Deep Learning Classification
Venue: arXiv cs.CV 2022
Year: 2022

High energy density physics (HEDP) experiments commonly involve a dynamic wave-front propagating inside a low-density foam. This effect affects its density and hence, its transparency. A common problem in foam production is the creation of defective foams. Accurate information on their dimension and homogeneity is required to classify the foams' quality. Therefore, those parameters are being characterized using a 3D-measuring laser confocal microscope. For each foam, five images are taken: two 2D images representing the top and bottom surface foam planes and three images of side cross-sections from 3D scannings. An expert has to do the complicated, harsh, and exhausting work of manually classifying the foam's quality through the image set and only then determine whether the foam can be used in experiments or not. Currently, quality has two binary levels of normal vs. defective. At the same time, experts are commonly required to classify a sub-class of normal-defective, i.e., foams that are defective but might be sufficient for the needed experiment. This sub-class is problematic due to inconclusive judgment that is primarily intuitive. In this work, we present a novel state-of-the-art multi-view deep learning classification model that mimics the physicist's perspective by automatically determining the foams' quality classification and thus aids the expert. Our model achieved 86\% accuracy on upper and lower surface foam planes and 82\% on the entire set, suggesting interesting heuristics to the problem. A significant added value in this work is the ability to regress the foam quality instead of binary deduction and even explain the decision visually. The source code used in this work, as well as other relevant sources, are available at: https://github.com/Scientific-Computing-Lab-NRCN/Multi-View-Foams.git
Virtual Reality Assisted Human Perception in ADAS Development: a Munich 3D Model Study
Venue: arXiv cs.HC 2022
Year: 2022

As the development of autonomous driving (AD) and advanced driver assistance systems (ADAS) progresses, the relevance of the comfort of users is gaining increasing interest. It becomes significant to test and validate perceived comfort performance from the early phase of system development before driving on roads. Most of the present ADAS test procedures are not efficient in performing such comfort evaluation. One of the main challenges is to integrate high-quality, realistic and predictable virtual traffic scenarios into an ADAS testing framework that has physics-based sensors capable of sensing the virtual environment. In this paper, we present our development of a virtual reality based ADAS testing framework that enhances human perception evaluation. The main contribution relies on three aspects. First, we introduce our development of a large and high-quality (realism, structure, texture) 3D traffic model of the Munich city in Germany. Second, we optimize the 3D model for virtual reality purpose, and real-time capable for human-in-the-loop ADAS testing. Finally, the model is then integrated into an ADAS framework for testing and validating ADAS functionalities and perceived comfort performance. The developed framework components are presented with illustrative examples.
Vehicle-road Cooperative Simulation and 3D Visualization System
Venue: arXiv cs.RO 2022
Year: 2022

The safety of single-vehicle autonomous driving technology is limited due to the limits of perception capability of on-board sensors. In contrast, vehicle-road collaboration technology can overcome those limits and improve the traffic safety and efficiency, by expanding the sensing range, improving the perception accuracy, and reducing the response time. However, such a technology is still under development; it requires rigorous testing and verification methods to ensure the reliability and trustworthiness of the technology. In this thesis, we focus on three major tasks: (1) analyze the functional characteristics related to the scenarios of vehicle-road cooperations, highlightening the differences between vehicle-road cooperative systems and traditional single-vehicle autonomous driving systems; (2) refine and classifiy the functional characteristics of vehicle-road cooperative systems; (3) design and develop a simulation system, and provide a visual interface to facilitate development and analysis. The efficiency and effectiveness the proposed method are verfied by experiments.
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning has become the method of choice to tackle real-world problems in different domains, partly because of its ability to learn from data and achieve impressive performance on a wide range of applications. However, its success usually relies on two assumptions: (i) vast troves of labeled datasets are required for accurate model fitting, and (ii) training and testing data are independent and identically distributed. Its performance on unseen target domains, thus, is not guaranteed, especially when encountering out-of-distribution data at the adaptation stage. The performance drop on data in a target domain is a critical problem in deploying deep neural networks that are successfully trained on data in a source domain. Unsupervised domain adaptation (UDA) is proposed to counter this, by leveraging both labeled source domain data and unlabeled target domain data to carry out various tasks in the target domain. UDA has yielded promising results on natural image processing, video analysis, natural language processing, time-series data analysis, medical image analysis, etc. In this review, as a rapidly evolving topic, we provide a systematic comparison of its methods and applications. In addition, the connection of UDA with its closely related tasks, e.g., domain generalization and out-of-distribution detection, has also been discussed. Furthermore, deficiencies in current methods and possible promising directions are highlighted.
Encoding Integers and Rationals on Neuromorphic Computers using Virtual Neuron
Venue: arXiv cs.NE 2022
Year: 2022

Neuromorphic computers perform computations by emulating the human brain, and use extremely low power. They are expected to be indispensable for energy-efficient computing in the future. While they are primarily used in spiking neural network-based machine learning applications, neuromorphic computers are known to be Turing-complete, and thus, capable of general-purpose computation. However, to fully realize their potential for general-purpose, energy-efficient computing, it is important to devise efficient mechanisms for encoding numbers. Current encoding approaches have limited applicability and may not be suitable for general-purpose computation. In this paper, we present the virtual neuron as an encoding mechanism for integers and rational numbers. We evaluate the performance of the virtual neuron on physical and simulated neuromorphic hardware and show that it can perform an addition operation using 23 nJ of energy on average using a mixed-signal memristor-based neuromorphic processor. We also demonstrate its utility by using it in some of the mu-recursive functions, which are the building blocks of general-purpose computation.
CTI4AI: Threat Intelligence Generation and Sharing after Red Teaming AI Models
Venue: arXiv cs.CR 2022
Year: 2022

As the practicality of Artificial Intelligence (AI) and Machine Learning (ML) based techniques grow, there is an ever increasing threat of adversarial attacks. There is a need to red team this ecosystem to identify system vulnerabilities, potential threats, characterize properties that will enhance system robustness, and encourage the creation of effective defenses. A secondary need is to share this AI security threat intelligence between different stakeholders like, model developers, users, and AI/ML security professionals. In this paper, we create and describe a prototype system CTI4AI, to overcome the need to methodically identify and share AI/ML specific vulnerabilities and threat intelligence.
Identifying Source Code File Experts
Venue: arXiv cs.SE 2022
Year: 2022

In software development, the identification of source code file experts is an important task. Identifying these experts helps to improve software maintenance and evolution activities, such as developing new features, code reviews, and bug fixes. Although some studies have proposed repository mining techniques to automatically identify source code experts, there are still gaps in this area that can be explored. For example, investigating new variables related to source code knowledge and applying machine learning aiming to improve the performance of techniques to identify source code experts. The goal of this study is to investigate opportunities to improve the performance of existing techniques to recommend source code files experts. We built an oracle by collecting data from the development history and surveying developers of 113 software projects. Then, we use this oracle to: (i) analyze the correlation between measures extracted from the development history and the developers source code knowledge and (ii) investigate the use of machine learning classifiers by evaluating their performance in identifying source code files experts. First Authorship and Recency of Modification are the variables with the highest positive and negative correlations with source code knowledge, respectively. Machine learning classifiers outperformed the linear techniques (F-Measure = 71% to 73%) in the public dataset, but this advantage is not clear in the private dataset, with F-Measure ranging from 55% to 68% for the linear techniques and 58% to 67% for ML techniques. Overall, the linear techniques and the machine learning classifiers achieved similar performance, particularly if we analyze F-Measure. However, machine learning classifiers usually get higher precision while linear techniques obtained the highest recall values.
Multi-level Contrast Network for Wearables-based Joint Activity Segmentation and Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Human activity recognition (HAR) with wearables is promising research that can be widely adopted in many smart healthcare applications. In recent years, the deep learning-based HAR models have achieved impressive recognition performance. However, most HAR algorithms are susceptible to the multi-class windows problem that is essential yet rarely exploited. In this paper, we propose to relieve this challenging problem by introducing the segmentation technology into HAR, yielding joint activity segmentation and recognition. Especially, we introduce the Multi-Stage Temporal Convolutional Network (MS-TCN) architecture for sample-level activity prediction to joint segment and recognize the activity sequence. Furthermore, to enhance the robustness of HAR against the inter-class similarity and intra-class heterogeneity, a multi-level contrastive loss, containing the sample-level and segment-level contrast, has been proposed to learn a well-structured embedding space for better activity segmentation and recognition performance. Finally, with comprehensive experiments, we verify the effectiveness of the proposed method on two public HAR datasets, achieving significant improvements in the various evaluation metrics.
Inhale: Enabling High-Performance and Energy-Efficient In-SRAM Cryptographic Hash for IoT
Venue: arXiv cs.CR 2022
Year: 2022

In the age of big data, information security has become a major issue of debate, especially with the rise of the Internet of Things (IoT), where attackers can effortlessly obtain physical access to edge devices. The hash algorithm is the current foundation for data integrity and authentication. However, it is challenging to provide a high-performance, high-throughput, and energy-efficient solution on resource-constrained edge devices. In this paper, we propose Inhale, an in-SRAM architecture to effectively compute hash algorithms with innovative data alignment and efficient read/write strategies to implicitly execute data shift operations through the in-situ controller. We present two variations of Inhale: Inhale-Opt, which is optimized for latency, throughput, and area-overhead; and Inhale-Flex, which offers flexibility in repurposing a part of last-level caches for hash computation. We thoroughly evaluate our proposed architectures on both SRAM and ReRAM memories and compare them with the state-of-the-art in-memory and ASIC accelerators. Our performance evaluation confirms that Inhale can achieve 1.4x - 14.5x higher throughput-per-area and about two-orders-of-magnitude higher throughput-per-area-per-energy compared to the state-of-the-art solutions.
Order-Invariance of Two-Variable Logic is coNExpTime-complete
Venue: arXiv cs.LO 2022
Year: 2022

We establish coNExpTime-completeness of the problem of deciding order-invariance of a given two variable first-order formula, improving and significantly simplifying coTwoNExpTime bound by Zeume and Harwath.