OpenRaiser/Intern-Atlas huggingface dataset:
Fairness Testing of Deep Image Classification with Adequacy Metrics
Venue: arXiv cs.LG 2021
Year: 2021

As deep image classification applications, e.g., face recognition, become increasingly prevalent in our daily lives, their fairness issues raise more and more concern. It is thus crucial to comprehensively test the fairness of these applications before deployment. Existing fairness testing methods suffer from the following limitations: 1) applicability, i.e., they are only applicable for structured data or text without handling the high-dimensional and abstract domain sampling in the semantic level for image classification applications; 2) functionality, i.e., they generate unfair samples without providing testing criterion to characterize the model's fairness adequacy. To fill the gap, we propose DeepFAIT, a systematic fairness testing framework specifically designed for deep image classification applications. DeepFAIT consists of several important components enabling effective fairness testing of deep image classification applications: 1) a neuron selection strategy to identify the fairness-related neurons; 2) a set of multi-granularity adequacy metrics to evaluate the model's fairness; 3) a test selection algorithm for fixing the fairness issues efficiently. We have conducted experiments on widely adopted large-scale face recognition applications, i.e., VGGFace and FairFace. The experimental results confirm that our approach can effectively identify the fairness-related neurons, characterize the model's fairness, and select the most valuable test cases to mitigate the model's fairness issues.
Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees
Venue: arXiv cs.CL 2021
Year: 2021

Automated Scoring (AS), the natural language processing task of scoring essays and speeches in an educational testing setting, is growing in popularity and being deployed across contexts from government examinations to companies providing language proficiency services. However, existing systems either forgo human raters entirely, thus harming the reliability of the test, or score every response by both human and machine thereby increasing costs. We target the spectrum of possible solutions in between, making use of both humans and machines to provide a higher quality test while keeping costs reasonable to democratize access to AS. In this work, we propose a combination of the existing paradigms, sampling responses to be scored by humans intelligently. We propose reward sampling and observe significant gains in accuracy (19.80% increase on average) and quadratic weighted kappa (QWK) (25.60% on average) with a relatively small human budget (30% samples) using our proposed sampling. The accuracy increase observed using standard random and importance sampling baselines are 8.6% and 12.2% respectively. Furthermore, we demonstrate the system's model agnostic nature by measuring its performance on a variety of models currently deployed in an AS setting as well as pseudo models. Finally, we propose an algorithm to estimate the accuracy/QWK with statistical guarantees (Our code is available at https://git.io/J1IOy).
Exploring Student Representation For Neural Cognitive Diagnosis
Venue: arXiv cs.AI 2021
Year: 2021

Cognitive diagnosis, the goal of which is to obtain the proficiency level of students on specific knowledge concepts, is an fundamental task in smart educational systems. Previous works usually represent each student as a trainable knowledge proficiency vector, which cannot capture the relations of concepts and the basic profile(e.g. memory or comprehension) of students. In this paper, we propose a method of student representation with the exploration of the hierarchical relations of knowledge concepts and student embedding. Specifically, since the proficiency on parent knowledge concepts reflects the correlation between knowledge concepts, we get the first knowledge proficiency with a parent-child concepts projection layer. In addition, a low-dimension dense vector is adopted as the embedding of each student, and obtain the second knowledge proficiency with a full connection layer. Then, we combine the two proficiency vector above to get the final representation of students. Experiments show the effectiveness of proposed representation method.
Minimum Cuts in Directed Graphs via Partial Sparsification
Venue: arXiv cs.DS 2021
Year: 2021

We give an algorithm to find a minimum cut in an edge-weighted directed graph with $n$ vertices and $m$ edges in $\tilde O(n\cdot \max(m^{2/3}, n))$ time. This improves on the 30 year old bound of $\tilde O(nm)$ obtained by Hao and Orlin for this problem. Our main technique is to reduce the directed mincut problem to $\tilde O(\min(n/m^{1/3}, \sqrt{n}))$ calls of {\em any} maxflow subroutine. Using state-of-the-art maxflow algorithms, this yields the above running time. Our techniques also yield fast {\em approximation} algorithms for finding minimum cuts in directed graphs. For both edge and vertex weighted graphs, we give $(1+ε)$-approximation algorithms that run in $\tilde O(n^2 / ε^2)$ time.
On the modification of the SPARUS II AUV for close range imaging survey platform
Venue: arXiv cs.RO 2021
Year: 2021

Based on the need for high resolution underwater visual surveys, this study presents the adaptation of an existing SPARUS II autonomous underwater vehicle (AUV) into an entirely hovering AUV fully capable of performing autonomous, close range imaging survey missions. This paper focuses on the enhancement of the AUVs maneuvering capability (enabling improved maneuvering control), implementation of an state of the art thruster allocation algorithm (allowing optimal thrusters allocation and thrusters redundancy), and the development of an upgraded path following controller to facilitate precise and delicate motions necessary for high resolution imaging missions. To facilitate the vehicles adaptation, a dynamic model is developed. The calibration process of the dynamic model coefficients initially obtained using well accepted formulas, by computational fluid dynamics and in real sea experiments is presented. The in house development of a pressure resistant imaging system is also presented. This system which includes a stereo camera and high power lightning strobes was developed and fitted as a dedicated AUV payload. Finally, the performance of the platform is demonstrated in an actual seabed visual survey mission.
LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networks
Venue: arXiv cs.GR 2021
Year: 2021

We consider the attributes of a point cloud as samples of a vector-valued volumetric function at discrete positions. To compress the attributes given the positions, we compress the parameters of the volumetric function. We model the volumetric function by tiling space into blocks, and representing the function over each block by shifts of a coordinate-based, or implicit, neural network. Inputs to the network include both spatial coordinates and a latent vector per block. We represent the latent vectors using coefficients of the region-adaptive hierarchical transform (RAHT) used in the MPEG geometry-based point cloud codec G-PCC. The coefficients, which are highly compressible, are rate-distortion optimized by back-propagation through a rate-distortion Lagrangian loss in an auto-decoder configuration. The result outperforms RAHT by 2--4 dB. This is the first work to compress volumetric functions represented by local coordinate-based neural networks. As such, we expect it to be applicable beyond point clouds, for example to compression of high-resolution neural radiance fields.
Nonlinear Intensity Sonar Image Matching based on Deep Convolution Features
Venue: arXiv cs.CV 2021
Year: 2021

With the continuous development of underwater vision technology, more and more remote sensing images could be obtained. In the underwater scene, sonar sensors are currently the most effective remote perception devices, and the sonar images captured by them could provide rich environment information. In order to analyze a certain scene, we often need to merge the sonar images from different periods, various sonar frequencies and distinctive viewpoints. However, the above scenes will bring nonlinear intensity differences to the sonar images, which will make traditional matching methods almost ineffective. This paper proposes a non-linear intensity sonar image matching method that combines local feature points and deep convolution features. This method has two key advantages: (i) we generate data samples related to local feature points based on the self-learning idea; (ii) we use the convolutional neural network (CNN) and Siamese network architecture to measure the similarity of the local position in the sonar image pair. Our method encapsulates the feature extraction and feature matching stage in a model, and directly learns the mapping function from image patch pairs to matching labels, and achieves matching tasks in a near-end-to-end manner. Feature matching experiments are carried out on the sonar images acquired by autonomous underwater vehicle (AUV) in the real underwater environment. Experiment results show that our method has better matching effects and strong robustness.
Subject Enveloped Deep Sample Fuzzy Ensemble Learning Algorithm of Parkinson's Speech Data
Venue: arXiv cs.SD 2021
Year: 2021

Parkinson disease (PD)'s speech recognition is an effective way for its diagnosis, which has become a hot and difficult research area in recent years. As we know, there are large corpuses (segments) within one subject. However, too large segments will increase the complexity of the classification model. Besides, the clinicians interested in finding diagnostic speech markers that reflect the pathology of the whole subject. Since the optimal relevant features of each speech sample segment are different, it is difficult to find the uniform diagnostic speech markers. Therefore, it is necessary to reconstruct the existing large segments within one subject into few segments even one segment within one subject, which can facilitate the extraction of relevant speech features to characterize diagnostic markers for the whole subject. To address this problem, an enveloped deep speech sample learning algorithm for Parkinson's subjects based on multilayer fuzzy c-mean (MlFCM) clustering and interlayer consistency preservation is proposed in this paper. The algorithm can be used to achieve intra-subject sample reconstruction for Parkinson's disease (PD) to obtain a small number of high-quality prototype sample segments. At the end of the paper, several representative PD speech datasets are selected and compared with the state-of-the-art related methods, respectively. The experimental results show that the proposed algorithm is effective signifcantly.
Average reduced model to simulate solutions for heat and mass transfer through porous material
Venue: arXiv cs.CE 2021
Year: 2021

The design of numerical tools to model the behavior of building materials is a challenging task. The crucial point is to save computational cost and maintain high accuracy of predictions. There are two main limitations on the time scale choice, which put an obstacle to solve the above issues. First one is the numerical restriction. A number of research is dedicated to overcome this limitation and it is shown that it can be relaxed with innovative numerical schemes. The second one is the physical restriction. It is imposed by properties of a material, phenomena itself and corresponding boundary conditions. This work is focused on the study of a methodology that enables to overcome the physical restriction on the time grid. So-called Average Reduced Model (ARM) is suggested. It is based on smoothing the time-dependent boundary conditions. Besides, the approximate solution is decomposed into average and fluctuating components. The primer is obtained by integrating the equations over time, whereas the latter is an user-defined empirical model. The methodology is investigated for both heat diffusion and coupled heat and mass transfer. It is demonstrated that the signal core of the boundary conditions is preserved and the physical restriction can be relaxed. The model proved to be reliable, accurate and efficient also in comparison with the experimental data of two years. The implementation of the scarce time-step of $1 \, \sf{h}$ is justified. It is shown, that by maintaining the tolerable error it is possible to cut computational effort up to almost four times in comparison with the complete model with the same time grid.
Bipartite Temporal Graphs and the Parameterized Complexity of Multistage 2-Coloring
Venue: arXiv cs.CC 2021
Year: 2021

We consider the algorithmic complexity of recognizing bipartite temporal graphs. Rather than defining these graphs solely by their underlying graph or individual layers, we define a bipartite temporal graph as one in which every layer can be 2-colored in a way that results in few changes between any two consecutive layers. This approach follows the framework of multistage problems that has received a growing amount of attention in recent years. We investigate the complexity of recognizing these graphs. We show that this problem is NP-hard even if there are only two layers or if only one change is allowed between consecutive layers. We consider the parameterized complexity of the problem with respect to several structural graph parameters, which we transfer from the static to the temporal setting in three different ways. Finally, we consider a version of the problem in which we only restrict the total number of changes throughout the lifetime of the graph. We show that this variant is fixed-parameter tractable with respect to the number of changes.
Motion Detection using CSI from Raspberry Pi 4
Venue: arXiv cs.CV 2021
Year: 2021

Monitoring behaviour in smart homes using sensors can offer insights into changes in the independent ability and long-term health of residents. Passive Infrared motion sensors (PIRs) are standard, however may not accurately track the full duration of movement. They also require line-of-sight to detect motion which can restrict performance and ensures they must be visible to residents. Channel State Information (CSI) is a low cost, unintrusive form of radio sensing which can monitor movement but also offers opportunities to generate rich data. We have developed a novel, self-calibrating motion detection system which uses CSI data collected and processed on a stock Raspberry Pi 4. This system exploits the correlation between CSI frames, on which we perform variance analysis using our algorithm to accurately measure the full period of a resident's movement. We demonstrate the effectiveness of this approach in several real-world environments. Experiments conducted demonstrate that activity start and end time can be accurately detected for motion examples of different intensities at different locations.
Automated Approach for Computer Vision-based Vehicle Movement Classification at Traffic Intersections
Venue: arXiv cs.CV 2021
Year: 2021

Movement specific vehicle classification and counting at traffic intersections is a crucial component for various traffic management activities. In this context, with recent advancements in computer-vision based techniques, cameras have emerged as a reliable data source for extracting vehicular trajectories from traffic scenes. However, classifying these trajectories by movement type is quite challenging as characteristics of motion trajectories obtained this way vary depending on camera calibrations. Although some existing methods have addressed such classification tasks with decent accuracies, the performance of these methods significantly relied on manual specification of several regions of interest. In this study, we proposed an automated classification method for movement specific classification (such as right-turn, left-turn and through movements) of vision-based vehicle trajectories. Our classification framework identifies different movement patterns observed in a traffic scene using an unsupervised hierarchical clustering technique Thereafter a similarity-based assignment strategy is adopted to assign incoming vehicle trajectories to identified movement groups. A new similarity measure was designed to overcome the inherent shortcomings of vision-based trajectories. Experimental results demonstrated the effectiveness of the proposed classification approach and its ability to adapt to different traffic scenarios without any manual intervention.
Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Data
Venue: arXiv cs.LG 2021
Year: 2021

Deep network models perform excellently on In-Distribution (ID) data, but can significantly fail on Out-Of-Distribution (OOD) data. While developing methods focus on improving OOD generalization, few attention has been paid to evaluating the capability of models to handle OOD data. This study is devoted to analyzing the problem of experimental ID test and designing OOD test paradigm to accurately evaluate the practical performance. Our analysis is based on an introduced categorization of three types of distribution shifts to generate OOD data. Main observations include: (1) ID test fails in neither reflecting the actual performance of a single model nor comparing between different models under OOD data. (2) The ID test failure can be ascribed to the learned marginal and conditional spurious correlations resulted from the corresponding distribution shifts. Based on this, we propose novel OOD test paradigms to evaluate the generalization capacity of models to unseen data, and discuss how to use OOD test results to find bugs of models to guide model debugging.
The Integrated List of Agile Practices -- A Tertiary Study
Venue: arXiv cs.SE 2021
Year: 2021

Context: Companies adapt agile methods, practices or artifacts for their use in practice since more than two decades. This adaptions result in a wide variety of described agile practices. For instance, the Agile Alliance lists 75 different practices in its Agile Glossary. This situation may lead to misunderstandings, as agile practices with similar names can be interpreted and used differently. Objective: This paper synthesize an integrated list of agile practices, both from primary and secondary sources. Method: We performed a tertiary study to identify existing overviews and lists of agile practices in the literature. We identified 876 studies, of which 37 were included. Results: The results of our paper show that certain agile practices are listed and used more often in existing studies. Our integrated list of agile practices comprises 38 entries structured in five categories. Contribution: The high number of agile practices and thus, the wide variety increased steadily over the past decades due to the adaption of agile methods. Based on our findings, we present a comprehensive overview of agile practices. The research community benefits from our integrated list of agile practices as a potential basis for future research. Also, practitioners benefit from our findings, as the structured overview of agile practices provides the opportunity to select or adapt practices for their specific needs.
Early DSE and Automatic Generation of Coarse Grained Merged Accelerators
Venue: arXiv cs.AR 2021
Year: 2021

Post-Moore's law area-constrained systems rely on accelerators to deliver performance enhancements. Coarse grained accelerators can offer substantial domain acceleration, but manual, ad-hoc identification of code to accelerate is prohibitively expensive. Because cycle-accurate simulators and high-level synthesis flows are so time-consuming, manual creation of high-utilization accelerators that exploit control and data flow patterns at optimal granularities is rarely successful. To address these challenges, we present AccelMerger, the first automated methodology to create coarse grained, control- and data-flow-rich, merged accelerators. AccelMerger uses sequence alignment matching to recognize similar function call-graphs and loops, and neural networks to quickly evaluate their post-HLS characteristics. It accurately identifies which functions to accelerate, and it merges accelerators to respect an area budget and to accommodate system communication characteristics like latency and bandwidth. Merging two accelerators can save as much as 99% of the area of one. The space saved is used by a globally optimal integer linear program to allocate more accelerators for increased performance. We demonstate AccelMerger's effectiveness using HLS flows without any manual effort to fine-tune the resulting designs. On FPGA-based systems, AccelMerger yields application performance improvements of up to 16.7x over software implementations, and 1.91x on average with respect to state-of-the-art early-stage design space exploration tools.
Induce, Edit, Retrieve: Language Grounded Multimodal Schema for Instructional Video Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

Schemata are structured representations of complex tasks that can aid artificial intelligence by allowing models to break down complex tasks into intermediate steps. We propose a novel system that induces schemata from web videos and generalizes them to capture unseen tasks with the goal of improving video retrieval performance. Our system proceeds in three major phases: (1) Given a task with related videos, we construct an initial schema for a task using a joint video-text model to match video segments with text representing steps from wikiHow; (2) We generalize schemata to unseen tasks by leveraging language models to edit the text within existing schemata. Through generalization, we can allow our schemata to cover a more extensive range of tasks with a small amount of learning data; (3) We conduct zero-shot instructional video retrieval with the unseen task names as the queries. Our schema-guided approach outperforms existing methods for video retrieval, and we demonstrate that the schemata induced by our system are better than those generated by other models.
The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage
Venue: arXiv cs.LG 2021
Year: 2021

The People's Speech is a free-to-download 30,000-hour and growing supervised conversational English speech recognition dataset licensed for academic and commercial usage under CC-BY-SA (with a CC-BY subset). The data is collected via searching the Internet for appropriately licensed audio data with existing transcriptions. We describe our data collection methodology and release our data collection system under the Apache 2.0 license. We show that a model trained on this dataset achieves a 9.98% word error rate on Librispeech's test-clean test set.Finally, we discuss the legal and ethical issues surrounding the creation of a sizable machine learning corpora and plans for continued maintenance of the project under MLCommons's sponsorship.