OpenRaiser/Intern-Atlas huggingface dataset:
AutoCoMet: Smart Neural Architecture Search via Co-Regulated Shaping Reinforcement
Venue: arXiv cs.LG 2022
Year: 2022

Designing suitable deep model architectures, for AI-driven on-device apps and features, at par with rapidly evolving mobile hardware and increasingly complex target scenarios is a difficult task. Though Neural Architecture Search (NAS/AutoML) has made this easier by shifting paradigm from extensive manual effort to automated architecture learning from data, yet it has major limitations, leading to critical bottlenecks in the context of mobile devices, including model-hardware fidelity, prohibitive search times and deviation from primary target objective(s). Thus, we propose AutoCoMet that can learn the most suitable DNN architecture optimized for varied types of device hardware and task contexts, ~ 3x faster. Our novel co-regulated shaping reinforcement controller together with the high fidelity hardware meta-behavior predictor produces a smart, fast NAS framework that adapts to context via a generalized formalism for any kind of multi-criteria optimization.
Development of a Scale to Measure Technology Acceptance in Smart Agriculture
Venue: arXiv cs.HC 2022
Year: 2022

This paper describes the development of a scale to measure technology acceptance in smart agriculture. The scale is intended for use in diverse situations, ranging for the evaluation of existing technologies already in widespread use, to the evaluation of prototype systems. A systematic screening of prior literature was conducted to identify initial scale items regarding how technology acceptance is currently understood and measured. These items were iteratively reviewed and systematically categorised to develop the final scale proposed in this paper. In future work, this scale will be validated through user studies. The purpose of this paper is to make the initial scale available to the research community with a view to initial use and further evaluation.
Contextual Information Based Anomaly Detection for a Multi-Scene UAV Aerial Videos
Venue: arXiv cs.CV 2022
Year: 2022

UAV based surveillance is gaining much interest worldwide due to its extensive applications in monitoring wildlife, urban planning, disaster management, campus security, etc. These videos are analyzed for strange/odd/anomalous patterns which are essential aspects of surveillance. But manual analysis of these videos is tedious and laborious. Hence, the development of computer-aided systems for the analysis of UAV based surveillance videos is crucial. Despite this interest, in literature, several computer aided systems are developed focusing only on CCTV based surveillance videos. These methods are designed for single scene scenarios and lack contextual knowledge which is required for multi-scene scenarios. Furthermore, the lack of standard UAV based anomaly detection datasets limits the development of these systems. In this regard, the present work aims at the development of a Computer Aided Decision support system to analyse UAV based surveillance videos. A new UAV based multi-scene anomaly detection dataset is developed with frame-level annotations for the development of computer aided systems. It holistically uses contextual, temporal and appearance features for accurate detection of anomalies. Furthermore, a new inference strategy is proposed that utilizes few anomalous samples along with normal samples to identify better decision boundaries. The proposed method is extensively evaluated on the UAV based anomaly detection dataset and performed competitively with respect to state-of-the-art methods.
Improving the Learnability of Machine Learning APIs by Semi-Automated API Wrapping
Venue: arXiv cs.SE 2022
Year: 2022

A major hurdle for students and professional software developers who want to enter the world of machine learning (ML), is mastering not just the scientific background but also the available ML APIs. Therefore, we address the challenge of creating APIs that are easy to learn and use, especially by novices. However, it is not clear how this can be achieved without compromising expressiveness. We investigate this problem for scikit-learn, a widely used ML API. In this paper, we analyze its use by the Kaggle community, identifying unused and apparently useless parts of the API that can be eliminated without affecting client programs. In addition, we discuss usability issues in the remaining parts, propose related design improvements and show how they can be implemented by semi-automated wrapping of the existing third-party API.
Deep Learning for Encrypted Traffic Classification and Unknown Data Detection
Venue: arXiv cs.CR 2022
Year: 2022

Despite the widespread use of encryption techniques to provide confidentiality over Internet communications, mobile device users are still susceptible to privacy and security risks. In this paper, a new Deep Neural Network (DNN) based user activity detection framework is proposed to identify fine grained user activities performed on mobile applications (known as in-app activities) from a sniffed encrypted Internet traffic stream. One of the challenges is that there are countless applications, and it is practically impossible to collect and train a DNN model using all possible data from them. Therefore, in this work we exploit the probability distribution of DNN output layer to filter the data from applications that are not considered during the model training (i.e., unknown data). The proposed framework uses a time window based approach to divide the traffic flow of an activity into segments, so that in-app activities can be identified just by observing only a fraction of the activity related traffic. Our tests have shown that the DNN based framework has demonstrated an accuracy of 90% or above in identifying previously trained in-app activities and an average accuracy of 79% in identifying previously untrained in-app activity traffic as unknown data when this framework is employed.
Trojan Horse Training for Breaking Defenses against Backdoor Attacks in Deep Learning
Venue: arXiv cs.CR 2022
Year: 2022

Machine learning (ML) models that use deep neural networks are vulnerable to backdoor attacks. Such attacks involve the insertion of a (hidden) trigger by an adversary. As a consequence, any input that contains the trigger will cause the neural network to misclassify the input to a (single) target class, while classifying other inputs without a trigger correctly. ML models that contain a backdoor are called Trojan models. Backdoors can have severe consequences in safety-critical cyber and cyber physical systems when only the outputs of the model are available. Defense mechanisms have been developed and illustrated to be able to distinguish between outputs from a Trojan model and a non-Trojan model in the case of a single-target backdoor attack with accuracy > 96 percent. Understanding the limitations of a defense mechanism requires the construction of examples where the mechanism fails. Current single-target backdoor attacks require one trigger per target class. We introduce a new, more general attack that will enable a single trigger to result in misclassification to more than one target class. Such a misclassification will depend on the true (actual) class that the input belongs to. We term this category of attacks multi-target backdoor attacks. We demonstrate that a Trojan model with either a single-target or multi-target trigger can be trained so that the accuracy of a defense mechanism that seeks to distinguish between outputs coming from a Trojan and a non-Trojan model will be reduced. Our approach uses the non-Trojan model as a teacher for the Trojan model and solves a min-max optimization problem between the Trojan model and defense mechanism. Empirical evaluations demonstrate that our training procedure reduces the accuracy of a state-of-the-art defense mechanism from >96 to 0 percent.
Modeling Users' Contextualized Page-wise Feedback for Click-Through Rate Prediction in E-commerce Search
Venue: arXiv cs.IR 2022
Year: 2022

Modeling user's historical feedback is essential for Click-Through Rate Prediction in personalized search and recommendation. Existing methods usually only model users' positive feedback information such as click sequences which neglects the context information of the feedback. In this paper, we propose a new perspective for context-aware users' behavior modeling by including the whole page-wisely exposed products and the corresponding feedback as contextualized page-wise feedback sequence. The intra-page context information and inter-page interest evolution can be captured to learn more specific user preference. We design a novel neural ranking model RACP(i.e., Recurrent Attention over Contextualized Page sequence), which utilizes page-context aware attention to model the intra-page context. A recurrent attention process is used to model the cross-page interest convergence evolution as denoising the interest in the previous pages. Experiments on public and real-world industrial datasets verify our model's effectiveness.
Understanding Code Snippets in Code Reviews: A Preliminary Study of the OpenStack Community
Venue: arXiv cs.SE 2022
Year: 2022

Code review is a mature practice for software quality assurance in software development with which reviewers check the code that has been committed by developers, and verify the quality of code. During the code review discussions, reviewers and developers might use code snippets to provide necessary information (e.g., suggestions or explanations). However, little is known about the intentions and impacts of code snippets in code reviews. To this end, we conducted a preliminary study to investigate the nature of code snippets and their purposes in code reviews. We manually collected and checked 10,790 review comments from the Nova and Neutron projects of the OpenStack community, and finally obtained 626 review comments that contain code snippets for further analysis. The results show that: (1) code snippets are not prevalently used in code reviews, and most of the code snippets are provided by reviewers. (2) We identified two high-level purposes of code snippets provided by reviewers (i.e., Suggestion and Citation) with six detailed purposes, among which, Improving Code Implementation is the most common purpose. (3) For the code snippets in code reviews with the aim of suggestion, around 68.1% was accepted by developers. The results highlight promising research directions on using code snippets in code reviews.
RGB-D Neural Radiance Fields: Local Sampling for Faster Training
Venue: arXiv cs.CV 2022
Year: 2022

Learning a 3D representation of a scene has been a challenging problem for decades in computer vision. Recent advances in implicit neural representation from images using neural radiance fields(NeRF) have shown promising results. Some of the limitations of previous NeRF based methods include longer training time, and inaccurate underlying geometry. The proposed method takes advantage of RGB-D data to reduce training time by leveraging depth sensing to improve local sampling. This paper proposes a depth-guided local sampling strategy and a smaller neural network architecture to achieve faster training time without compromising quality.
Earnings-22: A Practical Benchmark for Accents in the Wild
Venue: arXiv cs.CL 2022
Year: 2022

Modern automatic speech recognition (ASR) systems have achieved superhuman Word Error Rate (WER) on many common corpora despite lacking adequate performance on speech in the wild. Beyond that, there is a lack of real-world, accented corpora to properly benchmark academic and commercial models. To ensure this type of speech is represented in ASR benchmarking, we present Earnings-22, a 125 file, 119 hour corpus of English-language earnings calls gathered from global companies. We run a comparison across 4 commercial models showing the variation in performance when taking country of origin into consideration. Looking at hypothesis transcriptions, we explore errors common to all ASR systems tested. By examining Individual Word Error Rate (IWER), we find that key speech features impact model performance more for certain accents than others. Earnings-22 provides a free-to-use benchmark of real-world, accented audio to bridge academic and industrial research.
Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model
Venue: arXiv cs.IR 2022
Year: 2022

Science and technology big data contain a lot of cross-media information.There are images and texts in the scientific paper.The s ingle modal search method cannot well meet the needs of scientific researchers.This paper proposes a cross-media scientific research achievements retrieval method based on deep language model (CARDL).It achieves a unified cross-media semantic representation by learning the semantic association between different modal data, and is applied to the generation of text semantic vector of scientific research achievements, and then cross-media retrieval is realized through semantic similarity matching between different modal data.Experimental results show that the proposed CARDL method achieves better cross-modal retrieval performance than existing methods. Key words science and technology big data ; cross-media retrieval; cross-media semantic association learning; deep language model; semantic similarity
Towards the Future: Bring Program Correctness back to the focus
Venue: arXiv cs.SE 2022
Year: 2022

Program correctness used to be the main concern of computer software in the early days when formal semantics was a hot topic. But, the word "correct" was afterwards replaced by reliable, robust and trustworthy etc., a tradeoff situation then. This is not because correctness is no longer important, but because people found no way to get through in this direction. The tradeoff has led software engineers to focus on techniques and testing tools. Rapid development of software engineering has now reached a peak and programmers are now working freely without worrying too much about bugs, since bugs are not avoidable anyway. Is it meaningful to talk about program correctness today? Our answer is yes. It is the time to seriously consider correctness again, before it is too late, to prepare for the future. Future generation computer systems should be correct, both syntactically (statically) and semantically (dynamically). The book "OESPA: Semantic Oriented Theory of Programming" (2019) by the first author has opened a new direction for semantic study. Theoretically speaking, it is possible now, based on OESPA, to compute program semantics from program text so that program correctness could be proved. But, semantic computations and correctness proving cannot be done by hand when the size of a program is big. Automatic tools are necessary. This paper tries to lay a foundation for developing needed auto tools, so that OESPA is enriched to serve future need. To this end, a new concept named conditional semantic predicate is proposed. Concepts in OESPA, including semantic functions, semantic predicates, semantic formulas and semantic calculus, are re-represented in accordance. Such re-introduction is necessary since the book is the only publication on semantic calculus so far. The new version of semantic calculus illustrates how semantics auto-computation would be carried out.
Construction and Linearity of Z_pZ_{p^2}-Linear Generalized Hadamard Codes
Venue: arXiv cs.IT 2022
Year: 2022

The $\Z_p\Z_{p^2}$-additive codes are subgroups of $\Z_p^{α_1} \times \Z_{p^2}^{α_2}$, and can be seen as linear codes over $\Z_p$ when $α_2=0$, $\Z_{p^2}$-additive codes when $α_1=0$, or $\Z_2\Z_4$-additive codes when $p=2$. A $\Z_p\Z_{p^2}$-linear generalized Hadamard (GH) code is a GH code over $\Z_p$ which is the Gray map image of a $\Z_p\Z_{p^2}$-additive code. In this paper, we generalize some known results for $\Z_p\Z_{p^2}$-linear GH codes with $p=2$ to any $p\geq 3$ prime when $α_1 \neq 0$. First, we give a recursive construction of $\Z_p\Z_{p^2}$-additive GH codes of type $(α_1,α_2;t_1,t_2)$ with $t_1,t_2\geq 1$. Then, we show for which types the corresponding $\Z_p\Z_{p^2}$-linear GH codes are non-linear over $\Z_p$. Finally, according to some computational results, we see that, unlike $\Z_4$-linear GH codes, when $p\geq 3$ prime, the $\Z_{p^2}$-linear GH codes are not included in the family of $\Z_p\Z_{p^2}$-linear GH codes with $α_1\not =0$.
Nearly Minimax Algorithms for Linear Bandits with Shared Representation
Venue: arXiv cs.LG 2022
Year: 2022

We give novel algorithms for multi-task and lifelong linear bandits with shared representation. Specifically, we consider the setting where we play $M$ linear bandits with dimension $d$, each for $T$ rounds, and these $M$ bandit tasks share a common $k(\ll d)$ dimensional linear representation. For both the multi-task setting where we play the tasks concurrently, and the lifelong setting where we play tasks sequentially, we come up with novel algorithms that achieve $\widetilde{O}\left(d\sqrt{kMT} + kM\sqrt{T}\right)$ regret bounds, which matches the known minimax regret lower bound up to logarithmic factors and closes the gap in existing results [Yang et al., 2021]. Our main technique include a more efficient estimator for the low-rank linear feature extractor and an accompanied novel analysis for this estimator.
100 Gb/s High Throughput Serial Protocol (HTSP) for Data Acquisition Systems with Interleaved Streaming
Venue: arXiv cs.AR 2022
Year: 2022

Demands on Field-Programmable Gate Array (FPGA) data transport have been increasing over the years as frame sizes and refresh rates increase. As the bandwidths requirements increase the ability to implement data transport protocol layers using "soft" programmable logic becomes harder and start to require harden IP blocks implementation. To reduce the number of physical links and interconnects, it is common for data acquisition systems to require interleaving of streams on the same link (e.g. streaming data and streaming register access). This paper presents a way to leverage existing FPGA harden IP blocks to achieve a robust, low latency 100 Gb/s point-to-point link with minimal programmable logic overhead geared towards the needs of data acquisition systems with interleaved streaming requirements.
MAP-Gen: An Automated 3D-Box Annotation Flow with Multimodal Attention Point Generator
Venue: arXiv cs.CV 2022
Year: 2022

Manually annotating 3D point clouds is laborious and costly, limiting the training data preparation for deep learning in real-world object detection. While a few previous studies tried to automatically generate 3D bounding boxes from weak labels such as 2D boxes, the quality is sub-optimal compared to human annotators. This work proposes a novel autolabeler, called multimodal attention point generator (MAP-Gen), that generates high-quality 3D labels from weak 2D boxes. It leverages dense image information to tackle the sparsity issue of 3D point clouds, thus improving label quality. For each 2D pixel, MAP-Gen predicts its corresponding 3D coordinates by referencing context points based on their 2D semantic or geometric relationships. The generated 3D points densify the original sparse point clouds, followed by an encoder to regress 3D bounding boxes. Using MAP-Gen, object detection networks that are weakly supervised by 2D boxes can achieve 94~99% performance of those fully supervised by 3D annotations. It is hopeful this newly proposed MAP-Gen autolabeling flow can shed new light on utilizing multimodal information for enriching sparse point clouds.
Demonstration-Bootstrapped Autonomous Practicing via Multi-Task Reinforcement Learning
Venue: arXiv cs.RO 2022
Year: 2022

Reinforcement learning systems have the potential to enable continuous improvement in unstructured environments, leveraging data collected autonomously. However, in practice these systems require significant amounts of instrumentation or human intervention to learn in the real world. In this work, we propose a system for reinforcement learning that leverages multi-task reinforcement learning bootstrapped with prior data to enable continuous autonomous practicing, minimizing the number of resets needed while being able to learn temporally extended behaviors. We show how appropriately provided prior data can help bootstrap both low-level multi-task policies and strategies for sequencing these tasks one after another to enable learning with minimal resets. This mechanism enables our robotic system to practice with minimal human intervention at training time while being able to solve long horizon tasks at test time. We show the efficacy of the proposed system on a challenging kitchen manipulation task both in simulation and in the real world, demonstrating the ability to practice autonomously in order to solve temporally extended problems.
DRaCoN -- Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars
Venue: arXiv cs.CV 2022
Year: 2022

Acquisition and creation of digital human avatars is an important problem with applications to virtual telepresence, gaming, and human modeling. Most contemporary approaches for avatar generation can be viewed either as 3D-based methods, which use multi-view data to learn a 3D representation with appearance (such as a mesh, implicit surface, or volume), or 2D-based methods which learn photo-realistic renderings of avatars but lack accurate 3D representations. In this work, we present, DRaCoN, a framework for learning full-body volumetric avatars which exploits the advantages of both the 2D and 3D neural rendering techniques. It consists of a Differentiable Rasterization module, DiffRas, that synthesizes a low-resolution version of the target image along with additional latent features guided by a parametric body model. The output of DiffRas is then used as conditioning to our conditional neural 3D representation module (c-NeRF) which generates the final high-res image along with body geometry using volumetric rendering. While DiffRas helps in obtaining photo-realistic image quality, c-NeRF, which employs signed distance fields (SDF) for 3D representations, helps to obtain fine 3D geometric details. Experiments on the challenging ZJU-MoCap and Human3.6M datasets indicate that DRaCoN outperforms state-of-the-art methods both in terms of error metrics and visual quality.
Collaborative Learning and Patterns of Practice
Venue: arXiv cs.HC 2022
Year: 2022

In this article, an overview of the background, the research approaches and the patterns of practice in the field of collaborative learning are provided. A definition of collaborative learning and an overview of fundamental aspects that shape research and practice in this field are included. Pedagogies and learning theories that are used as foundations of the field alongside goals and objectives of collaborative learning approaches are discussed. Popular patterns of practice, exploring their application in classrooms and elaborating on the state of the art around those practices in research are outlined. A discussion about important topics, open questions and future directions are provided in conclusion.