OpenRaiser/Intern-Atlas huggingface dataset:
Prerequisite-driven Q-matrix Refinement for Learner Knowledge Assessment: A Case Study in Online Learning Context
Venue: arXiv cs.CY 2022
Year: 2022

The ever growing abundance of learning traces in the online learning platforms promises unique insights into the learner knowledge assessment (LKA), a fundamental personalized-tutoring technique for enabling various further adaptive tutoring services in these platforms. Precise assessment of learner knowledge requires the fine-grained Q-matrix, which is generally designed by experts to map the items to skills in the domain. Due to the subjective tendency, some misspecifications may degrade the performance of LKA. Some efforts have been made to refine the small-scale Q-matrix, however, it is difficult to extend the scalability and apply these methods to the large-scale online learning context with numerous items and massive skills. Moreover, the existing LKA models employ flexible deep learning models that excel at this task, but the adequacy of LKA is still challenged by the representation capability of the models on the quite sparse item-skill graph and the learners' exercise data. To overcome these issues, in this paper we propose a prerequisite-driven Q-matrix refinement framework for learner knowledge assessment (PQRLKA) in online context. We infer the prerequisites from learners' response data and use it to refine the expert-defined Q-matrix, which enables the interpretability and the scalability to apply it to the large-scale online learning context. Based on the refined Q-matrix, we propose a Metapath2Vec enhanced convolutional representation method to obtain the comprehensive representations of the items with rich information, and feed them to the PQRLKA model to finally assess the learners' knowledge. Experiments conducted on three real-world datasets demonstrate the capability of our model to infer the prerequisites for Q-matrix refinement, and also its superiority for the LKA task.
Why is the video analytics accuracy fluctuating, and what can we do about it?
Venue: arXiv cs.CV 2022
Year: 2022

It is a common practice to think of a video as a sequence of images (frames), and re-use deep neural network models that are trained only on images for similar analytics tasks on videos. In this paper, we show that this leap of faith that deep learning models that work well on images will also work well on videos is actually flawed. We show that even when a video camera is viewing a scene that is not changing in any human-perceptible way, and we control for external factors like video compression and environment (lighting), the accuracy of video analytics application fluctuates noticeably. These fluctuations occur because successive frames produced by the video camera may look similar visually, but these frames are perceived quite differently by the video analytics applications. We observed that the root cause for these fluctuations is the dynamic camera parameter changes that a video camera automatically makes in order to capture and produce a visually pleasing video. The camera inadvertently acts as an unintentional adversary because these slight changes in the image pixel values in consecutive frames, as we show, have a noticeably adverse impact on the accuracy of insights from video analytics tasks that re-use image-trained deep learning models. To address this inadvertent adversarial effect from the camera, we explore the use of transfer learning techniques to improve learning in video analytics tasks through the transfer of knowledge from learning on image analytics tasks. In particular, we show that our newly trained Yolov5 model reduces fluctuation in object detection across frames, which leads to better tracking of objects(40% fewer mistakes in tracking). Our paper also provides new directions and techniques to mitigate the camera's adversarial effect on deep learning models used for video analytics applications.
Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages
Venue: arXiv cs.CL 2022
Year: 2022

End-to-end (E2E) models have become the default choice for state-of-the-art speech recognition systems. Such models are trained on large amounts of labelled data, which are often not available for low-resource languages. Techniques such as self-supervised learning and transfer learning hold promise, but have not yet been effective in training accurate models. On the other hand, collecting labelled datasets on a diverse set of domains and speakers is very expensive. In this work, we demonstrate an inexpensive and effective alternative to these approaches by ``mining'' text and audio pairs for Indian languages from public sources, specifically from the public archives of All India Radio. As a key component, we adapt the Needleman-Wunsch algorithm to align sentences with corresponding audio segments given a long audio and a PDF of its transcript, while being robust to errors due to OCR, extraneous text, and non-transcribed speech. We thus create Shrutilipi, a dataset which contains over 6,400 hours of labelled audio across 12 Indian languages totalling to 4.95M sentences. On average, Shrutilipi results in a 2.3x increase over publicly available labelled data. We establish the quality of Shrutilipi with 21 human evaluators across the 12 languages. We also establish the diversity of Shrutilipi in terms of represented regions, speakers, and mentioned named entities. Significantly, we show that adding Shrutilipi to the training set of Wav2Vec models leads to an average decrease in WER of 5.8\% for 7 languages on the IndicSUPERB benchmark. For Hindi, which has the most benchmarks (7), the average WER falls from 18.8% to 13.5%. This improvement extends to efficient models: We show a 2.3% drop in WER for a Conformer model (10x smaller than Wav2Vec). Finally, we demonstrate the diversity of Shrutilipi by showing that the model trained with it is more robust to noisy input.
A Joint Framework to Privacy-Preserving Edge Intelligence in Vehicular Networks
Venue: arXiv cs.NI 2022
Year: 2022

The number of internet-connected devices has been exponentially growing with the massive volume of heterogeneous data generated from various devices, resulting in a highly intertwined cyber-physical system. Currently, the Edge Intelligence System (EIS) concept that leverages the merits of edge computing and Artificial Intelligence (AI) is utilized to provide smart cloud services with powerful computational processing and reduce decision-making delays. Thus, EIS offers a possible solution to realizing future Intelligent Transportation Systems (ITS), especially in a vehicular network framework. However, since the central aggregator server supervises the entire system orchestration, the existing EIS framework faces several challenges and is still potentially susceptible to numerous malicious attacks. Hence, to solve the issues mentioned earlier, this paper presents the notion of secure edge intelligence, merging the benefits of Federated Learning (FL), blockchain, and Local Differential Privacy (LDP). The blockchain-assisted FL approach efficiently improves traffic prediction accuracy and enhances user privacy and security by recording transactions in immutable distributed ledger networks and providing a decentralized reward mechanism system. Furthermore, LDP is empowered to strengthen the confidentiality of data sharing transactions, especially in protecting users' private data from various attacks. The proposed framework has been implemented in two scenarios, i.e., blockchain-based FL to efficiently develop the decentralized traffic management for vehicular networks and LDP-based FL to produce randomized privacy protection using the IBM Library for differential privacy.
Leveraging Synthetic Data to Learn Video Stabilization Under Adverse Conditions
Venue: arXiv cs.CV 2022
Year: 2022

Video stabilization plays a central role to improve videos quality. However, despite the substantial progress made by these methods, they were, mainly, tested under standard weather and lighting conditions, and may perform poorly under adverse conditions. In this paper, we propose a synthetic-aware adverse weather robust algorithm for video stabilization that does not require real data and can be trained only on synthetic data. We also present Silver, a novel rendering engine to generate the required training data with an automatic ground-truth extraction procedure. Our approach uses our specially generated synthetic data for training an affine transformation matrix estimator avoiding the feature extraction issues faced by current methods. Additionally, since no video stabilization datasets under adverse conditions are available, we propose the novel VSAC105Real dataset for evaluation. We compare our method to five state-of-the-art video stabilization algorithms using two benchmarks. Our results show that current approaches perform poorly in at least one weather condition, and that, even training in a small dataset with synthetic data only, we achieve the best performance in terms of stability average score, distortion score, success rate, and average cropping ratio when considering all weather conditions. Hence, our video stabilization model generalizes well on real-world videos and does not require large-scale synthetic training data to converge.
NeuralSI: Structural Parameter Identification in Nonlinear Dynamical Systems
Venue: arXiv cs.LG 2022
Year: 2022

Structural monitoring for complex built environments often suffers from mismatch between design, laboratory testing, and actual built parameters. Additionally, real-world structural identification problems encounter many challenges. For example, the lack of accurate baseline models, high dimensionality, and complex multivariate partial differential equations (PDEs) pose significant difficulties in training and learning conventional data-driven algorithms. This paper explores a new framework, dubbed NeuralSI, for structural identification by augmenting PDEs that govern structural dynamics with neural networks. Our approach seeks to estimate nonlinear parameters from governing equations. We consider the vibration of nonlinear beams with two unknown parameters, one that represents geometric and material variations, and another that captures energy losses in the system mainly through damping. The data for parameter estimation is obtained from a limited set of measurements, which is conducive to applications in structural health monitoring where the exact state of an existing structure is typically unknown and only a limited amount of data samples can be collected in the field. The trained model can also be extrapolated under both standard and extreme conditions using the identified structural parameters. We compare with pure data-driven Neural Networks and other classical Physics-Informed Neural Networks (PINNs). Our approach reduces both interpolation and extrapolation errors in displacement distribution by two to five orders of magnitude over the baselines. Code is available at https://github.com/human-analysis/neural-structural-identification
OSF: An Open-Source Framework for Synchronous Flooding over Multiple Physical Layers
Venue: arXiv cs.NI 2022
Year: 2022

Flooding protocols based on concurrent transmissions are regarded as the most reliable way to collect or disseminate data across a multi-hop low-power wireless mesh network. Recent works have shown that such protocols are effective for narrowband communication not only over IEEE 802.15.4, but also over the BLE 5 physical layers (PHYs). However, to date, existing literature has only built synchronous flooding solutions on top of a single PHY, and there has been no attempt to leverage different PHYs at runtime to increase performance. This paper fills this gap and presents OSF, an open-source framework that enables the design of multi-PHY synchronous flooding solutions thanks to a novel radio driver and middle-ware architecture capable of dynamically switching the underlying physical layer. This allows exploitation of the specific benefits of each PHY (e.g., higher data-rate, increased robustness) on-demand during each flood, increasing performance. We tailor OSF to the off-the-shelf nRF52840 platform, and showcase its benefits by comparing single-PHY and multi-PHY synchronous flooding solutions on a real-world testbed.
Ammunition Component Classification Using Deep Learning
Venue: arXiv cs.CV 2022
Year: 2022

Ammunition scrap inspection is an essential step in the process of recycling ammunition metal scrap. Most ammunition is composed of a number of components, including case, primer, powder, and projectile. Ammo scrap containing energetics is considered to be potentially dangerous and should be separated before the recycling process. Manually inspecting each piece of scrap is tedious and time-consuming. We have gathered a dataset of ammunition components with the goal of applying artificial intelligence for classifying safe and unsafe scrap pieces automatically. First, two training datasets are manually created from visual and x-ray images of ammo. Second, the x-ray dataset is augmented using the spatial transforms of histogram equalization, averaging, sharpening, power law, and Gaussian blurring in order to compensate for the lack of sufficient training data. Lastly, the representative YOLOv4 object detection method is applied to detect the ammo components and classify the scrap pieces into safe and unsafe classes, respectively. The trained models are tested against unseen data in order to evaluate the performance of the applied method. The experiments demonstrate the feasibility of ammo component detection and classification using deep learning. The datasets and the pre-trained models are available at https://github.com/hadi-ghnd/Scrap-Classification.
Constraining Pseudo-label in Self-training Unsupervised Domain Adaptation with Energy-based Model
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning is usually data starved, and the unsupervised domain adaptation (UDA) is developed to introduce the knowledge in the labeled source domain to the unlabeled target domain. Recently, deep self-training presents a powerful means for UDA, involving an iterative process of predicting the target domain and then taking the confident predictions as hard pseudo-labels for retraining. However, the pseudo-labels are usually unreliable, thus easily leading to deviated solutions with propagated errors. In this paper, we resort to the energy-based model and constrain the training of the unlabeled target sample with an energy function minimization objective. It can be achieved via a simple additional regularization or an energy-based loss. This framework allows us to gain the benefits of the energy-based model, while retaining strong discriminative performance following a plug-and-play fashion. The convergence property and its connection with classification expectation minimization are investigated. We deliver extensive experiments on the most popular and large-scale UDA benchmarks of image classification as well as semantic segmentation to demonstrate its generality and effectiveness.
Linear codes associated with the Desarguesian ovoids in $Q^+(7,q)$
Venue: arXiv cs.IT 2022
Year: 2022

The Desarguesian ovoids in the orthogonal polar space $Q^+(7,q)$ with $q$ even have first been introduced by Kantor by examining the $8$-dimensional absolutely irreducible modular representations of $\text{PGL}(2,q^3)$. We investigate this module for all prime power values of $q$. The shortest $\text{PGL}(2,q^3)$-orbit $O$ gives the Desarguesian ovoid in $Q^+(7,q)$ for even $q$ and it is known to give a complete partial ovoid of the symplectic polar space $W(7,q)$ for odd~$q$. We determine the hyperplane sections of $O$. As a corollary, we obtain the parameters $[q^3+1,8,q^3-q^2-q]_q$ and the weight distribution of the associated $\mathbb{F}_q$-linear code $C_O$ and the parameters $[q^3+1,q^3-7,5]_q$ of the dual code $C_O^\perp$ for $q \ge 4$. We also show that both codes $C_O$ and $C_O^\perp$ are length-optimal for all prime power values of $q$.
Tensor Decomposition based Personalized Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) is a new distributed machine learning framework that can achieve reliably collaborative training without collecting users' private data. However, due to FL's frequent communication and average aggregation strategy, they experience challenges scaling to statistical diversity data and large-scale models. In this paper, we propose a personalized FL framework, named Tensor Decomposition based Personalized Federated learning (TDPFed), in which we design a novel tensorized local model with tensorized linear layers and convolutional layers to reduce the communication cost. TDPFed uses a bi-level loss function to decouple personalized model optimization from the global model learning by controlling the gap between the personalized model and the tensorized local model. Moreover, an effective distributed learning strategy and two different model aggregation strategies are well designed for the proposed TDPFed framework. Theoretical convergence analysis and thorough experiments demonstrate that our proposed TDPFed framework achieves state-of-the-art performance while reducing the communication cost.
HiStore: Rethinking Hybrid Index in RDMA-based Key-Value Store
Venue: arXiv cs.DB 2022
Year: 2022

RDMA (Remote Direct Memory Access) is widely exploited in building key-value stores to achieve ultra low latency. In RDMA-based key-value stores, the indexing time takes a large fraction (up to 74%) of the overall operation latency as RDMA enables fast data accesses. However, the single index structure used in existing RDMA-based key-value stores, either hash-based or sorted index, fails to support range queries efficiently while achieving high performance for single-point operations. In this paper, we reconsider the adoption of hybrid index in the key-value stores based on RDMA, to combine the benefits of hash table and sorted index. We propose HiStore, an RDMA-based key-value store using hash table for single-point lookups and leveraging skiplist for range queries. To maintain strong consistency in a lightweight and efficient approach, HiStore introduces index groups where a skiplist corresponds to a hash table, and asynchronously applies updates to the skiplist within a group. Guided by previous work on using RDMA for key-value services, HiStore dedicatedly chooses different RDMA primitives to optimize the read and write performance. Furthermore, HiStore tolerates the failures of servers that maintain index structures with index replication for high availability. Our evaluation results demonstrate that HiStore improves the performance of both GET and SCAN operations (by up to 2.03x) with hybrid index.
A Federated Learning-enabled Smart Street Light Monitoring Application: Benefits and Future Challenges
Venue: arXiv cs.CV 2022
Year: 2022

Data-enabled cities are recently accelerated and enhanced with automated learning for improved Smart Cities applications. In the context of an Internet of Things (IoT) ecosystem, the data communication is frequently costly, inefficient, not scalable and lacks security. Federated Learning (FL) plays a pivotal role in providing privacy-preserving and communication efficient Machine Learning (ML) frameworks. In this paper we evaluate the feasibility of FL in the context of a Smart Cities Street Light Monitoring application. FL is evaluated against benchmarks of centralised and (fully) personalised machine learning techniques for the classification task of the lampposts operation. Incorporating FL in such a scenario shows minimal performance reduction in terms of the classification task, but huge improvements in the communication cost and the privacy preserving. These outcomes strengthen FL's viability and potential for IoT applications.
On Unsupervised Training of Link Grammar Based Language Models
Venue: arXiv cs.CL 2022
Year: 2022

In this short note we explore what is needed for the unsupervised training of graph language models based on link grammars. First, we introduce the ter-mination tags formalism required to build a language model based on a link grammar formalism of Sleator and Temperley [21] and discuss the influence of context on the unsupervised learning of link grammars. Second, we pro-pose a statistical link grammar formalism, allowing for statistical language generation. Third, based on the above formalism, we show that the classical dissertation of Yuret [25] on discovery of linguistic relations using lexical at-traction ignores contextual properties of the language, and thus the approach to unsupervised language learning relying just on bigrams is flawed. This correlates well with the unimpressive results in unsupervised training of graph language models based on bigram approach of Yuret.
Measuring the Prevalence of Anti-Social Behavior in Online Communities
Venue: arXiv cs.HC 2022
Year: 2022

With increasing attention to online anti-social behaviors such as personal attacks and bigotry, it is critical to have an accurate accounting of how widespread anti-social behaviors are. In this paper, we empirically measure the prevalence of anti-social behavior in one of the world's most popular online community platforms. We operationalize this goal as measuring the proportion of unmoderated comments in the 97 most popular communities on Reddit that violate eight widely accepted platform norms. To achieve this goal, we contribute a human-AI pipeline for identifying these violations and a bootstrap sampling method to quantify measurement uncertainty. We find that 6.25% (95% Confidence Interval [5.36%, 7.13%]) of all comments in 2016, and 4.28% (95% CI [2.50%, 6.26%]) in 2020-2021, are violations of these norms. Most anti-social behaviors remain unmoderated: moderators only removed one in twenty violating comments in 2016, and one in ten violating comments in 2020. Personal attacks were the most prevalent category of norm violation; pornography and bigotry were the most likely to be moderated, while politically inflammatory comments and misogyny/vulgarity were the least likely to be moderated. This paper offers a method and set of empirical results for tracking these phenomena as both the social practices (e.g., moderation) and technical practices (e.g., design) evolve.
An Empirical Study on the Usage of Automated Machine Learning Tools
Venue: arXiv cs.SE 2022
Year: 2022

The popularity of automated machine learning (AutoML) tools in different domains has increased over the past few years. Machine learning (ML) practitioners use AutoML tools to automate and optimize the process of feature engineering, model training, and hyperparameter optimization and so on. Recent work performed qualitative studies on practitioners' experiences of using AutoML tools and compared different AutoML tools based on their performance and provided features, but none of the existing work studied the practices of using AutoML tools in real-world projects at a large scale. Therefore, we conducted an empirical study to understand how ML practitioners use AutoML tools in their projects. To this end, we examined the top 10 most used AutoML tools and their respective usages in a large number of open-source project repositories hosted on GitHub. The results of our study show 1) which AutoML tools are mostly used by ML practitioners and 2) the characteristics of the repositories that use these AutoML tools. Also, we identified the purpose of using AutoML tools (e.g. model parameter sampling, search space management, model evaluation/error-analysis, Data/ feature transformation, and data labeling) and the stages of the ML pipeline (e.g. feature engineering) where AutoML tools are used. Finally, we report how often AutoML tools are used together in the same source code files. We hope our results can help ML practitioners learn about different AutoML tools and their usages, so that they can pick the right tool for their purposes. Besides, AutoML tool developers can benefit from our findings to gain insight into the usages of their tools and improve their tools to better fit the users' usages and needs.