Reviewerly/exHarmony huggingface dataset:
Mosaic: Extremely Low-resolution RFID Vision for Visually-anonymized Action Recognition

Despite the potential of vision-based personal monitoring (e.g., healthcare), private data leakage concerns hinder its wide deployment in personal spaces (e.g., bedrooms). A body of data anonymization designs was proposed throughout image processing and federated learning. They commonly store high-quality images and videos locally, which are anonymized via post-processing before cloud upload. However, the recent IoT camera hacking and local data leakage call for anonymized data at the sensing stage. Also, continuous and pervasive monitoring without blind spots in complicated indoor spaces requires a scalable and economic system. This paper present Mosaic, a vision-based end-to-end action recognition framework that (i) intrinsically achieves data anonymity from the sensing stage and (ii) battery-free operation for blind spot-free continuous monitoring. Mosaic leverages an extremely low resolution (eLR) Near-Infrared (NIR) image sensor with 6 × 10 pixels for video anonymity and RFID-compliant fully-passive tag with four solar cells for real-time eLR video streaming under as low as 50 lux (e.g., deep in the shelf without direct light). This is accompanied by light-weight action recognition neural network for real-time inference (18.4ms on Intel(R) Core i7-8700). Mosaic achieves an average of 98% accuracy on 10 action classes, hitting the balance between data anonymity and high-precision action recognition. By taking advantage of NIR (non-visible) frequency, Mosaic also works in dark without disturbing sleep. Lastly, wildfire detection reaching 20m was demonstrated, showcasing the potential for outdoor monitoring.
MicroDeblur: Image Motion Deblurring on Microcontroller-based Vision Systems

This paper introduces MicroDeblur, an on-device image motion deblur solution for resource-constrained microcontroller-based vision systems. Although motion blurs caused by the movement or shake of the device (camera) are pervasive in embedded, IoT, and mobile devices, it has been considered a hard nut to crack for many microcontrollers with extremely-limited resources (e.g., hundreds of KB of RAM). To tackle this problem, we combine the DNN (deep neural network) motion deblur method with the classical motion deblur approach and take the best of both worlds, i.e., 1) powerful pattern recognition ability of DNNs and 2) simplicity and stability of matrix-based classical algorithms. To deblur an image, MicroDeblur takes three steps: 1) blur kernel estimation, 2) blur image transformation, and 3) iterative clear image restoration. We propose 1) depth-independent convolution that efficiently estimates the blur kernel (pattern) and 2) Toeplitz-based motion blur modeling that enhances the time and space complexity of the deblurring process by and , respectively, compared to the existing methods. To the best of our knowledge, MicroDeblur is the first self-sufficient blind deconvolution solution for a stand-alone microcontroller that does not rely on extra hardware or external systems. We implement MicroDeblur on an ARM Cortex-M4F, achieving a competitive quality of deblurred images using 187x and 429x smaller memory and energy, respectively, compared to high-end GPU-based solutions.
Poster Abstract: A Testbed for Context Representation in Physical Spaces

The Internet of Things (IoT) offers transformative potential when combined with Machine Learning (ML), but labeling diverse IoT data remains challenging. To address this, we introduce SenseScape Testbed, an IoT experimentation platform for indoor environments with wireless sensor nodes, robots, and location-tracking nodes. This testbed enables IoT applications such as human activity recognition and indoor mobility tracking, supporting energy efficiency, occupant comfort, and context representation while providing a versatile environment for labeling and testing to advance ML algorithms tailored for IoT applications.
A SURVEY ON DETECTION OF LEUKEMIA USING IMAGE PROCESSING TECHNIQUES

Leukemia is a class of illnesses that often affects the blood and is sometimes referred to as blood cancer.Leukemia is a malignancy of white blood cells.If not caught in time, it could be dangerous.Hematologists examine microscopic pictures to find cancer.The procedure of detection becomes extremely difficult and timeconsuming.If discovered earlier, leukemia can be treated.However, the conventional approach causes the late identification of cancerous cells.Therefore, a machine-learning strategy might be utilised to lower the number of deaths brought on by late detection.In this article, 16 research papers are discussed.This review study's goal is to examine the literature on leukemia identification.and classification utilising deep learning, machine learning, and other image processing methods.
Poster Abstract: Fair Training of Multiple Federated Learning Models on Resource Constrained Network Devices

Federated learning (FL) is an increasingly popular form of distributed learning across devices such as sensors and smartphones. To amortize the effort and cost of setting up FL training in real world systems, in practice multiple machine learning tasks may be trained during one FL execution. However, given that the tasks have varying complexities, naïve methods of allocating resource-constrained devices to work on each task may lead to highly variable performance across the tasks. We instead propose an α -fair based allocation algorithm that dynamically allocates tasks to users during multi-model FL training, based on the prevailing loss levels.
POS: An Operator Scheduling Framework for Multi-model Inference on Edge Intelligent Computing

Edge intelligent applications, such as autonomous driving usually deploy multiple inference models on resource-constrained edge devices to execute a diverse range of concurrent tasks, given large amounts of input data. One challenge is that these tasks need to produce reliable inference results simultaneously with millisecond-level latency to achieve real-time performance and high quality of service (QoS). However, most of the existing deep learning frameworks only focus on optimizing a single inference model on an edge device. To accelerate multi-model inference on a resource-constrained edge device, in this paper we propose POS, a novel operator-level scheduling framework that combines four operator scheduling strategies. The key to POS is a maximum entropy reinforcement learning-based operator scheduling algorithm MEOS, which generates an optimal schedule automatically. Extensive experiments show that POS outperforms five state-of-the-art inference frameworks: TensorFlow, PyTorch, TensorRT, TVM, and IOS, by up to 1.2 × ∼ 3.9 × inference speedup consistently, with 40% improvement on GPU utilization. Meanwhile, MEOS reduces the scheduling overhead by 37% on average, compared to five baseline methods including sequential execution, dynamic programming, greedy scheduling, actor-critic, and coordinate descent search algorithms.
Demo Abstract: BiGuide: A Bi-level Data Acquisition Guidance for Object Detection on Mobile Devices

Real-time object detection (OD) is a key enabling technology for a wide range of emerging mobile system applications. However, deploying an OD model pre-trained on a public dataset (source domain) in a specific local environment (target domain) is known to lead to significant performance degradation because of the so-called domain gap between the dataset and the environment. Collecting local data and fine-tuning the OD model on this data is a commonly used approach for improving the robustness of OD models in real-world deployments. Yet, the question of how to collect this data is currently largely overlooked; unsupported data collection is likely to produce datasets that contain significant proportion of redundant or uninformative data for model training. In this demo, we present BiGuide, a bi-level image data acquisition guidance for OD tasks, to guide users to change their camera locations or angles to different extents (significantly or slightly) to obtain the data which benefits model training via image-level and object instance-level guidance. We showcase an interactive demonstration of collecting data for a lemur species detection application we are developing and deploying at the Duke Lemur Center. Demo participants will take pictures of lemur toys with the mobile phone under the real-time guidance and will observe the real-time display of the metrics that assess the importance of the captured data. They will develop an intuition for how real-time image importance assessment and bi-level guidance improve the quality of collected data.
Demo Abstract: Leveraging Side-Channels to Turn Processors into Low Overhead Radios

Traditionally, side channels have been exploited to uncover sensitive information such as cryptographic keys from computing devices. An attacker can extract private information from a device's processor and memory by using electromagnetic (EM) radiation while code and data are being executed. Rather than seeing side-channels and electromagnetic radiation as vulnerabilities, we consider them as potential wireless communication channels for resource-constrained devices. The main advantage of this approach is completely eliminating the need for a conventional radio and antenna, which offers energy savings, simplicity, and flexibility for resource-constrained devices.
Zero-Shot Domain Adaptation of Anomalous Samples for Semi-Supervised Anomaly Detection

Semi-supervised anomaly detection (SSAD) is a task where normal data and a limited number of anomalous data are available for training. In practical situations, SSAD methods suffer adapting to domain shifts, since anomalous data are unlikely to be available for the target domain in the training phase. To solve this problem, we propose a domain adaptation method for SSAD where no anomalous data are available for the target domain. First, we introduce a domain-adversarial network to a variational auto-encoder-based SSAD model to obtain domain-invariant latent variables. Since the decoder cannot reconstruct the original data solely from domain-invariant latent variables, we conditioned the decoder on the domain label. To compensate for the missing anomalous data of the target domain, we introduce an importance sampling-based weighted loss function that approximates the ideal loss function. Experimental results indicate that the proposed method helps adapt SSAD models to the target domain when no anomalous data are available for the target domain.
String Diagrams with Factorized Densities

A growing body of research on probabilistic programs and causal models has highlighted the need to reason compositionally about model classes that extend directed graphical models. Both probabilistic programs and causal models define a joint probability density over a set of random variables, and exhibit sparse structure that can be used to reason about causation and conditional independence. This work builds on recent work on Markov categories of probabilistic mappings to define a category whose morphisms combine a joint density, factorized over each sample space, with a deterministic mapping from samples to return values. This is a step towards closing the gap between recent category-theoretic descriptions of probability measures, and the operational definitions of factorized densities that are commonly employed in probabilistic programming and causal inference.
Making contract users safer: Towards building a Safe Browsing platform on Ethereum

There are many papers and tools regarding the detection of unsafe contracts, but few ways for detection results to practically benefit contract users and owners. This paper presents a Blockchain Safe Browsing (BSB) platform to safely disseminate those detection results. An encrypted blacklist will be generated to provide privacy preserving user warning before they make transactions with unsafe contracts. Contract owners will be notified that there are vulnerabilities in their contracts, and they can purchase related reports which record how to exploit the vulnerabilities. The profits inspire the researchers to contribute their update-to-date lists of unsafe contracts. An effective encryption scheme is developed to guarantee that only contract owners can decrypt the encrypted reports. Extensive evaluations demonstrate that our prototype can function as intended without sacrificing user experience.
Boosting generalized zero-shot learning with category-specific filters

Zero-Shot Learning (ZSL) has made significant progress driven by deep learning and is being promoted further with the advent of generative models. Despite the success of these methods, the type and number of unseen categories are nailed in the generative models, which makes it challenging to recognize unseen categories in an incremental manner, and the profits of some superior performance algorithms largely arise from their advanced capability of feature extraction, such as Transformers. This paper rigidly follows the assumptions introduced in conventional ZSL and proposes a visual feature filtering method based on a semantic mapping model, namely, filtering visual features through class-specific filters to effectively remove class-agnostic information. Extensive experiments are conducted on four benchmark datasets and have achieved very competitive performance.
An approach for Correcting the Word-level Mispronunciations for non-native English-speaking Indian Children

Speech Recognition and its potential applications in terms of “talking devices” have become indispensable in today’s world. Technological advances like mobiles, smart home assistants or tablets extensively use the techniques of automatic speech recognition that works good for adults but cannot always follow and understand children’s speech. The primary goal of this paper is to bridge the gap of communication between voice assistants and Indian children speaking English as secondary language. The issue of lack of children’s speech corpora with English as non-native language, is addressed by creating a dataset of children in the age group of 5-15 years, speaking Hindi or Marathi as their mother tongue and English as their second language. The analysis and implementation of the proposed work shows the accuracy of approximately 96% and potential for further scope by increasing the size of dataset in lower age group. The key contributions of our work are (i) creating speech dataset of Indian children whose mother-tongue is Hindi or Marathi, (ii) employing and evaluating hybrid Convolutional Neural Network (CNN) as an age classifier, (iii) language modeling to customize children vocabulary, (iv) checking accuracy and performance of the system.
Precise modeling of learning process based on multiple behavioral features for knowledge tracing

With the increase in needs for personalized learning of online students, knowledge tracing (KT), a technique aimed at tracing the state of a student’s knowledge mastery and predicting performance in future exercises, has become a hot topic in personalized learning research. The behavioral features exhibited during students’ learning process bear information that impacts the state of a student’s knowledge mastery. To study the influence of learning behaviors on students’ knowledge mastery state in the learning process, we propose a Precise Modeling of Learning Process based on Multiple Behavioral Features for Knowledge Tracing model (MBFKT), which models a student’s learning process by making use of these behavioral features. MBFKT initially processes these features through multi-head attention networks, memory networks, and recurrent neural networks to model students’ learning process into three memory links: memory decline link, memory enhancement link, and memory update link. Various update strategies are designed for each memory link, and the performance of numerous possible combinations of behavioral features in the memory links is compared, for the rules of learning and forgetting to be explained. Furthermore, we also study the contribution and degree of influence of different behavioral features on a student’s knowledge mastery state, by which MBFKT is improved, thus enhancing the accuracy of prediction. Through experiments on real online education datasets and comparison with existing benchmark methods, it is observed that MBFKT has evident advantages in predicting performance with good interpretability.
A novel ensemble model for identification and classification of cyber harassment on social media platform

Schools and universities shuttered as a result of the worldwide COVID-19 pandemic lockdown, and student screen time skyrocketed. Since the programs are delivered online, a spike in social media use during lockdown resulted in many pupils becoming victims of cyberbullying, which includes criticizing one another, posting sexual comments on images of young ladies, and using fake accounts to bully others. Machine Learning (ML) and Natural Language Processing (NLP) techniques are being used in a growing body of work on automated cyberbullying detection. Different machine learning methods, however, are unable to converge to the requisite accuracy. Thus, numerous classifier systems known as “ensemble learning” are proposed in order to improve predictive performance by aggregating the predictions from various models. In our proposed system, we use a novel method of detecting online harassment (cyberbullying) on the Instagram dataset. The attributes of abusive words are initially analyzed from feature selection and pre-trained word embedding language models like Bidirectional Encoder Representations from Transformers (BERT) and Embeddings from Language Models (ELMO). A knowledge-based frequent pattern method is used to find the intention of the harasser and is created by the Knowledge-BERT (K-BERT). The unsupervised approaches such as Latent Semantic Analysis (LSA), Frequent pattern growth (FP-Growth), and a clustering technique K-Means. The results from the detection models are ensembled using Extreme Gradient Boosting (XGBoost) for classifying the categories of online harassment. The performance of the ensemble model is then cross-validated using machine learning metrics and compared with various existing techniques. An ensemble model performs better with a higher F1 score of 92.04% with less error rate in the classification of harassment categories.
Regulating confidence by corner discrepancy and center score in corner-based object detection methods

This paper proposes an approach that regulates the confidence of predicted boxes for corner-based detection methods. Corner-based methods have achieved state-of-the-art performance on MS-COCO by predicting corners and grouping them to generate boxes. However, the box confidence is simply defined to be the average score of grouped corners, ignoring the score and tag discrepancy between them. The discrepancy may lead to the generation of more false positives (FPs) since a larger discrepancy often means that the grouped corners less likely belong to the same object. Observing this, this paper proposes introducing the discrepancy of corners (DoC) to decrease the box confidence. Also, the score and location of center (SLoC) of a detection box is utilized to further finely regulate the confidence. DoC and SLoC can effectively reduce FPs and missings and hence improve the detection performance without changing any model parameter. Experimental results on MS-COCO also show improvements.
A new fractional-order variational approach for joint de-hazing and segmentation of images

Segmenting outdoor images in the presence of haze, fog or smog (which fades the colors and diminishes the contrast of the observed objects) has been a challenging task in image processing with several important applications. In this paper, we propose a new fractional-order variational model that will be able to de-haze and segment a given image simultaneously. The proposed method incorporates the atmospheric veil estimation based on the dark channel prior (DCP). This transmission map can reduce significantly the edge artifacts and enhance estimation precision in the resulting image. The transmission map is then changed over to the high-quality depth map, with which the new fractional-order variational model can be framed to look for the haze free segmenting image for both grey and color outdoor images. An explicit gradient descent scheme is employed to find efficiently the minimizer of the proposed energy functional. Experimental tests on real world scenes show that the proposed method can jointly de-haze and segment hazy or foggy images effectively and efficiently.
PLDH: Pseudo-Labels Based Deep Hashing

Deep hashing has received a great deal of attraction in large-scale data analysis, due to its high efficiency and effectiveness. The performance of deep hashing models heavily relies on label information, which is very expensive to obtain. In this work, a novel end-to-end deep hashing model based on pseudo-labels for large-scale data without labels is proposed. The proposed hashing model consists of two major stages, where the first stage aims to obtain pseudo-labels based on deep features extracted by a pre-training deep convolution neural network. The second stage generates hash codes with high quality by the same neural network in the previous stage, coupled with an end-to-end hash layer, whose purpose is to encode data into a binary representation. Additionally, a quantization loss is introduced and interwound within these two stages. Evaluation experiments were conducted on two frequently-used image collections, CIFAR-10 and NUS-WIDE, with eight popular shallow and deep hashing models. The experimental results show the superiority of the proposed method in image retrieval.