Reviewerly/exHarmony huggingface dataset:
Multivariate Feature Extraction

We propose a general framework to extract features for learning problems involving multiple variables. First, we decompose multivariate dependence into different components to obtain the piece relevant to the learning task. Then, we establish a modal decomposition approach to represent the component as informative features. We further demonstrate the algorithm design for extracting such features from real data, which can incorporate and utilize existing deep feature extractors. We also present an application of our framework in learning tasks with side information.
Practical Control Design for the Deep Learning Age: Distillation of Deep RL-Based Controllers

Deep Reinforcement Learning (RL) methods produce often performance-leading controllers. Yet, in many applications (e.g., when safety and assurance are critical), they cannot be deployed. Deep Neural Nets (DNN) controllers are nearly impossible to verify and their behavior hard to interpret. In this paper, we first show, through a simple example, that there exist soft decision tree (SDT) controllers that are equivalent to neural network (NN) controllers in terms of input-output behavior. Based on imitation learning, we then show via three OpenAI gym environment case studies that it is possible to distill high-performance DNN controllers into non-DNN controllers (e.g., leveraging decision tree and support vector machine architectures) that, while sacrificing performance just a bit, can be simpler, more interpretable, hence amenable to verification and validation. We thus propose the natural control design paradigm that leverages the power of deep RL methods to design reference DNN controllers, and then distill them into non-DNN controllers which can be validated and deployed in real systems. Finally, we identify some distillation metrics that can be useful in assessing the quality of the distilled controllers.
Stabilizing Skeletal Pose Estimation using mmWave Radar via Dynamic Model and Filtering

In this paper, we illustrate a method to stabilize the position estimation of human skeleton using mmWave radar. In our previous study, an optimized CNN architecture was used to extract the positions of human skeleton accurately. However, the position estimation of the joints vibrates over time. In the field of digital signal processing, filters are used to remove unwanted parts of signal and widely applied in noise reduction, radar, audio and video processing, etc. In this paper, three types of filters i.e. Elliptic, Savitzky-Golay, and Whittaker-Eilers are discussed and applied to both positions and angles of the human skeleton. This paper further presents a humanoid robotics dynamic model, specifically forward kinematics, to recalculate joint positions with improved stability. We define the root joint, a world coordinate system, and “T” pose, to get the subsequent joints' rotation matrix using kinematics chain of the skeleton, then compute the Euler angles. After the filtering, we compare the effect of different filters using a method of Standard Deviation (SD) of the angle slope. In addition, we analyze the change of localization accuracy after recalculating the positions using forward kinematics based on the current angle, root position, and bone length information. The data collection and experimental evaluation have shown a motion stability improvement of 54.05% compared to the CNN predicted value.
Design and Implementation of Blockchain-based College Education Integrity System

The paper firstly takes the requirements of the college education integrity system (CEIS) issued by the local education authority as the research object, proposes a restricted blockchain model with auditing and its consensus algorithm, and then proposes two improved MPT tree search algorithms to improve the search efficiency. Finally, the system adopts the Ethereum blockchain for system design, implementation and verification. The system is deployed in the NodeJS, with the Truffle framework to configure and write smart contracts to create decentralized DApp. The front-end is built with LAYUI, and the back-end is written in Solidity for smart contracts. MongoDB and Redis are used for data interaction with DApp. Ganache is used to build an Ethereum simulation network for system testing and realize the application and transfer of Ethereum in units of accounts. The blockchain-based college integrity system can solve the problems of high cost, low efficiency and data reliability risks in centralized systems.
Online education course recommendation algorithm based on path factors

With the development of online education, reasonable course recommendation in online environment is becoming more and more important. In order to obtain the expected recommendation results, an online education course recommendation algorithm based on path factor is introduced in this paper. Based on the traditional recommendation algorithms, the path factors and the adaptive weight threshold of adjacent courses are introduced in the proposed algorithm to improve the similarities of candidate courses. Case analysis shows that the proposed recommendation algorithm achieves a more reasonable recommendation results under the online learning environment.
Strange Images with Non-monotonous Rate-Distortion Curves in Lossy Image Compression

This paper deals with lossy compression of grayscale and color images. A traditional assumption for any compression technique and any image is that a higher compression ratio leads to worse quality of any compressed image according to any quality metric, either standard one as MSE or PSNR or a visual quality metric. Meanwhile, it has been discovered recently that there are grayscale images (called “strange” by us) that are characterized by non-monotonous behavior of rate-distortion curves like dependence of PSNR on quantization step for the DCT-based coder AGU. In this paper, we show that there are also color images that can be identified as strange. Non-monotonicity of rate/distortion curve might also happen for JPEG coder. Moreover, non-monotonicity can be observed for the dependences of visual quality metrics on compression control parameters such as quality factor, bpp (bit per pixel), and quantization step. Such behavior can cause problems in providing the desired quality in lossy image compression. Then, “strange” images have to be detected in advance and processed (compressed) with care. We show that “strange” images are mainly those that have large homogeneous areas (background). They can be detected by entropy calculation and its comparison to a certain threshold. Examples of “strange” images and rate/distortion curves for them are presented.
Comparative analysis on different convolutional neural network (CNN) for classification

CNN is one of the representative algorithms of deep learning. With the development of theory and the improvement of numerical computing equipment, CNN has developed rapidly. A variety of models have been derived and applied to different places. Taking the classification of vegetables and fruits as the data reference, this paper studies three CNN models: AlexNet, VGG and NIN. Our purpose is to analyze which model is most suitable for this image classification task and compare the advantages and disadvantages of the three models. By adjusting different hyper parameters; relative learning rate (Lr), epochs(#epochs) and batch size(batch _size) on the training loss, train accuracy and test accuracy among three models. The VGGNet with Lr = 0.00001, #epochs = 80 and batch_size = 32 has the best qualified for this graphic classification task.
Improving Proof of Stake Economic Security via MEV Redistribution

Maximal Extractable Value (MEV) has generally been viewed as a negative, parasitic aspect of economic transactions on blockchains that increases costs for non-strategic users. Recent work has shown that MEV is not always bad for social welfare in crypto networks. In this note, we demonstrate that if rational validators in Proof of Stake (PoS) protocols are able to earn a portion of MEV revenue, by a process we call MEV redistribution, they are disincentivized to unstake and lower economic security. We construct a joint staking-lending dynamical system in which a fraction of MEV revenue is used to increase staking returns. We formally show that this MEV redistribution can avoid bad competitive equilibria between staking and lending in which no users stake under benign conditions on the reward inflation schedule of the protocol, and conduct numerical simulations that demonstrate this. This represents another potentially positive externality of MEV, provided that the mechanism for redistribution is well-designed.
Real-Time Scheduling of TrustZone-enabled DNN Workloads

Limited resources in embedded devices often hinder the execution of computation-heavy machine learning processes. Running deep neural network (DNN) workloads while preserving the integrity of the model parameters and without compromising temporal constraints of real-time applications, is a challenging problem. Although secure enclaves such as ARM TrustZone can ensure the integrity of applications, off-the-shelf implementations are often infeasible for DNN workloads - especially those with real-time requirements - due to additional resource and temporal constraints. This paper presents a real-time scheduling framework that enables the execution of resource-intensive DNN workloads inside TrustZone-enabled secure enclaves. Our approach reduces the resource overhead by fusing multiple layers of multiple tasks and running them all together inside the enclaves while retaining real-time grantees. We derive mathematical conditions that will allow the designer to test the feasibility of deploying DNN workload in a TrustZone-enabled system. Our comparisons with a standard fixed-priority real-time scheduler show that we can schedule up to 21.33% more tasksets in higher utilization (e.g., > 80%) scenarios.
Cryptocurrency: Its Impact and Forensic Worth

The popularity of Bitcoin and other cryptocurrencies has increased over the past few years. Although some find the idea of digital currency to be complex and untrustworthy, many others see opportunities for significant economic development and anonymity. Anonymity is one opportunity that draws those who seek to conceal their criminal behavior through illicit activities such as cybercrime and ransomware attacks. Accordingly, digital forensic analysts must now add cryptocurrency to their list of concerns as they further refine their craft of bringing forensic science into the courtroom. Under the Frye or Daubert standards, these analyses must be valid and reliable. This paper offers a background on the impact of cryptocurrencies and their association with digital forensics.
DynaTags: Low-Cost Fiducial Marker Mechanisms

Printed fiducial markers are inexpensive, easy to deploy, robust and deservedly popular. However, their data payload is also static, unable to express any state beyond being present. For this reason, more complex electronic tagging technologies exist, which can sense and change state, but either require special equipment to read or are orders of magnitude more expensive than printed markers. In this work, we explore an approach between these two extremes: one that retains the simple, low-cost nature of printed markers, yet has some of the expressive capabilities of dynamic tags. Our “DynaTags” are simple mechanisms constructed from paper that express multiple payloads, allowing practitioners and researchers to create new and compelling physical-digital experiences. We describe a library of 23 mechanisms that can be read by standard smartphone reader apps. Through a series of demo applications (augmenting reality through e.g., sounds, environmental lighting and graphics) we show how our tags can bring new interactivity to previously static experiences.
Sound Scope Pad: Controlling a VR Concert with Natural Movement

We developed Sound Scope Pad, an application that provides an active music listening experience that combines AI, virtual reality, and spatial acoustics. Users can emphasize the sounds of certain performers by turning their head to the left or right or bringing their hands closer to their ears to find and focus on the performer they want to listen to. In the Sound Scope Headphones that we previously built, the user’s head direction was detected by an accelerometer mounted on the arch of the headphones. In Sound Scope Pad, the head direction is detected by combining the angle information detected by the acceleration gyro sensor of a tablet and the angle information of the head recognized from the front camera image of the tablet.
Enabling Non-Technical Domain Experts to Create Robot-Assisted Therapeutic Scenarios via Visual Programming

In this paper, we present a visual programming software for enabling non-technical domain experts to create robot-assisted therapy scenarios for multiple robotic platforms. Our new approach is evaluated by comparing it with Choregraphe, the standard visual programming framework for the often used robotics platforms Pepper and NAO. We could show that our approach receives higher usability values and allows users to perform better in some practical tasks, including understanding, changing and creating small robot-assisted therapy scenarios.
Switchable CNN-Based Same-Resolution and Super-Resolution In-Loop Restoration for Next Generation Video Codecs

We present a common framework for in-loop same-resolution and super-resolution restoration for incorporation into a next-generation video codec. Building on the in-loop filtering pipeline in the AV1 video codec from the Alliance for Open Media (AOM), we first enhance it to support symmetric spatial down-up scaling with better down and upscaling filters, followed by adding switchable frame-level CNNs to restore the reconstructed frames. Furthermore, the architectures for the CNNs used are constrained to be very simple with a relatively small number of parameters and multiply-add operations per decoded pixel to make them practically feasible in hardware. Preliminary results are presented on test sets being used by AOM for their ongoing next-generation video codec (AV2) development effort.
Diverse Generative Perturbations on Attention Space for Transferable Adversarial Attacks

Adversarial attacks with improved transferability –the ability of an adversarial example crafted on a known model to also fool unknown models –have recently received much attention due to their practicality. Nevertheless, existing transferable attacks craft perturbations in a deterministic manner and often fail to fully explore the loss surface, thus falling into a poor local optimum and suffering from low transferability. To solve this problem, we propose Attentive-Diversity Attack (ADA), which disrupts diverse salient features in a stochastic manner to improve transferability. Primarily, we perturb the image attention to disrupt universal features shared by different models. Then, to effectively avoid poor local optima, we disrupt these features in a stochastic manner and explore the search space of transferable perturbations more exhaustively. More specifically, we use a generator to produce adversarial perturbations that each disturbs features in different ways depending on an input latent code. Extensive experimental evaluations demonstrate the effectiveness of our method, outperforming the transferability of state-of-the-art methods. Codes are available at https://github.com/wkim97/ADA.
Smile: Sequence-to-Sequence Domain Adaptation with Minimizing Latent Entropy for Text Image Recognition

Excellent text recognition results have been obtained by training recognition models with synthetic images. However, recognizing text from real-world images still faces challenges due to the domain shift between synthetic and real-world text images. One strategy to eliminate this domain difference without manual annotation is unsupervised domain adaptation (UDA). Due to the characteristics of sequential labeling tasks, most popular UDA methods cannot be directly applied to text recognition. To tackle this problem, we proposed a UDA method that minimizes latent entropy on sequence-to-sequence attention-based models with class-balanced self-paced learning. Experimental results show that our proposed framework achieves better recognition results than the existing methods on most UDA text recognition benchmarks. All codes are publicly available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Measuring Class-Imbalance Sensitivity of Deterministic Performance Evaluation Metrics

The class-imbalance issue is intrinsic to many real-world machine learning tasks, particularly to the rare-event classification problems. Although the impact and treatment of imbalanced data is widely known, the magnitude of a metric’s sensitivity to class imbalance has attracted little attention. As a result, often the sensitive metrics are dismissed while their sensitivity may only be marginal. In this paper, we introduce an intuitive evaluation framework that quantifies metrics’ sensitivity to the class imbalance. Moreover, we reveal an interesting fact that there is a logarithmic behavior in metrics’ sensitivity meaning that the higher imbalance ratios are associated with the lower sensitivity of metrics. Our framework builds an intuitive understanding of the class-imbalance impact on metrics. We believe this can help avoid many common mistakes, specially the less-emphasized and incorrect assumption that all metrics’ quantities are comparable under different class-imbalance ratios.
Self-Supervised Class-Cognizant Few-Shot Classification

Unsupervised learning is argued to be the dark matter of human intelligence <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> . To build in this direction, this paper focuses on unsupervised learning from an abundance of unlabeled data followed by few-shot fine-tuning on a downstream classification task. To this aim, we extend a recent study on adopting contrastive learning for self-supervised pre-training by incorporating class-level cognizance through iterative clustering and re-ranking and by expanding the contrastive optimization loss to account for it. To our knowledge, our experimentation both in standard and cross-domain scenarios demonstrate that we set a new state-of-the-art (SoTA) in (5-way, 1 and 5-shot) settings of standard mini-ImageNet benchmark as well as the (5-way, 5 and 20-shot) settings of cross-domain CDFSL benchmark. Our code and experimentation can be found in our GitHub repository: https://github.com/ojss/c3lr.
Relation-Guided Network for Image-Text Retrieval

Image-text retrieval has made great progress, but it remains challenging due to heterogeneity between images and text. Enhancing the interaction by exploring the relationship between the image and text can reduce this problem, to some extent. How to explore and use the relationship between image and text to enhance the interaction between them is a critical problem. In this paper, we design an asymmetric structure network (RGN) to represent image and text. First, we mine the relationship between image and text, and extract the specific text information. Then we exploit this relationship to guide the generation of text embeddings, which can capture the rich and representative embeddings. Results on two datasets, Flickr30K dataset and MSCOCO dataset, show that our model can achieve competitive results.
Efficient Inference Of Image-Based Neural Network Models In Reconfigurable Systems With Pruning And Quantization

Neural networks (NN) for image processing in embedded systems expose two conflicting requirements: increasing computing power needs as models become more complex and constrained resource budget. In order to alleviate this problems, model compression based on quantization and pruning techniques are common. Derived models then need to fit on reconfigurable systems such as FPGAs for the embedded system to work properly. In this paper, we present HLSinf, an open source framework for the development of custom NN accelerators for FPGAs which provides efficient support to quantized and pruned NN models. With HLSinf, significant inference speedups can be obtained for typical medical image-based applications. In particular, we obtain up to 90x speedup factor when we combine quantization/pruning with the flexibility of HLSinf compared to CPU.
Neuro-Inspired Deep Neural Networks with Sparse, Strong Activations

While end-to-end training of Deep Neural Networks (DNNs) yields state of the art performance in an increasing array of applications, it does not provide insight into, or control over, the features being extracted. We report here on a promising neuro-inspired approach to DNNs with sparser and stronger activations. We use standard stochastic gradient training, supplementing the end-to-end discriminative cost function with layer-wise costs promoting Hebbian ("fire together," "wire together") updates for highly active neurons, and anti-Hebbian updates for the remaining neurons. Instead of batch norm, we use divisive normalization of activations (suppressing weak outputs using strong outputs), along with implicit ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> normalization of neuronal weights. Experiments with standard image classification tasks on CIFAR-10 demonstrate that, relative to baseline end-to-end trained architectures, our proposed architecture (a) leads to sparser activations (with only a slight compromise on accuracy), (b) exhibits more robustness to noise (without being trained on noisy data), (c) exhibits more robustness to adversarial perturbations (without adversarial training).
Adversarial Examples for Good: Adversarial Examples Guided Imbalanced Learning

Adversarial examples are inputs for machine learning models that have been designed by attackers to cause the model to make mistakes. In this paper, we demonstrate that adversarial examples can also be utilized for good to improve the performance of imbalanced learning. We provide a new perspective on how to deal with imbalanced data: adjust the biased decision boundary by training with Guiding Adversarial Examples (GAEs). Our method can effectively increase the accuracy of minority classes while sacrificing little accuracy on majority classes. We empirically show, on several benchmark datasets, our proposed method is comparable to the state-of-the-art method. To our best knowledge, we are the first to deal with imbalanced learning with adversarial examples.
Mixture of Teacher Experts for Source-Free Domain Adaptive Object Detection

Unsupervised domain adaptive object detection methods transfer knowledge from the labelled source domain to a visually distinct and unlabeled target domain. Most methods achieve this by training the detector model with the help of both labeled source and unlabeled target data. However, in real-world scenarios, gaining access to source data is not practical due to privacy concerns, legal issues and inefficient data transmission. To this end, we tackle the problem of Source-Free Domain Adaptive Object Detection, where during adaptation, we do not have access to the source data but only the source trained model. Specifically, we introduce Mixture of Teacher Experts (MoTE) method, where our key idea is to exploit the prediction uncertainty through a mixture of teacher models and progressively train the student model. We evaluate the proposed method by conducting extensive experiments on several object detection benchmark datasets to demonstrate the effectiveness of the proposed mixture of teacher expert based student-teacher training, specifically for source-free adaptation.