Reviewerly/exHarmony huggingface dataset:
Exploring the Intersection between Neural Architecture Search and Continual Learning

Despite the significant advances achieved in Artificial Neural Networks (ANNs), their design process remains notoriously tedious, depending primarily on intuition, experience and trial-and-error. This human-dependent process is often time-consuming and prone to errors. Furthermore, the models are generally bound to their training contexts, with no considerations to their surrounding environments. Continual adaptiveness and automation of neural networks is of paramount importance to several domains where model accessibility is limited after deployment (e.g IoT devices, self-driving vehicles, etc.). Additionally, even accessible models require frequent maintenance post-deployment to overcome issues such as Concept/Data Drift, which can be cumbersome and restrictive. By leveraging and combining approaches from Neural Architecture Search (NAS) and Continual Learning (CL), more robust and adaptive agents can be developed. This study conducts the first extensive review on the intersection between NAS and CL, formalizing the prospective Continually-Adaptive Neural Networks (CANNs) paradigm and outlining research directions for lifelong autonomous ANNs.
Evolutionary Multi-Task Injection Testing on Web Application Firewalls

Web application firewall (WAF) plays an integral role nowadays to protect web applications from various malicious injection attacks such as SQL injection, XML injection, and PHP injection, to name a few. However, given the evolving sophistication of injection attacks and the increasing complexity of tuning a WAF, it is challenging to ensure that the WAF is free of injection vulnerabilities such that it will block all malicious injection attacks without wrongly affecting the legitimate message. Automatically testing the WAF is, therefore, a timely and essential task. In this paper, we propose DaNuoYi, an automatic injection testing tool that simultaneously generates test inputs for multiple types of injection attacks on a WAF. Our basic idea derives from the cross-lingual translation in the natural language processing domain. In particular, test inputs for different types of injection attacks are syntactically different but may be semantically similar. Sharing semantic knowledge across multiple programming languages can thus stimulate the generation of more sophisticated test inputs and discovering injection vulnerabilities of the WAF that are otherwise difficult to find. To this end, in DaNuoYi, we train several injection translation models by using multi-task learning that translates the test inputs between any pair of injection attacks. The model is then used by a novel multi-task evolutionary algorithm to co-evolve test inputs for different types of injection attacks facilitated by a shared mating pool and domain-specific mutation operators at each generation. We conduct experiments on three real-world open-source WAFs and six types of injection attacks, the results reveal that DaNuoYi generates up to 3.8x and 5.78x more valid test inputs (i.e., bypassing the underlying WAF) than its state-of-the-art single-task counterparts and the context-free grammar-based injection construction.
Robust Time Series Denoising with Learnable Wavelet Packet Transform

Signal denoising is a key preprocessing step for many applications, as the performance of a learning task is closely related to the quality of the input data. In this paper, we apply a signal processing based deep neural network architecture, a learnable extension of the wavelet packet transform. As main advantages, this model has few parameters, an intuitive initialization and strong learning capabilities. Moreover, we show that it is possible to easily modify the parameters of the model after the training step to tailor to different noise intensities. Two case studies are conducted to compare this model with the state of the art and commonly used denoising procedures. The first experiment uses standard signals to study denoising properties of the algorithms. The second experiment is a real application with the objective to remove audio background noises. We show that the learnable wavelet packet transform has the learning capabilities of deep learning methods while maintaining the robustness of standard signal processing approaches. More specifically, we demonstrate that our approach maintains excellent denoising performances on signal classes separate from those used during the training step. Moreover, the learnable wavelet packet transform was found to be robust when different noise intensities, noise varieties and artifacts are considered.
Bipartite instances of INFLUENCE

The game INFLUENCE is a scoring combinatorial game that has been introduced in 2020 by Duchene et al. It is a good representative of Milnor's universe of scoring games, i.e. games where it is never interesting for a player to miss his turn. New general results are first given for this universe, by transposing the notions of mean and temperature derived from non-scoring combinatorial games. Such results are then applied to INFLUENCE to refine the case of unions of segments. The computational complexity of the score of the game is also solved and proved to be PSPACE-complete. We finally focus on some specific cases of INFLUENCE when the graph is bipartite, by giving explicit strategies and bounds on the optimal score on structures like grids, hypercubes or torus.
Memorization-Dilation: Modeling Neural Collapse Under Label Noise

The notion of neural collapse refers to several emergent phenomena that have been empirically observed across various canonical classification problems. During the terminal phase of training a deep neural network, the feature embedding of all examples of the same class tend to collapse to a single representation, and the features of different classes tend to separate as much as possible. Neural collapse is often studied through a simplified model, called the unconstrained feature representation, in which the model is assumed to have "infinite expressivity" and can map each data point to any arbitrary representation. In this work, we propose a more realistic variant of the unconstrained feature representation that takes the limited expressivity of the network into account. Empirical evidence suggests that the memorization of noisy data points leads to a degradation (dilation) of the neural collapse. Using a model of the memorization-dilation (M-D) phenomenon, we show one mechanism by which different losses lead to different performances of the trained network on noisy data. Our proofs reveal why label smoothing, a modification of cross-entropy empirically observed to produce a regularization effect, leads to improved generalization in classification tasks.
Generation of photons from vacuum in cavity via time-modulation of a qubit invisible to the field

We propose a scheme for generation of photons from vacuum due to time-modulation of a quantum system coupled indirectly to the cavity field through some ancilla quantum subsystem. We consider the simplest case when the modulation is applied to an artificial 2-level atom (we call t-qubit), while the ancilla is a stationary qubit coupled via the dipole interaction both to the cavity and t-qubit. We find that tripartite entangled states with a small number of photons can be generated from the system ground state under resonant modulations, even when the t-qubit is far detuned from both the ancilla and the cavity, provided its bare and modulation frequencies are properly adjusted as function of other system parameters. We attest our approximate analytic results by numeric simulations and show that photon generation from vacuum persists in the presence of common dissipation mechanisms.
Convergence for score-based generative modeling with polynomial complexity

Score-based generative modeling (SGM) is a highly successful approach for learning a probability distribution from data and generating further samples. We prove the first polynomial convergence guarantees for the core mechanic behind SGM: drawing samples from a probability density $p$ given a score estimate (an estimate of $\nabla \ln p$) that is accurate in $L^2(p)$. Compared to previous works, we do not incur error that grows exponentially in time or that suffers from a curse of dimensionality. Our guarantee works for any smooth distribution and depends polynomially on its log-Sobolev constant. Using our guarantee, we give a theoretical analysis of score-based generative modeling, which transforms white-noise input into samples from a learned data distribution given score estimates at different noise scales. Our analysis gives theoretical grounding to the observation that an annealed procedure is required in practice to generate good samples, as our proof depends essentially on using annealing to obtain a warm start at each step. Moreover, we show that a predictor-corrector algorithm gives better convergence than using either portion alone.
Utilization of deep learning and semantic analysis for opinion mining in information extraction: a review

In concern of the increasing availability and popularity of the opinion information sources at a different platform like individual blogs, online feedback, and social network are proliferating and gaining new opportunities and challenges that can be actively exploited using information technology to seek and comprehend people's opinions. In today's field people or entrepreneurs before taking any decision they must be considering the opinion of peoples or information networks. Most of them express the view or opinion through social media platform like Tweeter, Facebook, or blogs on the internet. Therefore, it is essential to analyze to automatically analyze the immense amount of social data available on the internet. Deep learning (DL) has appeared as an influential machine learning (ML) method that studies the properties of different layers or data and provides more advanced predictive results. The study of DL, along with success in many other practical fields, has been widely used in opinion and emotion analysis in recent years. This review explores new challenges in brainstorming and facilitates DL study and the use of semantic analysis. The analysis focuses mainly on methods that try to solve new challenges identified by empathy programs, compared to those already in place. It also conducts extensive research on its current uses in emotion analysis.
Breast invasive ductal carcinoma diagnosis using machine learning models and Gabor filter method of histology images

Breast cancer is the most common type of cancer in women and the leading cause of death from a malignant growth in the world. Machine learning methods have been created to help with cancer detection accuracy. There are several methods for detecting cancer. Histopathological images are more accurate. In this study, we employed the Gabor filter to extract statistical features from invasive ductal carcinoma histopathology images. From the histopathological images, we chose 100, 200, 400, 1000, and 2000 at random. These statistical features were used to train several models to classify these images as malignant or benign, including the decision tree, quadratic discriminant analysis, extra randomized trees, gradient boosting, Gaussian process, Naive Bayes, nearest centroid, multilayer perceptron, and support vector machine. The models' accuracy, sensitivity, specificity, precision, and F1_score were examined. The models produced the highest results when there were 100 images and a wavenumber of 0.2. While as the number of images increased, the models' effectiveness reduced. The most obvious finding to emerge from this study is that we suggest using deep learning instead of machine learning models for large datasets.
Investigating data partitioning strategies for crosslinguistic low-resource ASR evaluation

Many automatic speech recognition (ASR) data sets include a single pre-defined test set consisting of one or more speakers whose speech never appears in the training set. This "hold-speaker(s)-out" data partitioning strategy, however, may not be ideal for data sets in which the number of speakers is very small. This study investigates ten different data split methods for five languages with minimal ASR training resources. We find that (1) model performance varies greatly depending on which speaker is selected for testing; (2) the average word error rate (WER) across all held-out speakers is comparable not only to the average WER over multiple random splits but also to any given individual random split; (3) WER is also generally comparable when the data is split heuristically or adversarially; (4) utterance duration and intensity are comparatively more predictive factors of variability regardless of the data split. These results suggest that the widely used hold-speakers-out approach to ASR data partitioning can yield results that do not reflect model performance on unseen data or speakers. Random splits can yield more reliable and generalizable estimates when facing data sparsity.
A Probabilistic Interpretation of Transformers

We propose a probabilistic interpretation of exponential dot product attention of transformers and contrastive learning based off of exponential families. The attention sublayer of transformers is equivalent to a gradient ascent step of the log normalizer, which is the log-sum-exp term in the Hopfield theory of attention. This ascent step induces a parallel expansion of points, which is counterbalanced by a contraction from layer normalization. We also state theoretical limitations of our theory and the Hopfield theory and suggest directions for resolution.
Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance

We propose Guided-TTS, a high-quality text-to-speech (TTS) model that does not require any transcript of target speaker using classifier guidance. Guided-TTS combines an unconditional diffusion probabilistic model with a separately trained phoneme classifier for classifier guidance. Our unconditional diffusion model learns to generate speech without any context from untranscribed speech data. For TTS synthesis, we guide the generative process of the diffusion model with a phoneme classifier trained on a large-scale speech recognition dataset. We present a norm-based scaling method that reduces the pronunciation errors of classifier guidance in Guided-TTS. We show that Guided-TTS achieves a performance comparable to that of the state-of-the-art TTS model, Grad-TTS, without any transcript for LJSpeech. We further demonstrate that Guided-TTS performs well on diverse datasets including a long-form untranscribed dataset.
GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments

We propose GANav, a novel group-wise attention mechanism to identify safe and navigable regions in off-road terrains and unstructured environments from RGB images. Our approach classifies terrains based on their navigability levels using coarse-grained semantic segmentation. Our novel group-wise attention loss enables any backbone network to explicitly focus on the different groups' features with low spatial resolution. Our design leads to efficient inference while maintaining a high level of accuracy compared to existing SOTA methods. Our extensive evaluations on the RUGD and RELLIS-3D datasets shows that GANav achieves an improvement over the SOTA mIoU by 2.25-39.05% on RUGD and 5.17-19.06% on RELLIS-3D. We interface GANav with a deep reinforcement learning-based navigation algorithm and highlight its benefits in terms of navigation in real-world unstructured terrains. We integrate our GANav-based navigation algorithm with ClearPath Jackal and Husky robots, and observe an increase of 10% in terms of success rate, 2-47% in terms of selecting the surface with the best navigability and a decrease of 4.6-13.9% in trajectory roughness. Further, GANav reduces the false positive rate of forbidden regions by 37.79%. Code, videos, and a full technical report are available at https://gamma.umd.edu/offroad/.
Local available quantum correlations of non-symmetric X states

Local available quantum correlations (LAQC), as defined by Mundarain et al., are analyzed for non-symmetric 2-qubit X states, that is, X-states that are not invariant under the exchange of subsystems and therefore have local Bloch vectors whose norms are different. A simple analytic expression for their LAQC quantifier is obtained. As an example, we analyze the local application of the amplitude damping channel for Werner states and general X states. Although this local quantum channel can create quantum discord in some cases, no such outcome is possible for LAQC, which hints toward their monotonicity under LOCC operations. This work, along with our previous result for so-called symmetric and anti-symmetric X states, completes the pursuit of exact analytical expressions for the LAQC quantifier for 2-qubit X states.
Revisiting Distance Metric Learning for Few-Shot Natural Language Classification

Distance Metric Learning (DML) has attracted much attention in image processing in recent years. This paper analyzes its impact on supervised fine-tuning language models for Natural Language Processing (NLP) classification tasks under few-shot learning settings. We investigated several DML loss functions in training RoBERTa language models on known SentEval Transfer Tasks datasets. We also analyzed the possibility of using proxy-based DML losses during model inference. Our systematic experiments have shown that under few-shot learning settings, particularly proxy-based DML losses can positively affect the fine-tuning and inference of a supervised language model. Models tuned with a combination of CCE (categorical cross-entropy loss) and ProxyAnchor Loss have, on average, the best performance and outperform models with only CCE by about 3.27 percentage points -- up to 10.38 percentage points depending on the training dataset.
Distance Metric Learning Loss Functions in Few-Shot Scenarios of Supervised Language Models Fine-Tuning

This paper presents an analysis regarding an influence of the Distance Metric Learning (DML) loss functions on the supervised fine-tuning of the language models for classification tasks. We experimented with known datasets from SentEval Transfer Tasks. Our experiments show that applying the DML loss function can increase performance on downstream classification tasks of RoBERTa-large models in few-shot scenarios. Models fine-tuned with the use of SoftTriple loss can achieve better results than models with a standard categorical cross-entropy loss function by about 2.89 percentage points from 0.04 to 13.48 percentage points depending on the training dataset. Additionally, we accomplished a comprehensive analysis with explainability techniques to assess the models' reliability and explain their results.
EResFD: Rediscovery of the Effectiveness of Standard Convolution for Lightweight Face Detection

This paper analyzes the design choices of face detection architecture that improve efficiency of computation cost and accuracy. Specifically, we re-examine the effectiveness of the standard convolutional block as a lightweight backbone architecture for face detection. Unlike the current tendency of lightweight architecture design, which heavily utilizes depthwise separable convolution layers, we show that heavily channel-pruned standard convolution layers can achieve better accuracy and inference speed when using a similar parameter size. This observation is supported by the analyses concerning the characteristics of the target data domain, faces. Based on our observation, we propose to employ ResNet with a highly reduced channel, which surprisingly allows high efficiency compared to other mobile-friendly networks (e.g., MobileNetV1, V2, V3). From the extensive experiments, we show that the proposed backbone can replace that of the state-of-the-art face detector with a faster inference speed. Also, we further propose a new feature aggregation method to maximize the detection performance. Our proposed detector EResFD obtained 80.4% mAP on WIDER FACE Hard subset which only takes 37.7 ms for VGA image inference on CPU. Code is available at https://github.com/clovaai/EResFD.
GAMR: A Guided Attention Model for (visual) Reasoning

Humans continue to outperform modern AI systems in their ability to flexibly parse and understand complex visual scenes. Here, we present a novel module for visual reasoning, the Guided Attention Model for (visual) Reasoning (GAMR), which instantiates an active vision theory -- positing that the brain solves complex visual reasoning problems dynamically -- via sequences of attention shifts to select and route task-relevant visual information into memory. Experiments on an array of visual reasoning tasks and datasets demonstrate GAMR's ability to learn visual routines in a robust and sample-efficient manner. In addition, GAMR is shown to be capable of zero-shot generalization on completely novel reasoning tasks. Overall, our work provides computational support for cognitive theories that postulate the need for a critical interplay between attention and memory to dynamically maintain and manipulate task-relevant visual information to solve complex visual reasoning tasks.
PILC: Practical Image Lossless Compression with an End-to-end GPU Oriented Neural Framework

Generative model based image lossless compression algorithms have seen a great success in improving compression ratio. However, the throughput for most of them is less than 1 MB/s even with the most advanced AI accelerated chips, preventing them from most real-world applications, which often require 100 MB/s. In this paper, we propose PILC, an end-to-end image lossless compression framework that achieves 200 MB/s for both compression and decompression with a single NVIDIA Tesla V100 GPU, 10 times faster than the most efficient one before. To obtain this result, we first develop an AI codec that combines auto-regressive model and VQ-VAE which performs well in lightweight setting, then we design a low complexity entropy coder that works well with our codec. Experiments show that our framework compresses better than PNG by a margin of 30% in multiple datasets. We believe this is an important step to bring AI compression forward to commercial use.
PROBLEMATIKA PELAKSANAAN ASESMEN NASIONAL BERBASIS KOMPUTER (ANBK) SEKOLAH DI KABUPATEN SERUYAN

This study aims to explore the problems of the implementation of the Computer-Based National Assessment (ANBK) which was carried out at SMPN 1 and SMAN 1 Danau Seluluk, Seruyan Regency, which are schools that are geographically as well as located far from urban centers and the availability of internet networks is not optimal.The research method uses descriptive qualitative research.The research subjects in this study were school principals, vice principals and implementing proctors.The research results obtained in the field included, SMPN 1 and SMAN 1 Danau Seluluk implemented semi-online ANBK.This was done because the condition of the internet network was unstable, besides that the lack of complete ANBK facilities and infrastructure, this is evidenced by the limited number of laptops and computers available.Furthermore, students also experience difficulties in answering ANBK questions related to literacy and numeracy.This occurs because of the gap between the material provided by the teacher and the questions given at ANBK so that teachers need training and assistance in developing learning in accordance with the demands of ANBK.