Reviewerly/exHarmony huggingface dataset:
$Λ$-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells

Differentiable neural architecture search (DARTS) is a popular method for neural architecture search (NAS), which performs cell-search and utilizes continuous relaxation to improve the search efficiency via gradient-based optimization. The main shortcoming of DARTS is performance collapse, where the discovered architecture suffers from a pattern of declining quality during search. Performance collapse has become an important topic of research, with many methods trying to solve the issue through either regularization or fundamental changes to DARTS. However, the weight-sharing framework used for cell-search in DARTS and the convergence of architecture parameters has not been analyzed yet. In this paper, we provide a thorough and novel theoretical and empirical analysis on DARTS and its point of convergence. We show that DARTS suffers from a specific structural flaw due to its weight-sharing framework that limits the convergence of DARTS to saturation points of the softmax function. This point of convergence gives an unfair advantage to layers closer to the output in choosing the optimal architecture, causing performance collapse. We then propose two new regularization terms that aim to prevent performance collapse by harmonizing operation selection via aligning gradients of layers. Experimental results on six different search spaces and three different datasets show that our method ($\Lambda$-DARTS) does indeed prevent performance collapse, providing justification for our theoretical analysis and the proposed remedy.
Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2

One approach to meet the challenges of deep lifelong reinforcement learning (LRL) is careful management of the agent's learning experiences, to learn (without forgetting) and build internal meta-models (of the tasks, environments, agents, and world). Generative replay (GR) is a biologically inspired replay mechanism that augments learning experiences with self-labelled examples drawn from an internal generative model that is updated over time. We present a version of GR for LRL that satisfies two desiderata: (a) Introspective density modelling of the latent representations of policies learned using deep RL, and (b) Model-free end-to-end learning. In this paper, we study three deep learning architectures for model-free GR, starting from a na\"ive GR and adding ingredients to achieve (a) and (b). We evaluate our proposed algorithms on three different scenarios comprising tasks from the Starcraft-2 and Minigrid domains. We report several key findings showing the impact of the design choices on quantitative metrics that include transfer learning, generalization to unseen tasks, fast adaptation after task change, performance wrt task expert, and catastrophic forgetting. We observe that our GR prevents drift in the features-to-action mapping from the latent vector space of a deep RL agent. We also show improvements in established lifelong learning metrics. We find that a small random replay buffer significantly increases the stability of training. Overall, we find that "hidden replay" (a well-known architecture for class-incremental classification) is the most promising approach that pushes the state-of-the-art in GR for LRL and observe that the architecture of the sleep model might be more important for improving performance than the types of replay used. Our experiments required only 6% of training samples to achieve 80-90% of expert performance in most Starcraft-2 scenarios.
Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors

Current deep learning models trained to generate radiology reports from chest radiographs are capable of producing clinically accurate, clear, and actionable text that can advance patient care. However, such systems all succumb to the same problem: making hallucinated references to non-existent prior reports. Such hallucinations occur because these models are trained on datasets of real-world patient reports that inherently refer to priors. To this end, we propose two methods to remove references to priors in radiology reports: (1) a GPT-3-based few-shot approach to rewrite medical reports without references to priors; and (2) a BioBERT-based token classification approach to directly remove words referring to priors. We use the aforementioned approaches to modify MIMIC-CXR, a publicly available dataset of chest X-rays and their associated free-text radiology reports; we then retrain CXR-RePaiR, a radiology report generation system, on the adapted MIMIC-CXR dataset. We find that our re-trained model--which we call CXR-ReDonE--outperforms previous report generation methods on clinical metrics, achieving an average BERTScore of 0.2351 (2.57% absolute improvement). We expect our approach to be broadly valuable in enabling current radiology report generation systems to be more directly integrated into clinical pipelines.
Very Large Language Model as a Unified Methodology of Text Mining

Text data mining is the process of deriving essential information from language text. Typical text mining tasks include text categorization, text clustering, topic modeling, information extraction, and text summarization. Various data sets are collected and various algorithms are designed for the different types of tasks. In this paper, I present a blue sky idea that very large language model (VLLM) will become an effective unified methodology of text mining. I discuss at least three advantages of this new methodology against conventional methods. Finally I discuss the challenges in the design and development of VLLM techniques for text mining.
Denoising Masked AutoEncoders Help Robust Classification

In this paper, we propose a new self-supervised method, which is called Denoising Masked AutoEncoders (DMAE), for learning certified robust classifiers of images. In DMAE, we corrupt each image by adding Gaussian noises to each pixel value and randomly masking several patches. A Transformer-based encoder-decoder model is then trained to reconstruct the original image from the corrupted one. In this learning paradigm, the encoder will learn to capture relevant semantics for the downstream tasks, which is also robust to Gaussian additive noises. We show that the pre-trained encoder can naturally be used as the base classifier in Gaussian smoothed models, where we can analytically compute the certified radius for any data point. Although the proposed method is simple, it yields significant performance improvement in downstream classification tasks. We show that the DMAE ViT-Base model, which just uses 1/10 parameters of the model developed in recent work arXiv:2206.10550, achieves competitive or better certified accuracy in various settings. The DMAE ViT-Large model significantly surpasses all previous results, establishing a new state-of-the-art on ImageNet dataset. We further demonstrate that the pre-trained model has good transferability to the CIFAR-10 dataset, suggesting its wide adaptability. Models and code are available at https://github.com/quanlin-wu/dmae.
Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution

Knowledge distillation (KD) is an effective framework that aims to transfer meaningful information from a large teacher to a smaller student. Generally, KD often involves how to define and transfer knowledge. Previous KD methods often focus on mining various forms of knowledge, for example, feature maps and refined information. However, the knowledge is derived from the primary supervised task and thus is highly task-specific. Motivated by the recent success of self-supervised representation learning, we propose an auxiliary self-supervision augmented task to guide networks to learn more meaningful features. Therefore, we can derive soft self-supervision augmented distributions as richer dark knowledge from this task for KD. Unlike previous knowledge, this distribution encodes joint knowledge from supervised and self-supervised feature learning. Beyond knowledge exploration, we propose to append several auxiliary branches at various hidden layers, to fully take advantage of hierarchical feature maps. Each auxiliary branch is guided to learn self-supervision augmented task and distill this distribution from teacher to student. Overall, we call our KD method as Hierarchical Self-Supervision Augmented Knowledge Distillation (HSSAKD). Experiments on standard image classification show that both offline and online HSSAKD achieves state-of-the-art performance in the field of KD. Further transfer experiments on object detection further verify that HSSAKD can guide the network to learn better features. The code is available at https://github.com/winycg/HSAKD.
Quantum Reinforcement Learning: the Maze problem

Quantum Machine Learning (QML) is a young but rapidly growing field where quantum information meets machine learning. Here, we will introduce a new QML model generalizing the classical concept of Reinforcement Learning to the quantum domain, i.e. Quantum Reinforcement Learning (QRL). In particular we apply this idea to the maze problem, where an agent has to learn the optimal set of actions in order to escape from a maze with the highest success probability. To perform the strategy optimization, we consider an hybrid protocol where QRL is combined with classical deep neural networks. In particular, we find that the agent learns the optimal strategy in both the classical and quantum regimes, and we also investigate its behaviour in a noisy environment. It turns out that the quantum speedup does robustly allow the agent to exploit useful actions also at very short time scales, with key roles played by the quantum coherence and the external noise. This new framework has the high potential to be applied to perform different tasks (e.g. high transmission/processing rates and quantum error correction) in the new-generation Noisy Intermediate-Scale Quantum (NISQ) devices whose topology engineering is starting to become a new and crucial control knob for practical applications in real-world problems. This work is dedicated to the memory of Peter Wittek.
On Calibration of Scene-Text Recognition Models

In this work, we study the problem of word-level confidence calibration for scene-text recognition (STR). Although the topic of confidence calibration has been an active research area for the last several decades, the case of structured and sequence prediction calibration has been scarcely explored. We analyze several recent STR methods and show that they are consistently overconfident. We then focus on the calibration of STR models on the word rather than the character level. In particular, we demonstrate that for attention based decoders, calibration of individual character predictions increases word-level calibration error compared to an uncalibrated model. In addition, we apply existing calibration methodologies as well as new sequence-based extensions to numerous STR models, demonstrating reduced calibration error by up to a factor of nearly 7. Finally, we show consistently improved accuracy results by applying our proposed sequence calibration method as a preprocessing step to beam-search.
Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text

Machine Learning has seen tremendous growth recently, which has led to larger adoption of ML systems for educational assessments, credit risk, healthcare, employment, criminal justice, to name a few. The trustworthiness of ML and NLP systems is a crucial aspect and requires a guarantee that the decisions they make are fair and robust. Aligned with this, we propose a framework GYC, to generate a set of counterfactual text samples, which are crucial for testing these ML systems. Our main contributions include a) We introduce GYC, a framework to generate counterfactual samples such that the generation is plausible, diverse, goal-oriented, and effective, b) We generate counterfactual samples, that can direct the generation towards a corresponding condition such as named-entity tag, semantic role label, or sentiment. Our experimental results on various domains show that GYC generates counterfactual text samples exhibiting the above four properties. GYC generates counterfactuals that can act as test cases to evaluate a model and any text debiasing algorithm.
Expert System for Predictive Maintenance Transformer using J48 Algorithm

Predictive maintenance can reduce the risk of sudden transformer failure which causes the risk of plant to stop operating. One of transformer predictive maintenance technique is the Dissolved Gas Analysis (DGA) Test Oil Transformer. The gas is interpreted and analyzed to find out and get conclusions about the health condition and also possible problems in the transformer based on IEEE Standards and IEC Standards. To facilitate monitoring, a Decision Support System for Interpretation of Test Results of DGA Oil Immersed Transformer was created to form a database containing transformer data with the amount of main gas from the DGA test results. Next, decision tree was made using the J48 algorithm. The decision tree simplifying and speed up the decision-making process for recommended actions that are displayed on the system. The system also displays a trending graph of the last transformer test and quickly displays a dashboard of transformer status, i.e. normal, alarm, or danger. Engineer will get notification email if any transformer is in danger status. In addition, the system is able to provide information on possible fault types for each transformer. The benefits of this system are that the health condition of the transformer can be monitored properly and corrective action can be taken immediately on a problem based on the results of the decision support system. This will reduce the risk of shutdown and support the reliability of plant operations.
Variational Auto Encoder fused with Gaussian Process for Unsupervised Anomaly Detection

Abstract The identification of abnormal data in high-dimensional and high-complexity situations is a challenging subject. In order to improve the accuracy of abnormal data detection, in this article, we first use Variational Auto Encoder (VAE) to extract the features of high-dimensional data to achieve the effect of data dimensionality reduction. Then the Gaussian process model is applied to establish the latent space's feature distribution field and guide the data's displacement in the latent space. The joint probability distribution matrix of the latent space data is adopted to establish the feedback channel between latent space data distribution and VAE data mapping. Finally, the characteristic distribution field of the Gaussian process model is fed back to the VAE to realize the separation of normal data and abnormal data at the data mapping level.
Utilization of Website-Based Information Systems to Optimize Data Management in Khaela Hijab

Khaela Hijab is a company which sells hijab products. The place is in Sedayu District, Bantul Regency, Yogyakarta. Khaela Hijab produces their own hijab and markets its products via offline and online. Khaela Hijab uses social media as a medium to promote the products. Every sales and purchase transactions are recorded conventionally in a special note book. The transaction writing model causes constraints in presentation of information, where the information could not provided in real time. Beside of that, the calculation of sales profit product also had constraint which caused by recording of production and sales costs that often unbalanced. The result of this activity is the usage of sales information system which uses website as a base - with assistance and material distribution about the use of website to optimize the data management.
Computational Thinking Improvement Through Speech Recognition Application Development for Pahoa High School Students

Computational Thinking (CT) skills are needed in jobs in the Industrial 4.0 era. Activities to improve CT have been widely carried out in the world of education. Learning programming and artificial intelligence education is one of the ways to improve CT. This community service activity (PkM) at Pahoa High School aims to improve CT through the development of a speech recognition application using the MIT App Inventor. Through the stages of goal setting, the need for students is defined. Then in the preparatory stage, training modules are drawn up. At the implementation stage, PkM activities are carried out online and as an evaluation stage, at the end of the activity students are given a test. This activity was followed by 44 students online. As the results, 68% of students received test scores of more than 60 and an average of 70% of students answered correctly questions related to algorithm design.
Assistance in Making Interactive Learning Using Online Media for Teachers of SDN 24 Pinggir

Sekolah Dasar Negeri (SDN) 24 Pinggir is a mentoring partner who has limitations in meeting the demands of learning during Covid-19 pandemic and in digital era, so it has the potential to have degradation in education. The educators of SDN 24 Pinggir need assistance in opening their horizons and knowledge in utilizing the right media in making teaching materials, even evaluating with learning methods and models that are in accordance with the needs of their students. The platforms chosen to be used as mentoring materials are wordwall, wizer and liveworksheet. The platform was chosen with the consideration that it is easy to access, simple and easy to understand, has features that can accommodate the needs of educators and there are free of charge options. Training activities begin with observations in the form of questions and answers with teachers and school principal which are then continued with training. Furthermore, participants did hands-on practice, and were given feedback through post-training evaluations. the initial survey, the average teacher was hesitant (46.30%) in making and distributing learning material using the platform. However, there was an increase in knowledge of 98.15% (Knowing in theory and practice) after the training was given.
A Hybrid Interval Evidential Reasoning-Regret Decision Making Model and its Application to the Behavior of Chinese Herbal Medicine Planting Selection

The choice of Chinese herbal medicine (CHM) planting behavior is a complex issue, as it requires consideration of a series of attributes that are not independent relationships and a wealth of participants. To tackle the planting decision making problem of CHM planting behavior, the proposal introduces a novel group decision making mechanism, which includes the following three steps. The step one is to establish three information conversion mechanisms based on interval evidential reasoning (IER) to convert different forms of information into an identical interval belief structure. The step two is to establish a model based on the area method and obtain the weight of the attribute. Finally, based on the result obtained from the last two steps, the extended interval regret theory (IRT) is proposed to sort the alternatives. To drill the operation process of the proposal in detail, a real case was put in to analyze the planting decision behavior of CHM. The flexibility and dependability of the proposal are verified by comparative experiments and sensitivity analysis.
SECURITY ANALYSIS ON WEBSITES USING THE INFORMATION SYSTEM ASSESSMENT FRAMEWORK (ISSAF) AND OPEN WEB APPLICATION SECURITY VERSION 4 (OWASPv4) USING THE PENETRATION TESTING METHOD

&lt;p&gt;At this time in the rapid development of technology, there must be advantages and disadvantages of a system or technology that was created. Within the scope of the website, there are also many security holes that irresponsible parties can enter. The state of the website at the Telkom Purwokerto Institute of Technology, both University and Faculty websites, already uses Hypertext Transfers Protocol Secure (HTTPS). This study used the Information System Security Assessment Framework (ISSAF) and Open Web Application Project (OWASP) frameworks with the Penetration Testing method. This study aims to determine vulnerabilities on the website s1if.ittelkom-pwt.ac.id. The result of performing vulnerabilities is several vulnerabilities to the Institut Teknologi Telkom Purwokerto (ITTP) Informatics Study Program website, including not updating jquery on the ITTP website. Ten tests have been carried out, five tests using ISSAF and five tests using OWSP version 4. When performing vulnerabilities in the ISSAF framework, found robots files.txt on the S1 Informatics website which is quite crucial for s1if.ittelkom-pwt.ac.id website which contains an exploitable sitemap. &lt;/p&gt;
THE WORLD, CREATED BY HUMAN MINDS

The article is about the absence of independent physical presence of the ideas formatted by the brain mind, though each individual may act upon the ideas or the encoded information created by the brain, which we call a process. These ideas may be stored mentally, which could be remembered by the individual, or physically stored in the language used by the individual. The ideas created by the brain is generally using a language or a figure represented by an image or material, which represents the object or the idea. The mental image or the idea represents a physical reality or a functional state, allowing the person to work upon the physical entity. The mind could create several entities, using language or symbolically, which may not have a real presence in the world.The mind has created several such entities and even represented them physically by materials or symbolically. People have been creating mental ideas of divine forces and visualizing the, and believe such forces really exist and control them and the world. The mental concepts could be symbolic and one could visualize them if they could be treated with an image. Sensory abilities may help visual or auditory appearances of them or have sensorial experiences with them if their brain is going through a hallucinatory state. We have already reported the phantom phenomenon when the affected person experiences a limb or body, which is truly absent in the person. The orbitofrontal cortex is engaged in social conditioning, and when the input signals do not reach this area, violent responses may be made at the amygdala level.
Right-Adjoints for Datalog Programs, and Homomorphism Dualities over Restricted Classes

A Datalog program can be viewed as a syntactic specification of a functor from database instances over some schema to database instances over another schema. The same holds more generally for $\exists$Datalog. We establish large classes of Datalog and $\exists$Datalog programs for which the corresponding functor admits a generalized right-adjoint. We employ these results to obtain new insights into the existence of, and methods for constructing, homomorphism dualities within restricted classes of instances. We also derive new results regarding the existence of uniquely characterizing data examples for database queries.