Reviewerly/exHarmony huggingface dataset:
Stability Analysis of Recurrent-Neural-Based Controllers Using Dissipativity Domain

This paper proposes a method for the stability analysis of dynamic neural networks. The stability analysis of dynamic neural networks is a challenging task due to internal feedback connections. In this research work, we propose an algorithm based on the Reduction of Dissipativity Domain (RODD) algorithm. The RODD algorithm is a numerical technique for the detection of the stability of nonlinear dynamic systems. The method works by using an approximation of the reachable set. This paper proposes linear and quadratic approximations of reachable sets. RODD-LB uses a linear approximation, RODD-EB uses a quadratic approximation, and the RODD-Hybrid algorithm uses a combination of the linear and quadratic approximations. The accuracy and convergence of these algorithms were derived through numerical dynamic systems.
Parameter-Efficient Fine-Tuning Method for Task-Oriented Dialogue Systems

The use of Transformer-based pre-trained language models has become prevalent in enhancing the performance of task-oriented dialogue systems. These models, which are pre-trained on large text data to grasp the language syntax and semantics, fine-tune the entire parameter set according to a specific task. However, as the scale of the pre-trained language model increases, several challenges arise during the fine-tuning process. For example, the training time escalates as the model scale grows, since the complete parameter set needs to be trained. Furthermore, additional storage space is required to accommodate the larger model size. To address these challenges, we propose a new new task-oriented dialogue system called PEFTTOD. Our proposal leverages a method called the Parameter-Efficient Fine-Tuning method (PEFT), which incorporates an Adapter Layer and prefix tuning into the pre-trained language model. It significantly reduces the overall parameter count used during training and efficiently transfers the dialogue knowledge. We evaluated the performance of PEFTTOD on the Multi-WOZ 2.0 dataset, a benchmark dataset commonly used in task-oriented dialogue systems. Compared to the traditional method, PEFTTOD utilizes only about 4% of the parameters for training, resulting in a 4% improvement in the combined score compared to the existing T5-based baseline. Moreover, PEFTTOD achieved an efficiency gain by reducing the training time by 20% and saving up to 95% of the required storage space.
Tiny adversarial multi-objective one-shot neural architecture search

Abstract The widely employed tiny neural networks (TNNs) in mobile devices are vulnerable to adversarial attacks. However, more advanced research on the robustness of TNNs is highly in demand. This work focuses on improving the robustness of TNNs without sacrificing the model’s accuracy. To find the optimal trade-off networks in terms of the adversarial accuracy, clean accuracy, and model size, we present TAM-NAS, a tiny adversarial multi-objective one-shot network architecture search method. First, we build a novel search space comprised of new tiny blocks and channels to establish a balance between the model size and adversarial performance. Then, we demonstrate how the supernet facilitates the acquisition of the optimal subnet under white-box adversarial attacks, provided that the supernet significantly impacts the subnet’s performance. Concretely, we investigate a new adversarial training paradigm by evaluating the adversarial transferability, the width of the supernet, and the distinction between training subnets from scratch and fine-tuning. Finally, we undertake statistical analysis for the layer-wise combination of specific blocks and channels on the first non-dominated front, which can be utilized as a design guideline for the design of TNNs.
The methodology of using digital tools for the development of educational content in accordance with a given learning goal

The article presents a methodically based approach to the creation of teachers’ own educational content using digital tools following a given learning goal. The composition of the described methodology includes seven main elements, starting from the content to be studied and ending with the implementation of the created educational content. The theory of digital transformation of education, the theory of personality-oriented learning, the theory of a system-activity approach in teaching constitute, the methodological basis of the study. The object of the research is the process of creating educational content, the subject is the methodological conditions of the application of digital tools for the development of educational content following a given learning goal. In the course of the research, such methods as analysis of psychological, pedagogical, and methodological literature on the research topic, analysis of digital tools, theoretical modeling, and experimental training were used. As the results of research, the role of educational content created with the help of digital tools was determined, the need for teachers to develop their own educational content using digital tools was proved, the methodology for using digital tools to develop educational content following a given learning goal was described, and finally an example of the implementation of the methodology in the process of creating the web quest «Cybercriminals: fiction or reality?», addressed to V grade students was presented. Authors of the article have not only developed a technique of applying digital tools for creation of the educational content according to the set purpose of training but also have created and implemented the additional professional program of professional development “Digital tools of development of an educational content” in the educational process. The value of the presented research is that the methodology developed by the authors can be implemented by teachers who have no special IT training. It allows teachers to switch to a personalized and effective learning model in any lesson.
Fluorescence microscopy images denoising via deep convolutional sparse coding

Fluorescence microscopy images captured in low light and short exposure time conditions are always contaminated by photons and readout noises, which reduce the fluorescence microscopy images quality. In most cases, this kind of noise can be modeled as Poisson–Gaussian noise. Correspondingly, its denoising task has always been a hot but challenging topic in recent years. In this paper, by integrating model-driven and learning-driven methodologies, we propose an end-to-end supervised neural network for fluorescence microscopy images denoising, named MCSC-net, which embeds the multi-layer learned iterative soft threshold algorithm (ML-LISTA) into deep convolutional neural network (DCNN). Our approach not only uses the strong learning ability of DCNN to adaptively update all parameters in the ML-LISTA, but also introduces dilated convolution into network training without additional parameters to improve denoising performance. In addition, compared with several related methods on a real data set of fluorescence microscopy images, MCSC-net achieves the best denoising effects both in qualitative and quantitative aspects, which shows its strong appeal in practical denoising applications.
Multi-scale deep feature fusion based sparse dictionary selection for video summarization

The explosive growth of video data constitutes a series of new challenges in computer vision, and the function of video summarization (VS) is becoming more and more prominent. Recent works have shown the effectiveness of sparse dictionary selection (SDS) based VS, which selects a representative frame set to sufficiently reconstruct a given video. Existing SDS based VS methods use conventional handcrafted features or single-scale deep features, which could diminish their summarization performance due to the underutilization of frame feature representation. Deep learning techniques based on convolutional neural networks (CNNs) exhibit powerful capabilities among various vision tasks, as the CNN provides excellent feature representation. Therefore, in this paper, a multi-scale deep feature fusion based sparse dictionary selection (MSDFF-SDS) is proposed for VS. Specifically, multi-scale features include the directly extracted features from the last fully connected layer and the global average pooling (GAP) processed features from intermediate layers, then VS is formulated as a problem of minimizing the reconstruction error using the multi-scale deep feature fusion. In our formulation, the contribution of each scale of features can be adjusted by a balance parameter, and the row-sparsity consistency of the simultaneous reconstruction coefficient is used to select as few keyframes as possible. The resulting MSDFF-SDS model is solved by using an efficient greedy pursuit algorithm. Experimental results on two benchmark datasets demonstrate that the proposed MSDFF-SDS improves the F-score of keyframe based summarization more than 3% compared with the existing SDS methods, and performs better than most deep-learning methods for skimming based summarization.
A Java-like calculus with heterogeneous coeffects

We propose a Java-like calculus where declared variables can be annotated by coeffects specifying constraints on their use, e.g., affinity or privacy levels. Such coeffects are heterogeneous, in the sense that different kinds of coeffects can be used in the same program; combining coeffects of different kinds leads to the trivial coeffect. We prove subject reduction, which includes preservation of coeffects, and show several examples. In a Java-like language, coeffects can be expressed in the language itself, as expressions of user-defined classes.
Enhancing human activity recognition using features reduction in IoT edge and Azure cloud

The Internet of Things (IoT), cloud computing, and machine learning opened an opportunity for new smart systems. These technologies have triggered huge traffic and delay by continuously transmitting telemetry data to the cloud. IoT edge choice made decision-making closer to the environment, which decreases traffic and speeds up response time. Human activity recognition (HAR) systems, among other IoT applications, require systems with quick response time; reduce costs using constrained traffic to the cloud while maintaining accurate prediction results. This study proposes an application of HAR for predicting activities using up to three smartphone accelerometers. Three models are developed, trained, and deployed to achieve the necessary accuracy at the IoT edge and in the cloud, with an acceptable response time. Since each coordinate value from the three accelerometers has different importance in activity category prediction, focusing on the data from the most related values can help minimize the amount of information transferred from the edge to the cloud. Six models were trained in the cloud; three were deployed and tested at the edge with different features by selecting the most important ones using Principal Component Analysis (PCA). Different experiments showed that traffic and processing time decreased significantly based on the time required to predict HAR categories with acceptable accuracy. Since there is significant latency between the edge and the cloud and within the cloud, sending samples for verification save bandwidth, and processing requests locally at the edge speed up predictions. Results illustrate that the time required to serve one request from the environment where smartphones generate traffic through the internet connection to the cloud took about 5.8 s on average, including transmission delays and the prediction process. During this time, the model at the edge can serve 150 requests with the same accuracy using nine features. In addition, the edge can serve 286 requests in 5.8 s with 94.8 % accuracy when choosing the top four features at the edge.
A Model for Chinese Named Entity Recognition Based on Global Pointer and Adversarial Learning

To solve the problem that the Chinese named entity recognition (NER) models have poor antiinterference ability and inaccurate entity boundary recognition, this paper proposes the RGP-with-FGM model which is based on global pointer and adversarial learning.Firstly, the RoBERTa-WWM model is used to optimize the semantic representation of the text, and fast gradient method is used to add perturbation to the word embedding layer to enhance the robustness of the model.Then, BiGRU is used to focus on the timing information of Chinese characters to enhance the semantic connection.Finally, the global pointer is constructed in the decoding layer to obtain more accurate entity boundary recognition results.In order to verify the effectiveness of the model proposed in this paper, we construct Uyghur names dataset (UHND) to train the Chinese NER model, and perform extensive experiments with public Chinese NER data sets.Experimental results show that for UHND, the F1 value is 95.12%, which is 3.09% higher than that of the RoBERTa-WWM-BiGRU-CRF model.For the Resume data set, the Precision and F1 value are 96.28% and 96.10% respectively.
Representation of Semantic Word Embeddings Based on SLDA and Word2vec Model

To solve the problem of semantic loss in text representation, this paper proposes a new embedding method of word representation in semantic space called wt2svec based on supervised latent Dirichlet allocation (SLDA) and Word2vec.It generates the global topic embedding word vector utilizing SLDA which can discover the global semantic information through the latent topics on the whole document set.It gets the local semantic embedding word vector based on the Word2vec.The new semantic word vector is obtained by combining the global semantic information with the local semantic information.Additionally, the document semantic vector named doc2svec is generated.The experimental results on different datasets show that wt2svec model can obviously promote the accuracy of the semantic similarity of words, and improve the performance of text categorization compared with Word2vec.
A Novel Robust Online Extreme Learning Machine for the Non-Gaussian Noise

Samples collected from most industrial processes have two challenges: one is contaminated by the non-Gaussian noise, and the other is gradually obsolesced. This feature can obviously reduce the accuracy and generalization of models. To handle these challenges, a novel method, named the robust online extreme learning machine (RO-ELM), is proposed in this paper, in which the least mean p-power criterion is employed as the cost function which is to boost the robustness of the ELM, and the forgetting mechanism is introduced to discard the obsolescence samples. To investigate the performance of the RO-ELM, experiments on artificial and real-world datasets with the non-Gaussian noise are performed, and the datasets are from regression or classification problems. Results show that the RO-ELM is more robust than the ELM, the online sequential ELM (OS-ELM) and the OS-ELM with forgetting mechanism (FOS-ELM). The accuracy and generalization of the RO-ELM models are better than those of other models for online learning.
Passive solar tracker using a bimetallic strip activator with an integrated night return mechanism

Passive solar trackers face challenges in returning PV panels to the east position before sunrise. Specifically, bimetallic strip deflection-based trackers are unreliable due to ambient temperature changes at night, resulting in inconsistency across different locations and times. This study addresses this issue by designing a passive solar tracker with a bimetallic strip deflector independent of nighttime temperature differences. Field tests were conducted comparing a fixed system to the proposed tracker. Performance parameters included power output, power density, and energy collection. The proposed tracker achieved a daily average power output of 70-120 W, while the fixed system ranged from 10 to 120 W. The proposed system exhibited steady changes in power output, providing an additional 47 W per unit area compared to the fixed system. This optimization maximized power output within a given area, resulting in a cost savings of $71.75 per unit area. The proposed system also demonstrated 24.86% higher energy collection efficiency relative to the fixed system and 96.4% accuracy compared to an ideal tracking system. It is a cost-effective alternative to fixed-panel PV and easy to install and operate. Additionally, it can be used in conjunction with active tracking systems, offering comparable efficiency at a lower operational cost. The designed system presents a reliable solution by being independent of nighttime temperature fluctuations, making it more stable relative to other passive solar trackers.
Knowledge-enhanced event relation extraction via event ontology prompt

Identifying temporal and subevent relationships between different events (i.e., event relation extraction) is an important step towards event-centric natural language processing, which can help understand how events evolve and potentially facilitate many downstream tasks, such as timeline generation and event knowledge graph construction. Existing work has extensively leveraged external knowledge to improve the performance of relation extraction. Despite the progress made, the current knowledge-enhanced approach still has some shortcomings, e.g., knowledge missing, knowledge noise, and suboptimal knowledge injection. In this paper, we propose OntoEnhance, a novel event relation extraction framework that fuses semantic information from event ontologies to enhance event representation. Specifically, we first inject the latent knowledge in the event ontology into the prompt text to address the issue of knowledge missing. Then a dual-stack attention fusion mechanism is further introduced to enhance the injection of key knowledge to alleviate knowledge noise. In order to prevent the knowledge in the event ontology from being wrongly dominated, we use the event direction induction mechanism to obtain the event context-based relational sequence representation. Finally, a gate mechanism is used to fuse ontology-based knowledge and context-based event features dynamically. Extensive experiments demonstrate that OntoEnhance outperforms all comparison baselines by a large margin on all four datasets under both standard and few-shot settings.
Piezoelectric MEMS-based physical reservoir computing system without time-delayed feedback

Abstract In this study, a physical reservoir computing system, a hardware-implemented neural network, was demonstrated using a piezoelectric MEMS resonator. The transient response of the resonator was used to incorporate short-term memory characteristics into the system, eliminating commonly used time-delayed feedback. In addition, the short-term memory characteristics were improved by introducing a delayed signal using a capacitance-resistor series circuit. A Pb(Zr,Ti)O 3 -based piezoelectric MEMS resonator with a resonance frequency of 193.2 Hz was employed as an actual node, and computational performance was evaluated using a virtual node method. Benchmark tests using random binary data indicated that the system exhibited short-term memory characteristics for two previous data and nonlinearity. To obtain this level of performance, the data bit period must be longer than the time constant of the transient response of the resonator. These outcomes suggest the feasibility of MEMS sensors with machine-learning capability.
Covariance Matrix Adaptation MAP-Annealing

Single-objective optimization algorithms search for the single highest-quality solution with respect to an objective. Quality diversity (QD) optimization algorithms, such as Covariance Matrix Adaptation MAP-Elites (CMA-ME), search for a collection of solutions that are both high-quality with respect to an objective and diverse with respect to specified measure functions. However, CMA-ME suffers from three major limitations highlighted by the QD community: prematurely abandoning the objective in favor of exploration, struggling to explore flat objectives, and having poor performance for low-resolution archives. We propose a new quality diversity algorithm, Covariance Matrix Adaptation MAP-Annealing (CMA-MAE), that addresses all three limitations. We provide theoretical justifications for the new algorithm with respect to each limitation. Our theory informs our experiments, which support the theory and show that CMA-MAE achieves state-of-the-art performance and robustness.
How the Morphology Encoding Influences the Learning Ability in Body-Brain Co-Optimization

Embedding the learning of controllers within the evolution of morphologies has emerged as an effective strategy for the co-optimization of agents' bodies and brains. Intuitively, that is how nature shaped animal life on Earth. Still, the design of such co-optimization is a complex endeavor; one issue is the choice of the genetic encoding for the morphology. Such choice can be crucial for the effectiveness of learning, i.e., how fast and to what degree agents adapt, through learning, during their life. Here we evolve the morphologies of voxel-based soft agents with two different encodings, direct and indirect while learning the controllers with reinforcement learning. We experiment with three tasks, ranging from cave crawling to beam toppling, and study how the encoding influences the learning outcome. Our results show that the direct encoding corresponds to increased ability to learn, mostly in terms of learning speed. The same is not always true for the indirect one. We link these results to different shades of the Baldwin effect, consisting of morphologies being selected for increasing an agent's ability to learn during its lifetime.
Optimizing retroreflective marker set for motion capturing props

One of the most widely used motion capture (MoCap) methods depends on retroreflective markers placed on an object. Through cameras, the MoCap system captures the moving prop’s (3D object) motion. However, noise in MoCap data caused by the shape of the captured volume, motion between frames, ghost points, and markers’ self-occlusion could impact the quality of the captured data. To improve the quality of capturing the motion of props, we tackle the problem of finding an optimal marker set configuration for a given input prop while considering various constraints. By “props,” we mean any objects or handheld items of different shapes and sizes on which markers can be precisely placed to reduce MoCap errors. We propose an approach to optimize the placement of optical (retroreflective) markers over props while encountering various constraints, such as the visibility of markers, the number of makers used, the symmetry of the marker set, and markers’ overlapping. We solve the marker set configuration problem using an optimization-based method, the reversible-jump Markov chain Monte Carlo. We provide marker set configurations for various props and constraints obtained through several simulations we ran to evaluate the performance of our method.
Balance Loss for MultiAttention-Based YOLOv4

Aiming to address the advantages and disadvantages of various target detection algorithms, we optimized the data augmentation algorithm, attention mechanism and loss backpropagation function across different neural networks, and proposed an improved YOLOv4-based target detection algorithm. The new model introduces a Mixup algorithm with Mosaic in the data augmentation block, adds a multi-attention block that includes channel attention and spatial attention between the backbone and FPN, and replaces CIoU in the Loss function with Balance IoU and cascade confidence loss called Balance Loss to enhance the neural network's target detection capability. We trained and validated the improved algorithm in the VOC2007 dataset, and experimental results showed that this algorithm has a more robust feature extraction capability and higher target detection accuracy than the original YOLOv4 algorithm. The mAP increases by 1.2%. The model also performs better in detecting small and medium target objects.