Reviewerly/exHarmony huggingface dataset:
The Use of Background Features, Template Synthesis and Deep Neural Networks in Document Forgery Detection

Document manipulation is a recently arising problem, especially with the rapid spread of fabrication technology. The tools to alter documents are now publicly available and can result in high quality forgeries, indistinguishable from genuine ones. Forged documents may wreak havoc on many processes dependent on the validity of the document, leading to lasting consequences such as financial loss. Therefore, the process of identifying a document that has been altered is essential. A system that is capable of scrutinizing documents as either forged or genuine through discriminative features (such as distortions or character misalignment) can assist industries with heavily reliance on documents for processes such as identity verification. Most of the documents involved in such processes have sufficiently complex backgrounds. We present a computer-vision-based system that detects changes in the background of the aforementioned documents as a result of manipulations made to its contents through the use of image subtraction. The system takes an image as input and then classifies the document as genuine or forged. Our proposed system produces an accuracy of 95% using CNN on unaligned images as well as 100% for aligned images.
Early-Exiting DNN MIMO Detector Design with Fast Signal Estimation Process

Multi-input multi-output (MIMO) receiver design has been continuously investigated in order to satisfy the massive requirements of six-generation (6G) wireless networks regarding the accuracy-centric signal estimation under the dynamic channel state information. In addtion, the architectures MIMO detector is a pioneering work for applying artificial intelligence techniques. Eventhough the DL-based MIMO detector can be able to provide accurate signal estimation, huge execution time and energy consumption of immensely large deep neural network (DNN) models in size has not been studied. To this end, we introduce a concept of early-exiting DNN MIMO detector and fast signal estimation, which outputting the estimated transmitted signal without a full forward and back propagation computation of the conventional DNN model. The early-exiting model and the fast signal estimation concept is believed to be suitable for scenarios of practical time sensitive and resouce limited applications.
Dual Policy-Based TD-Learning for Model Predictive Control

Temporal difference learning for model predictive control (TD-MPC) is the state-of-the-art in data-driven reinforcement learning-based MPC. Although TD-MPC maintains two different policies: 1) a model predictive path integral-based policy and 2) a parameterized policy, it applies only the first policy action to the environment. In this paper, we propose an extension of TD-MPC, called TD-MPC+, which selects a better policy that generates more practical actions, which exploits the agent's current estimated models greedily to get the most reward. It is noted that TD-MPC+ does not require an additional computational cost of model training than the original TD-MPC. In a comparison study with diverse the DeepMind Control Suite tasks, TD-MPC+ has higher sample efficiency and performance than TD-MPC.
On Modern Text-to-SQL Semantic Parsing Methodologies for Natural Language Interface to Databases: A Comparative Study

NLIDB research has gained popularity recently, mainly as a means of enhancing outcomes and performance. This study makes an effort to give readers background information on how the subject has evolved recently using different text-to-SQL procedures and approaches, as well as an appraisal of the advantages and disadvantages of each methodology. In contrast with past studies, this paper describes the search and selection processes and provide an overview of the complete process for each approach under review before making comparisons. The authors also evaluated the performance of each methodology against a widely recognized benchmark dataset. Along with model performance, each model was compared and assessed based on its overall structure and associated processes, such as using pre-trained language models and intermediate representations. The results of this study show that the field of text-to-SQL semantic parsing has advanced significantly in recent years, as seen by the improved performance of the models under consideration. It was clear that most recent developments concentrated on the encoder side, even if each technique follows an encoder-decoder design. The imbalance opens up much room for decoder advancement in subsequent studies. Using pre-trained language models was also noteworthy for improving the models' performances; the authors will consider this for future efforts. The selection of intermediate representations, on the other hand, is wholly arbitrary.
VM consolidation considering cores, memory, load for energy-constrained data centers

This work evaluates different virtual machine (VM) placement strategies in energy-constrained data centers powered by solar and hydrogen energy. Given the strict power consumption restrictions coming from the use of renewable energies, the placement problem is traditionally formulated as a linear optimization problem with the goal of minimizing the energy consumption under power-related constraints over the set of the possible placement of a given number of VM types. Each VM type is characterized by a triplet (c, m, l) where c represents the number of allocated CPU cores, m - the amount of allocated RAM, and l - the maximum allowed CPU load. Our obtained optimized placement is compared with the two reference strategies: Equal Load Balancing (ELB) and Least Number of Servers (LNS). It is important to note that the best placement strategy is hardware-dependent; our results show that the LNS strategy improves energy efficiency compared to ELB. The novelty of our work lies in using real servers and physical sensors to obtain energy consumption measurements which leads to a practically helpful comparison of placement strategies.
A Natural Language Understanding Approach Toward Extraction of Specifications from Request for Proposals

Industry 4.0 has witnessed a widespread use of Artificial Intelligence (AI), which, however, often focuses on the operational aspects. In contrast, the life-cycle of any industrial project begins much earlier. Motivated by this, we present an intent-based approach toward bid engineering. In particular, we consider the use of AI to automatically extract the intended specifications-technical and non-technical-of customers from Requests for Proposals (RFPs) by defining relevant data models. Subsequently, we annotate texts from real-life RFPs to train an AI model. In addition, we also design RfpAnno, an end-to-end solution to annotate documents, train models, and extract specifications as structured data. Experimental results indicate that the AI model has about 85% precision and recall, on average, using the test data set. Overall, RfpAnno can potentially reduce the time and effort required by bid engineers to manually copy requirements from RFPs.
Study of validation methods for augmented data

This paper introduces a study to verify whether the expanded data through various data augmentation methods are valid in terms of accuracy and bias. Data augmentation is a method of processing and generating other types of data with similar characteristics based on the characteristics of the obtained data, rather than directly collecting data when there is not enough data to increase analysis accuracy. However, unverified and augmented data may actually degrade the results of the analysis. Before using the amplified data for analysis, it is a very important verification factor whether it is accurately propagated in terms of similarity to the source data, and whether bias occurs because only a specific part is concentrated and propagated as a result of the propagation. Therefore, in this paper, a verification method is presented from these two perspectives.
Time Series Anomaly Detection Using Contrastive Learning based One-Class Classification

Time series anomaly detection in industrial processes has recently attracted attention. However, since there are no labels in the manufacturing process data collected in real time, there are limitations in using supervised learning-based classification models. Therefore, the proposed method newly defines an objective function that simultaneously learns the OCC model and contrastive learning. In addition, by applying data augmentation techniques suitable for periodic time series data in contrastive learning, feature extraction that preserves temporal characteristics is possible. The effectiveness of representation extraction was verified by showing high anomaly detection performance even in datasets with similar normal and anomaly data forms.
Identification of Misogyny on Social Media in Indonesian Using Bidirectional Encoder Representations From Transformers (BERT)

Misogyny is a behavior that hates or dislikes women Text classification can be used to identify misogyny text. One text classification method currently popular and proven to have good performance is the Bidirectional Encoder From Transformers (BERT). Fine-tuning is a method to transfer knowledge from a trained model to a new model to complete a new task. This study focuses on building a misogyny identification model with IndoBert pre-trained model provided by IndoNLU. The identification of Misogyny model obtained the best results with an accuracy value of 83.74% and by using K-fold cross-validation, the average validation value is 77.86%.
Malicious Model Detection for Federated Learning Empowered Energy Storage Systems

Renewable energy plays an essential role in the energy sector and reducing carbon emissions. Energy storage is the key to releasing the full potential of renewable energy because it offers grid flexibility to ensure uninterrupted power to consumers. As a result, monitoring the operation of energy storage systems and ensuring it functions properly are foremost. Because of data scarcity and privacy concerns, multiple energy storage systems can collectively identify battery failures in a federated manner. However, such a federated learning paradigm introduces vulnerability to the system. Compromised energy storage systems can provide malicious models to jeopardize the convergence and accuracy of the global model. In order to address this problem, we propose an autoencoder-backed malicious model detection framework for federated learning empowered energy storage systems. We construct an autoencoder to calculate the reconstruction error for each updated model. By thresholding the reconstruction error with a predefined value, we are able to identify the malicious model parameters and stop them from model aggregation. Real-world experiments show that the proposed countermeasure efficaciously detects compromised model updates under strong attacks and outperforms state-of-art defense schemes.
Generalized Spatio-Temporal Adaptive Normalization Framework

In this paper, we propose Generalized Spatio-Temporal Adaptive Normalization (GSTAN) Framework for Generative Adversarial and Deep Learning Inference Architectures. By leveraging higher-order derivatives based temporal feature maps along with spatial feature map, our normalization approach leads to: (a) efficient generation of high-quality videos with better details and enhanced temporal coherence, and, (b) higher accuracy inference on multiple tasks. In order to evaluate model generalization, we performed experimental evaluation on multiple tasks including: video to video generation, video segmentation and activity recognition (classify the activity out of 101 activity classes, for a given input video). Detailed experimental analysis over a variety of datasets including CityScape, UCF101 and CK+ demonstrates superior performance of GSTAN and also provides the impact of its various configurations, including parallel GSTAN and sequential GSTAN.
Multivariate Time Series Anomaly Detection via Temporal Encoder with Normalizing Flow

In the recent manufacturing process, as the introduction of smart factories spreads, high-dimensional data are being collected in real-time from various sensors of production facilities. However, existing anomaly detection models often do not reflect temporal factors, and even if they do, models that reflect temporal information are separately trained, resulting in a problem of falling into local optima. Therefore, it is very difficult to detect process anomalies in real-time by reflecting both correlations between high-dimensional variables and temporary dependency. This study proposes Temporal Encoder with Normalizing Flow (TENF), which can reflect both the correlation between variables and the time dependency in real-time using a relatively simple structure model. TENF consists of a Temporal Encoder for reflecting temporal dependencies and a NF Module for learning the distribution of high-dimensional data and is learned in an end-to-end manner. Experiments on multivariate time series data with similar characteristics to those generated in the manufacturing process demonstrate experimentally superior anomaly detection performance compared to existing models.
Federated Learning-Enabled Digital Twin for Smart Additive Manufacturing Industry

This work introduces a novel architecture of federated learning (FL)-enabled digital twin (DT) for the smart additive manufacturing industry, especially 3D printing. The proposed architecture tackles the previous limitation of the centralized approach that requires a large number of communication costs by efficiently updating the fault detection model on each server with distributed learning methods. A CNN-based model is also proposed to efficiently learn sensory data from a 3D printer for a fast and reliable fault detection model. To provide a robust system in intelligent manufacturing, a DT platform is also designed for seamless monitoring and control purposes. The proposed DT platform is able to initiate, monitor, and terminate the 3D printing process of physical assets via a virtual environment. Based on the simulation results, the FL process demonstrates that the proposed CNN-based model is superior to other DL models with 8% accuracy enlargement while maintaining the low training period. Furthermore, experimental work is conducted to evaluate the proposed architecture with real-world devices. Finally, the findings indicate that the overall latency given by the proposed system is relatively low, with an average of 1026.16 ms from the physical 3D printer to the DT platform.
Effect of Optimization Techniques on Feedback Alignment Learning of Neural Networks

The error backpropagation algorithm is a representative learning method that has been used in most deep network models. However, the error backpropagation algorithm, despite its decent performance, clearly has limits to its biological plausibility. Unlike the learning mechanism of the actual brain, the error backpropagation algorithm must reuse the weights used in the forward calculation for the backward error propagation. In order to overcome these limitations, the feedback alignment method, which uses a fixed random weight for the backpropagation computation, was proposed. The feedback alignment algorithm showed performances comparable to the original error backpropagation on several benchmark data sets. However, it is still in the preliminary stage of analysis, and various analysis on its learning behavior and practical efficiency are needed. In this paper, we combine feedback alignment learning method with popular optimization techniques such as RMSprop and Adam, and investigate its effect on the learning performances through computational experiments on benchmark data sets.
A Method of PV Power Generation Forecasting using Constrained Transformer Network

This paper introduces a method using a physics-constrained Transformer model attached to an edge computer for photovoltaic power generation that considers specific domain knowledge and increases the performance of the system. It attempts to address the drawback of current machine learning algorithms, which are applied based simply on enormous amounts of data, readily creating projections that are irrational. The experimental results indicate that the proposed physics-constrained Transformer model attached to an edge computer for photovoltaic power generation Transformer model possesses stronger prediction capability than other methods and is completely suitable with the sparse data in practice.
An inquiry into the semantic transparency and productivity of German particle verbs and derivational affixation

Abstract This study addresses the relation between morphological productivity and semantic transparency. Using distributional semantics, we compare German word formation using particles with derivational word formation. We observed that derivational suffixes, but not particles, tend to make strong independent semantic contributions to their carrier words. In two-dimensional t-SNE maps, complex words show clustering by affix, but not by particle. Furthermore, the semantic vectors of suffixed words are predictable from their base words with higher accuracy than is possible for particle verbs. For particle verbs, but not affixed verbs, semantic similarity within the set of complex words correlated negatively with the number of types. Furthermore, only for particle verbs, a greater number of observed types predicted a reduced probability of observing unseen types. We propose that particle verbs primarily serve the onomasiological function of labeling, resulting in relatively idiosyncratic semantic vectors. By contrast, words sharing derivational affixes form distinct clusters in semantic space while maintaining strong and consistent semantic relations with their base words. This enables these words to serve not only as labels, but also allows them to be used with an anaphoric function in discourse.
Differential Image-based Fast and Compatible Convolutional Layers for Multi-core Processors

Convolutional neural networks with powerful visual image analysis for artificial intelligence are gaining popularity in many research fields, leading to the development of various high-performance algorithms for convolution operators present in these networks. One of these approaches is implemented with general matrix multiplication (GEMM) using the well-known im2col transform for fast convolution operations. In this paper, we propose a multi-core processor-based convolution technique for high-speed convolutional neural networks (CNNs) using differential images. The proposed method improves the convolutional layer's response speed by reducing the computational complexity and using multi-thread technology. In addition, the proposed algorithm has the advantage of being compatible with all types of CNNs. We use the darknet network to evaluate the convolutional layer's performance and show the best performance of the proposed algorithm when using 4-thread parallel processing.
Cloud Security Requirement Based Threat Analysis

Threat analysis (TA) is a process to identify, detect, and evaluate security vulnerabilities systematically. Specifically, the TA, which focuses on threats that can potentially violate the customer’s data ownership requirements of security and performance, is named Requirement Based Threat Analysis (RBTA). Despite the importance of RBTA, the current manual RBTA process is both time intensive and makes no assurance of completeness of the analysis. Thus, we develop a systematic analytic technique that enumerates customers’ requirements and then determines all possible direct/indirect dependencies across them to conduct a generalized threat analysis from their requirements. The approach is validated for its effectiveness on actual Cloud customer requirements and can be generalized to apply to other requirements.
Semantic Similarity-based Visual Reasoning without Language Information

In this research, we propose new training data for the visual reasoning task based on semantic similarity and proposed a deep learning model that utilizes the data. The first contribution of this study is the construction of training data. Based on a total of 40 object attributes, we created a visual inference problem using only image data. As a result, a total of 6,000 datasets were built to create training and test data. We also propose a visual inference model as the second contribution of this work. The inference model shown in this study was evaluated for two tasks using ResNet50 and Vision Transformer, respectively. Based on the experimental evaluation results, we investigated the suitable pre-trained model for both single-choice binary reasoning and multiple-selection reasoning, respectively.
Impact of Grammar on Language Model Comprehension

Machine Learning and Natural Language Processing are playing an increasingly vital role in many different areas, including cybersecurity in Information Technology and Operational Technology networking, with many associated research challenges. In this paper, we introduce a new language model based on transformers with the addition of syntactical information into the embedding process. We show that our proposed Structurally Enriched Transformer (SET) language model outperforms baseline datasets on a number of downstream tasks from the GLUE benchmark. Our model improved CoLA classification by 11 points over the BERT-Base model. The performance of attention-based models has been demonstrated to be significantly better than that of traditional algorithms in several NLP tasks. Transformers are comprised of multi attention heads stacked on top of each others. A Transformer is capable of generating abstract representations of tokens input to an encoder based on their relationship to all tokens in a sequence. Despite the fact that such models can learn syntactic features based on examples alone, researchers have found that explicitly feeding this information to deep learning models can significantly boost their performance. A complex model like transformers may benefit from leveraging syntactic information such as part of speech (POS).
A New Approach to Lidar and Camera Fusion for Autonomous Driving

In this paper, we introduce an object detection model that combines a camera and a LiDAR sensor. In previous object detection studies have mainly focused on using one sensor, and mainly camera and LiDAR sensors were used. Research was mainly conducted in the direction of utilizing a single sensor, and typically cameras and LiDAR sensors were used. However, Camera and Li-DAR sensors have disadvantages such as being vulnerable to environmental changes or having sparse expressive power, so the method to improve them is needed for a stable cognitive system. In this paper, we propose the LiDAR Camera Fusion Network, a sensor fusion object detection model that uses the advantages of each sensor to improve the disadvantages of cameras and Li-DAR sensors. The sensor fusion object detector developed in this study has the feature of estimating the location of an object through LiDAR Clustering. Extraction speed is about 58 times faster than Selective search without prior learning, reducing the number of candidate regions from 2000 to 98, despite reducing the number of candidate regions, compared to existing methods, the ratio of the correct answer candidate areas among the total location candidate regions was 10 times larger. Due to the above characteristics, efficient learning and inference were possible compared to the existing method, and this model finally extracts the probability value of the object, the bounding box correction value, and the distance value from the object. Due to the characteristic of our research, we used KITTI data because LiDAR and image data were needed. As a result, we compare the results with object detection models that are often used in the object detection area.