Reviewerly/exHarmony huggingface dataset:
Action recognition method based on lightweight network and rough-fine keyframe extraction

To address the issues of large number of parameters and low recognition accuracy of action recognition networks, we propose an effective action recognition method based on lightweight network and rough-fine keyframe extraction. The method consists of three modules. The first module proposes a keyframe extraction network based on grayscale and feature descriptors, and employs a rough-fine idea to extract video keyframe. It reduces the redundancy of keyframe and enhances their ability to express action semantics. The second module introduces an attention-based feature extraction network, which combines decoupling ideas with attention mechanisms to enhance the accuracy of the action recognition network, while significantly reducing the network parameters. The third module is an improved attention module which optimizes the representation of local information. Finally, addition of the residual module fuses feature information between different convolutional layers. Experiments on two different datasets show that the number of parameters in the proposed method is only 6.4M. On publicly available datasets of HMDB51 and UCF101, the method achieves recognition accuracy of 75.69% and 93.18% without pre-training, respectively. The proposed method is valid and feasible on multiple public datasets.
On the methodology of three-way structured merge in version control systems: Top-down, bottom-up, or both

Three-way merging is an essential component of version control systems. Despite the efficiency of the conventional line-based textual methods, syntax-based structured approaches have demonstrated benefits in improving merge accuracy. Prior structured merging approaches visit abstract syntax trees in a top-down manner, which struggles to identify and merge shifted code generally. This work introduces a novel methodology combining a top-down and a bottom-up visit of abstract syntax trees, which manipulates shifted code effectively and elegantly. Especially, we reduce the merge problem of ordered lists to computing a topological sort of strongly connected components of the constraint graph. Compared with four representative merge tools in 40,533 real-world merge scenarios, our approach achieves the highest merge accuracy while being 2.5 x as fast as a state-of-the-art structured merge tool.
Online detection of analog signal anomaly based on the evaluation of local trends

Abstract Online anomaly detection (AD) of analog signals plays an important role in equipment fault diagnosis and predictive maintenance. However, the signal often deviates slightly from those seen previously in the early stage of equipment failure, and the anomaly is invisible to the human eye. This kind of anomaly belongs to the typical contextual anomaly. Whether this anomaly can be effectively detected determines whether the failure of the equipment can be detected in the early stage, which is of great significance for safety in production. This study aimed to propose an online AD method for the analog signals of the quasi-sine waveform class. The sample similarity in the sliding window was evaluated using a sample trend rather than sample amplitude deviation to detect anomalies based on the principle that the trend of the quasi-sinusoid waveform signal in the adjacent space was similar. Compared with the traditional method, the proposed method was sensitive to contextual anomalies and did not need a complete sample data set for model training. The proposed method was finally validated by three data sets with good results.
Towards Extraction of Validation Rules from OPC UA Companion Specifications

Interoperability is the key to industrial automation. OPC UA aims to provide interoperability between industrial machines at the network, protocol, and semantic layers. In order to achieve it, the implementation of a machine and its nodeset file should comply with the OPC UA standard. However, there are a few areas for improvement in technology to automatically check compliance since the compliance rules/validation rules are expressed in textual format in specification documents. In other words, they are not machine-interpretable. Converting the text-based specification into machine-readable form is a complex problem since each companion specification is domain-specific and written by a set of industry experts from all over the world from diverse language backgrounds. The specifications are majorly unique, with little commonality between them. Because of these reasons, it is challenging to develop a generic information extraction approach to retrieve rules that work for all specifications. In this study, we aim to handle this challenge to extract the textual rules from specifications automatically by applying Natural Language Processing and Machine Learning technologies. We present our methodology based on Named Entity Recognition for OPC UA documents for information extraction and text classification to identify rules. To achieve this goal, we created five named entity data sets from five selected OPC UA companion specifications. The sentences in the data set and the entities in the sentences were annotated by two OPC UA experts. We point out a deeper analysis of the data sets to highlight common and unique entities in them and show their usage in identifying the rules in the specifications.
Ontology based knowledge graph for information and knowledge management in factory planning

The amount of necessary information and knowledge to fulfill the tasks within the factory planning domain is rapidly growing. Currently, data and documents containing relevant information are stored as single artifacts in various established systems which makes information gathering time-consuming. Therefore it would be helpful to connect the existing artifacts so that it is possible to get all information of different origins at a glance. As Semantic Web technologies are recently evolving from academic research to industry applications, Knowledge Graphs (KG) as one of their specific implementations seem to be a suitable option to manage and connect these information artifacts in a sustainable matter. This can effectively support knowledge management in factory planning and provides machine-readable access to domain information that can be used by currently emerging Artificial Intelligence (AI) applications. This paper focuses on the conceptual challenges related to the creation of a knowledge graph within the factory planning domain and proposes a concept for a platform to collect and access existing information and experiences from domain experts.
Complexity of Structured Text in IEC 61499 Function Blocks: A Survey.

The rising complexity of modern production systems raises the need for evaluating the code quality also in control software engineering. Complexity metrics can help identifying code that is difficult to maintain and may benefit from a refactoring. Although complexity metrics have already been adapted to the domain-specific languages (DSLs) used for cyber-physical production systems (CPPSs), literature on whether developers benefit from using these metrics is limited. Also, the complexity perception of code by the domain experts remains to be investigated. The perceived difficulty of comprehending a program can vary among developers and also between different implementation variants of the same functionality. We therefore asked industrial experts, who have experience in the respective DSLs, to rate the complexity of semantically equivalent implementations to find out which metric best represents their complexity perception and whether there are differences between implementation variants. The examples used in the study were inspired, in parts, by an open source library for IEC 61131-3. We found that the Cognitive Complexity metric is a useful representation of the software complexity in the domain, as it best matched the domain experts’ perception of complexity. These metrics should therefore be supported by the analysis tools.
BE-AWARE: an ontology-based adaptive robotic manipulation framework

Autonomous service robots are conceived to work in semi-structured and complex human environments performing a wide range of tasks and, hence, one of their main challenges is to be able to adapt the stages of the perceive-plan-execute cycle to perturbations ranging from small deviations on the poses of objects to large unexpected changes in the environment, as well as to recover from potential failures.To advance in this direction, this paper proposes an ontology-based manipulation framework where reasoning is used to enhance perception with situation awareness, planning with domain awareness and execution with the awareness of the execution structures.The combination of these different types of awareness allows the robot to have different adaptation capabilities.The conceptual schema of the framework is presented and discussed and the main future implementation challenges are pointed out.
Bad Smells in IEC 61499-based Control Software: Sniffing out Shotgun Surgery

Bad smells are code patterns which have a negative influence on software, often impacting its quality and maintainability. Because of these negative effects, it is important to detect and avoid them. In the domain of software engineering, the topic of bad smells has been widely researched. On the other hand, in the area of industrial automation and PLC programming languages, bad smells have not received the same amount of attention. Shotgun Surgery is a code smell which we can find when a functionality is used in a certain number of places in code. It starts smelling when every change to the functionality requires many smaller changes to the places where it’s used. If we take a closer look at Shotgun Surgery in the domain of IEC 61499, we see that the general trend persists and that there is no adequate research done on it. In this paper, we want to identify Shotgun Surgery scenarios and showcase multiple existing and new approaches for its detection in IEC 61499.
InteLiv: An Architecture for Graph-Based Dynamic Context Modeling for Smart Living

Advancements in the Internet of Things (IoT) applied by automation systems lead to an increasing and flexible interconnection of various devices and the generation of voluminous data. In the domain of home automation for instance, having a multitude of interconnected data leveraged to knowledge could enhance user-centeredness and energy efficiency, among other use cases. However, the data as well as data sources are dynamic and heterogeneous, posing a great challenge to unfold their full capabilities. Thus, a unifying and correspondingly dynamic architecture is needed for supporting the flexibility and extensibility of such systems and data sources. In this paper, we present the concept and implementation of a context-aware INTElligent LIVing (inteLiv) architecture. Consisting of a data layer, a context layer and a service layer, the architecture abstracts the heterogeneous data of the linked devices via a middleware. The resulting system enables the collection of data across networked heterogeneous devices and models the collected data by using a unified dynamic context model. A hybrid approach was chosen for the designed context model: A combination of an ontology-based and property graph-based model represent a core aspect of this work as opposed to common static ontological representations and pre-defined context-based use cases. In the following contribution, the inteLiv ontology is first defined to describe the concepts of the intelligent living system. Based on the modeled context, reasoning algorithms also enable the derivation of new context information. The context is stored as a knowledge graph and can be used for context-aware applications, which can be designed during runtime. Furthermore, a user interaction with the system is possible via a web interface.
Automatic Generation of Submodel-Specific Classes with Predefined Meta-information Based on Submodel Templates

Asset Administration Shells are an essential component of Industry 4.0, and submodels are crucial to their functioning. However, the current method of generating submodels involves writing custom code, which can lead to code duplication and inconsistencies across submodels. To address these challenges, we propose an automatic generation of submodel-specific classes with predefined meta-information based on submodel templates. Our proposed solution aims to reduce code duplication, provide support in integrated development environments, ensure consistency across submodels, and facilitate the immediate publishing of submodel software development kits. This paper discusses the current approaches for instantiating submodels and their problems, the proposed approach, its requirements, advantages, and an outlook.
FPGA Accelerated CNN Inference and Online Retraining for UAVs with Minimal Power and Performance Overhead

Unmanned Aerial Vehicles (UAVs) are quickly becoming a very important component in many applications, such as search and rescue. In many of these applications, Deep Neural Networks (DNNs) are used, especially for obstacle detection and avoidance. Furthermore, online retraining may be required for adaptation to new environments. This paper shows how to enable UAVs equipped with an FPGA-based controller to achieve the required accuracy, whether in the inference phase or in the retraining phase, with minimal impact on power and performance. Dynamic Function eXchange (DFX) is used to download the retraining module bitstream utilizing Posit16 multipliers when the UAV navigates a new environment. Otherwise, a Posit8 multiplier (lower in precision) based implementation is used for the inference procedure that requires less precision than the retraining procedure. An implementation on the EP4CE22F17C6 FPGA is presented to show that a Posit16 multiplier has very close power utilization and performance to that of a Posit8 multiplier.
Knowledge Representation of Asset Information and Performance in OT Environments

To make a management decision for the Operational Technology (OT) environment, obtaining OT asset information from a plant is essential. Key Performance Indicators (KPIs) can play a crucial role in evaluating the performance of the manufacturing process, minimizing production costs, assessing process effectiveness, and investigating various settings to improve performance, operations, and quality in OT. This paper aims to provide a detailed examination of how Asset Information Mining Sources (AIMSs) support the process for acquiring both static and runtime OT asset information. Once we have obtained accurate OT asset information, we can calculate the KPIs and incorporate them into a knowledge representation using an ontology-based model. This model can assist the management team in making informed decisions and provide a roadmap for exploring further automation possibilities.
Asset Administration Shells as Data Layer for Enabling Automated Simulation-based Engineering

The Asset Administration Shell offers the capabilities for unified and interoperable data exchange across the entire life-cycle of an asset. Therefore, using Asset Administration Shells as a data layer is a practical option for facilitating the exchange of information in Simulation-based Engineering. The feasibility of this approach is demonstrated through an implementation, showing that incorporating the Asset Administration Shell as a data layer for standardized data exchange can yield significant benefit for the Simulation-based Engineering process.
Development of Anomaly Detection with Variable Contexts on Refrigeration Data

Anomaly detection in time series data is a well-known research topic and of great interest to the industry. However, industrial systems are complex due to the nature of real world plants and their interrelationships in the overall system. Industrial plants often cannot be considered in isolation because their environment affects them. Thus, industrial plants have variable contexts in which they are embedded. The combination of a variable context with the information of the plant poses a challenge in modeling AI approaches for anomaly detection.In the field of refrigeration, machine learning approaches are still in their early stages. In live systems, simple threshold values are often still used. There are only a few published approaches to detect error situations using machine learning on such data.In this work, the challenges to such a model architecture are analyzed and a dataset of real refrigeration plants is built. Then, models for detection and classification are explored and their applicability to this type of data is evaluated. Furthermore, models are combined to add context to improve the results compared to models without context information.
Dirichlet Process Log Skew-Normal Mixture with a Missing-at-Random-Covariate in Insurance Claim Analysis

In actuarial practice, the modeling of total losses tied to a certain policy is a nontrivial task due to complex distributional features. In the recent literature, the application of the Dirichlet process mixture for insurance loss has been proposed to eliminate the risk of model misspecification biases. However, the effect of covariates as well as missing covariates in the modeling framework is rarely studied. In this article, we propose novel connections among a covariate-dependent Dirichlet process mixture, log-normal convolution, and missing covariate imputation. As a generative approach, our framework models the joint of outcome and covariates, which allows us to impute missing covariates under the assumption of missingness at random. The performance is assessed by applying our model to several insurance datasets of varying size and data missingness from the literature, and the empirical results demonstrate the benefit of our model compared with the existing actuarial models, such as the Tweedie-based generalized linear model, generalized additive model, or multivariate adaptive regression spline.
Multi-object detection for crowded road scene based on ML-AFP of YOLOv5

Aiming at the problem of multi-object detection such as target occlusion and tiny targets in road scenes, this paper proposes an improved YOLOv5 multi-object detection model based on ML-AFP (multi-level aggregation feature perception) mechanism. Since tiny targets such as non-motor vehicle and pedestrians are not easily detected, this paper adds a micro target detection layer and a double head mechanism to improve the detection ability of tiny targets. Varifocal loss is used to achieve a more accurate ranking in the process of non-maximum suppression to solve the problem of target occlusion, and this paper also proposes a ML-AFP mechanism. The adaptive fusion of spatial feature information at different scales improves the expression ability of network model features, and improves the detection accuracy of the model as a whole. Our experimental results on multiple challenging datasets such as KITTI, BDD100K, and show that the accuracy, recall rate and mAP value of the proposed model are greatly improved, which solves the problem of multi-object detection in crowded road scenes.
Similarity Distribution Density: An Optimized Approach to Outlier Detection

When dealing with uncertain data, traditional model construction methods often ignore or filter out noise data to improve model performance. However, this simple approach can lead to insufficient data utilization, model bias, reduced detection ability, and decreased robustness of detection models. Outliers can be considered as data that are inconsistent with other patterns at certain specific moments and are not always negative data, so their emergence is not always bad. In the process of data analysis, outliers play a crucial role in sample vector recognition, missing value processing, and model stability verification. In addition, unsupervised models have very high computation costs when recognizing outliers, especially non-parameterized unsupervised models. To solve the above problems, we used semi-supervised learning processes and used similarity as a negative selection criterion to propose a local density verification detection model (Vd-LOD). This model establishes similarity pseudo-labels for multi-label and multi-type samples, verifies the accuracy of outlier values based on local outlier factors, and increases the detector’s sensitivity to outliers. The experimental results show that under different parameter settings with varying outlier quantities, Vd-LOD outperforms other detection models in terms of the significant increase in average time consumption caused by verifying the presence of relationships, while also achieving an approximate 6% improvement in average detection accuracy.
KVMod—A Novel Approach to Design Key-Value NoSQL Databases

The growth of structured, semi-structured, and unstructured data produced by the new applications is a result of the development and expansion of social networks, the Internet of Things, web technology, mobile devices, and other technologies. However, as traditional databases became less suitable to manage the rapidly growing quantity of data and variety of data structures, a new class of database management systems named NoSQL was required to satisfy the new requirements. Although NoSQL databases are generally schema-less, significant research has been conducted on their design. A literature review presented in this paper lets us claim the need to create modeling techniques to describe how to structure data in NoSQL databases. Key-value is one of the NoSQL families that has received too little attention, especially in terms of its design methodology. Most studies have focused on the other families, like column-oriented and document-oriented. This paper aims to present a design approach named KVMod (key-value modeling) specific to key-value databases. The purpose is to provide to the scientific community and engineers with a methodology for the design of key-value stores using the maximum automation and therefore the minimum human intervention, which equals the minimum number of errors. A software tool called KVDesign has been implemented to automate the proposed methodology and, thus, the most time-consuming database modeling tasks. The complexity is also discussed to assess the efficiency of our proposed algorithms.
Research on performance variations of classifiers with the influence of pre-processing methods for Chinese short text classification

Text pre-processing is an important component of a Chinese text classification. At present, however, most of the studies on this topic focus on exploring the influence of preprocessing methods on a few text classification algorithms using English text. In this paper we experimentally compared fifteen commonly used classifiers on two Chinese datasets using three widely used Chinese preprocessing methods that include word segmentation, Chinese specific stop word removal, and Chinese specific symbol removal. We then explored the influence of the preprocessing methods on the final classifications according to various conditions such as classification evaluation, combination style, and classifier selection. Finally, we conducted a battery of various additional experiments, and found that most of the classifiers improved in performance after proper preprocessing was applied. Our general conclusion is that the systematic use of preprocessing methods can have a positive impact on the classification of Chinese short text, using classification evaluation such as macro-F1, combination of preprocessing methods such as word segmentation, Chinese specific stop word and symbol removal, and classifier selection such as machine and deep learning models. We find that the best macro-f1s for categorizing text for the two datasets are 92.13% and 91.99%, which represent improvements of 0.3% and 2%, respectively over the compared baselines.