Miracle-dz/UniFAR-FTUs huggingface dataset:
Investigation of Day-ahead Price Forecasting Models in the Finnish Electricity Market

: The electricity market is a rather complex market and the prices depend on several different factors. The price dynamics are bound to get even more volatile, with a stronger integration between European electricity markets and the increasing share of renewable energy sources. Therefore, the development of accurate electricity price forecasting methods has increasing importance in the ﬁeld. This paper investigates the performance of several deep learning models for the Finnish electricity market. The investigation comprehends different architectures types, data aggregation schemes as well as pre-training method. In this manner, this work does not only presents new forecasting methods but also gives valuable comparison between approaches.
Anomaly Detection in Communication Networks of Cyber-physical Systems using Cross-over Data Compression

: Anomaly detection in operational communication data of cyber-physical systems is an important part of any monitoring activity in such systems. This paper suggests a new method of anomaly detection named cross-over data compression (CDC). The method belongs to the group of information theoretic approaches and is based on the notion of Kullback-Leibler Divergence. Data blocks are compressed by a Sequitur-like algorithm and the resulting grammars describing the compression are applied cross-over to the all the other data blocks. Divergences are calculated from the length of the different compressions and the mean values of these divergences are used to classify the data in normal and anomalous. The paper describes the method in detail and shows the results derived from a real-world example (communication data from a substation).
Privacy-preserving Surveillance Methods using Homomorphic Encryption

Abstract: Data analysis and machine learning methods often involve the processing of cleartext data, and where this could breach the rights to privacy. Increasingly, we must use encryption to protect all states of the data: in-transit , at-rest , and in-memory . While tunnelling and symmetric key encryption are often used to protect data in-transit and at-rest, our major challenge is to protect data within memory, while still retaining its value. Ho-momorphic encryption, thus, could have a major role in protecting the rights to privacy, while providing ways to learn from captured data. Our work presents a novel use case and evaluation of the usage of homomorphic encryption and machine learning for privacy respecting state surveillance.
Object Detection for Autonomous Driving: Motion-aid Feature Calibration Network

: Object detection is a critical task for autonomous driving. The latest progress in deep learning research for object detection has built a solid contribution to the development of autonomous driving. However, direct employment of the state-of-the-art object detectors from image to video is problematic. Object appearances in videos have more variations, e.g., video defocus, motion blur, truncation, etc. Such variations of objects in a video have fewer occurrences in still image and could compromise the detection results. To address these problems, we build a fast and accurate deep learning framework, motion-assist calibration network (MFCN) for video detection. Our model leverages the motion pattern of temporal coherence on video features. It calibrates and aggregates features of detected objects from previous frames along the spatial changes to improve the feature representations on the current frame. The whole model architecture is trained end-to-end which boosts the detection accuracy. Validations on the KITTI and ImageNet dataset show that MFCN can improve the baseline results by 10.3% and 9.9% mAP respectively. Compared with other state-of-the-art models, MFCN achieves a leading performance on KITTI benchmark competition. Results indicate the effectiveness of the proposed model which could facilitate the autonomous driving system.
Evaluating the Effect of Justification and Confidence Information on User Perception of a Privacy Policy Summarization Tool

Privacy policies are long and cumbersome for users to read. To support understanding of the information contained in privacy policies, automated analysis of textual data can be used to obtain a summary of their content, which can then be presented in a shorter, more usable format. However, these tools are not perfect and users indicate concern about the trustworthiness of their results. Although some of these tools provide information about their performance, the effect if this information has not been investigated. In order to address this, we conducted an experimental study to evaluate whether providing explanatory information such as result confidence and justification influences users’s understanding of the privacy policy content and perception of the tool. The results suggest that presenting a justification of the results, in the form of a policy fragment, can increase intention to use the tool and improve perception of trustworthiness and usefulness. On the other hand, showing only a result confidence percentage did not improve perception of the tool, nor did it help to communicate the possibility of incorrect results. We discuss these results and their implications for the design of privacy policy summarization tools.
Android Run-time Permission Exploitation User Awareness by Means of Formal Methods

: Our mobile devices store a lot of sensitive and critical information. Moreover, considering the ability of smartphones and tables to detect the position and to record audio, it is not absolutely an exaggeration to admit that potentially our devices can easily spy on us. The ability to perform these crucial tasks must be granted by the user at the time of their ﬁrst use by accepting the so-called permissions. The problem is that in Android, once these permissions are granted, the application can always use them without notifying the user. In this paper, we propose a methodology, based on formal methods, aimed to detect the exact point in the code (in term of package, class and method) of an Android application where a permission is invoked at run-time. Moreover we design a tool able to advise the user whether a permission is invoked, in this way the user can be informed about the application behaviour.
An Automated and Distributed Machine Learning Framework for Telecommunications Risk Management

: Automation and scalability are currently two of the main challenges of Machine Learning. This paper proposes an automated and distributed ML framework that automatically trains a supervised learning model and pro-duces predictions independently of the dataset and with minimum human input. The framework was designed for the domain of telecommunications risk management, which often requires supervised learning models that need to be quickly updated by non-ML-experts and trained on vast amounts of data. Thus, the architecture assumes a distributed environment, in order to deal with big data, and Automated Machine Learning (Au-toML), to select and tune the ML models. The framework includes several modules: task detection (to detect if classiﬁcation or regression), data preprocessing, feature selection, model training, and deployment. In this paper, we detail the model training module. In order to select the computational technologies to be used in this module, we ﬁrst analyzed the capabilities of an initial set of ﬁve modern AutoML tools: Auto-Keras, Auto-Sklearn, Auto-Weka, H2O AutoML, and TransmogrifAI. Then, we performed a benchmarking of the only two tools that address distributed ML (H2O AutoML and TransmogrifAI). Several comparison experiments were held using three real-world datasets from the telecommunications domain (churn, event forecasting, and fraud detection), allowing us to measure the computational effort and predictive capability of the AutoML tools.
On Satisfisfiability Modulo Theories in Continuous Multi-Agent Path Finding: Compilation-based and Search-based Approaches Compared

: Multi-agent path ﬁnding (MAPF) in continuous space and time with geometric agents, i.e. agents of various geometric shapes moving smoothly between predeﬁned positions, is addressed in this paper. We analyze a new solving approach based on satisﬁability modulo theories (SMT) that is designed to obtain makespan optimal solutions. The standard MAPF is a task of navigating agents in an undirected graph from given starting vertices to given goal vertices so that agents do not collide with each other in vertices or edges of the graph. In the continuous version (MAPF R ), agents move in an n -dimensional Euclidean space along straight lines that interconnect predeﬁned positions. Agents themselves are geometric objects of various shapes occupying certain volume of the space - circles, polygons, etc. For simplicity, we work with circular omni-directional agents having constant velocities in the 2D plane. As agents can have different shapes/sizes and are moving smoothly along lines, a movement along certain lines done with small agents can be non-colliding while the same movement may result in a collision if performed with larger agents. Such a distinction rooted in the geometric reasoning is not present in the standard MAPF. The SMT-based approach for MAPF R called SMT-CBS R reformulates the well established Conﬂict-based Search (CBS) algorithm in terms of SMT. Lazy generation of decision variables and constraints is the key idea behing SMT-CBS. Each time a new conﬂict is discovered, the underlying encoding is extended with new variables and constraints to eliminate the conﬂict. We compared SMT-CBS R and adaptations of CBS for the continuous variant of MAPF experimentally.
Hair Shading Style Transfer for Manga with cGAN

: Coloring line drawings is an important process in creating artwork. In coloring, a shading style is where an artist’s style is the most noticeable. Artists spend a great deal of time and effort creating art. It is thus difﬁcult for beginners to draw speciﬁc shading styles, and even experienced people have a hard time trying to draw many different styles. Features of a shading styles appear prominently in the hair region of human characters. In many cases, hair is drawn using a combination of the base color, the highlights, and the shadows. In this study, we propose a method for transferring the shading style used on hair in one drawing to another drawing. This method uses a single reference image for training and does not need a large data set. This paper describes the framework, transfer results, and discussions. The transfer results show the following: when transferring the shading style to the line drawing by the same artist, the method can detect the hair region relatively well, and the transfer result is indistinguishable from the transfer target in some shading styles. In addition, the evaluation results show that our method has higher scores than an existing automatic colorizing service.
Email Image Spam Classification based on ResNet Convolutional Neural Network

The problem with email image spam classification is known from the year 2005. There are several approaches to this task. Lately, those approaches use convolutional neural networks (CNN). We propose a novel approach to the image spam classification task. Our approach is based on CNN and transfer learning, namely Resnet v1 used for semantic feature extraction and one layer Feedforward Neural Network for classification. We have shown that this approach can achieve state-of-the-art performance on publicly available datasets. 99% F1score on two datasets (Dredze et al., 2007), Princeton and 96% F1-score on the combination of these datasets. Due to the availability of GPUs, this approach may be used for just-in-time classification in anti-spam systems handling huge amounts of emails. We have observed also that mentioned publicly available datasets are no longer representative. We overcame this limitation by using a much richer dataset from a one-week long real traffic of the freemail provider Email.cz. The training data annotation was created by user labeling of the emails. The image spam (and image ham even more) tackles privacy issues. We overcame it by publishing extracted feature vectors with associated classes (instead of images itself). This data does not violate privacy issues. We have published Email.cz image spam dataset v1 via the AcademicTorrents platform and propose a system, which achieves up to 96% F1-score with presented model architecture on this novel dataset. Providing our dataset to the community may help others with solving similar tasks.
Deep Learning of Heuristics for Domain-independent Planning

: Automated planning deals with the problem of ﬁnding a sequence of actions leading from a given state to a desired state. The state-of-the-art automated planning techniques exploit informed forward search guided by a heuristic, where the heuristic (under)estimates a distance from a state to a goal state. In this paper, we present a technique to automatically construct an efﬁcient heuristic for a given domain. The proposed approach is based on training a deep neural network using a set of solved planning problems from the domain. We use a novel way of generating features for states which doesn’t depend on usage of existing heuristics. The trained network can be used as a heuristic on any problem from the domain of interest without any limitation on the problem size. Our experiments show that the technique is competitive with popular domain-independent heuristic.
Reaching Agreement in an Interactive Group Recommender System

: For a group recommender system, it is important to recommend an item that can be accepted by all group members. This paper proposes a group recommender system where preferences elicited from group members are used to select an item that is agreeable to all of them. In this system, an agent that corresponds to each group member manages estimation of the corresponding user’s preferences. Virtual negotiation is conducted among these agents to ﬁnd an appropriate item to recommend, and the selected item is presented to group members. If it is not accepted, the system asks members to relax their requirements and accordingly updates its recommendation. We report and discuss the results of simulation experiments with different personality types of conﬂict resolution and different conversation strategies.
Transfer Learning for Digital Heritage Collections: Comparing Neural Machine Translation at the Subword-level and Character-level

: Transfer learning via pre-training has become an important strategy for the efﬁcient application of NLP methods in domains where only limited training data is available. This paper reports on a focused case study in which we apply transfer learning in the context of neural machine translation (French–Dutch) for cultural heritage metadata (i.e. titles of artistic works). Nowadays, neural machine translation (NMT) is commonly applied at the subword level using byte-pair encoding (BPE), because word-level models struggle with rare and out-of-vocabulary words. Because unseen vocabulary is a signiﬁcant issue in domain adaptation, BPE seems a better ﬁt for transfer learning across text varieties. We discuss an experiment in which we compare a subword-level to a character-level NMT approach. We pre-trained models on a large, generic corpus and ﬁne-tuned them in a two-stage process: ﬁrst, on a domain-speciﬁc dataset extracted from Wikipedia, and then on our metadata. While our experiments show comparable performance for character-level and BPE-based models on the general dataset, we demonstrate that the character-level approach nevertheless yields major downstream performance gains during the subsequent stages of ﬁne-tuning. We therefore conclude that character-level translation can be beneﬁcial compared to the popular subword-level approach in the cultural heritage domain.
Conflicts Resolution and Situation Awareness in Heterogeneous Multi-agent Missions using Publish-subscribe Technique and Inferential Reasoning

: In this paper, we propose a priority-based publish-subscribe approach to tackle reasoning in beliefs conﬂicts for a heterogeneous multi-agent mission. Agents subscribe to other agents’ topics and rank them based on agents’ situation awareness. Bayesian Belief Network (BBN) was used in maintaining agents’ belief and recorded mission information could be used for the BBN training using conjugate gradient descent or expectation-maximization algorithms. The output of the training is the learned network for agents’ predictions, estimations, and conclusions. We also propose an agent’s self presumption inferential reasoning where agents learned heuristics and used them for future inferences. We test the system by using a team of heterogeneous Unmanned Aerial Vehicles (UAVs) with different sensor proﬁles and capacities tasked together to perform forest ﬁre searching. To verify belief and settle conﬂicts, agents follow these steps: sequentially assess the prioritized publish-subscribe topics, inferential reasoning using the learned network, inferential reasoning using logical propositions, and learning process. From our experiment, the BBN training and prediction perfection grow up with the increase in the number of training data. Future work focuses on obtaining the optimal number of samples needed for effective prediction, effective agents’ beliefs merging, communication protocol, and bandwidth utilization.
A Study of Classification of Texts into Categories of Cybersecurity Incident and Attack with Topic Models

To improve and automate cybersecurity incident handling in security operations centers (SOCs) and computer emergency response teams (CERTs), security intelligences extracted from various internal and external sources, including incident response playbooks, incident reports in each SOCs and CERTs, the National Vulnerability Database, and social media, must be utilized. In this paper, we apply various topic models to classify text related to cybersecurity intelligence and incidents according to topics derived from incidents and cyber attacks. We analyze cybersecurity incident reports and related text in our CERT and security blog posts using naive latent Dirichlet allocation (LDA), seeded LDA, and labeled LDA topic models. Labeling text based on designated categories is difficult and time-consuming. Training the seeded model does not require text to be labeled; instead, seed words are given to allow the model to infer topic-word and document-topic distributions for the text. We show that a seeded topic model can be used to extract and classify intelligence in our CERT, and we infer text more precisely compared with a supervised topic model.
Hybrid Tiled Convolutional Neural Networks (HTCNN) Text Sentiment Classification

The tiled convolutional neural network (tiled CNN) has been applied only to computer vision for learning invariances. We adjust its architecture to NLP to improve the extraction of the most salient features for sentiment analysis. Knowing that the major drawback of the tiled CNN in the NLP field is its inflexible filter structure, we propose a novel architecture called hybrid tiled CNN that applies a filter only on the words that appear in the similar contexts and on their neighbor words (a necessary step for preventing the loss of some n-grams). The experiments on the datasets of IMDB movie reviews and SemEval 2017 demonstrate the efficiency of the hybrid tiled CNN that performs better than both CNN and tiled CNN.
Sentence Compression on Domains with Restricted Labeled Data Availability

: The majority amount of information available on the Web remains unstructured, i.e., text documents from articles, news, blog posts, product reviews, forums discussions, among others. Given the huge amount of textual content continuously produced on the Web, it has been challenging for users to read and consume every document. Text summarization refers to the technique of shortening long pieces of text. The intention is to create a coherent and ﬂuent summary having only the main points outlined in the document. Sentence compression can improve text summarization by removing redundant information, preserving the grammaticality and the important content of the original sentences. In this paper, we propose a sentence compression neural network model that achieved promising results compared to other neural network-based models, even when trained with smaller amounts of data. Rather than training the model only with the words from the training set, the proposed model was trained with different features extracted from the texts. This improves the ability of the model to decide whether or not to retain each word in the compressed sentence.
Radical Text Detection based on Stylometry

: The Internet has become an effective tool for terrorist and radical groups to spread their propaganda. One of the current problems is to detect these radical messages in order to block them or promote counter-narratives. In this work, we propose the use of stylometric methods for characterizing radical messages. We have used a machine learning approach to classify radical texts based on a corpus of news from radical sources such as the so-called ISIS online magazines Dabiq and Rumiyah, as well as news from general newspapers. The results show that stylometric features are effective for radical text classiﬁcation.
Improving Semantic Similarity of Words by Retrofitting Word Vectors in Sense Level

: This paper presents an approach for retroﬁtting pre-trained word representations into sense level representations to improve semantic distinction of words. We use semantic relations as positive and negative examples to reﬁne the results of a pre-trained model instead of integrating them into the objective functions used during training. We experimentally evaluate our approach on two word similarity tasks by retroﬁtting six datasets generated from three widely used techniques for word representation using two different strategies. Our approach signiﬁcantly and consistently outperforms three state-of-the-art retroﬁtting approaches.
Predicting a Song Title from Audio Embeddings on a Pretrained Image-captioning Network

: Finding the name of a song from a piece played without the lyrics remains a long-standing challenge to music recognition services. In this work, we propose the use of a neural architecture that combines deep learned image features and sequence modeling to automate the task of predicting the song title from an audio time series. To feed our network with a visual representation, we transform the sound signal into a two-dimensional spectrogram. Our novelty lies in model training on the state-of-the-art Conceptual Captions dataset to generate image descriptions, jointly with inference on the Million Song and Free Music Archive test sets to produce song titles. We present extensive quantitative analysis of our experiments and show that using k -beam search our model achieved an out-domain BLEU score of 45.1 compared to in-domain performance of 61.3.