Reviewerly/exHarmony huggingface dataset:
Activity Recognition and Segmentation Approaches to Multimodal Lifelog Data

Lifelogging is a phenomenon whereby an individual digitally records his/her personal life experiences, for a variety of purposes. Activity recognition and segmentation is fundamental to many of the use cases in lifelogging. However, detecting sufficiently robust user activity boundaries that could be deployed with confidence in a subjective real-world setting remains a challenge. In this paper, we extend our previous work on identifying a better activity recognition and segmentation approach to multimodal lifelog data, primarily through the introduction of automatic thresholding techniques, but also through revising the criteria for selecting the most appropriate size of sliding window when evaluating the proposed algorithms. We use an open and publicly available lifelog test collection over a time period of 27 days with manual annotations and manually groundtruthed activities.
The Impact of Executive Functions-Based Intervention on School Engagement and Academic Self-Concept of Students with Dyslexia

The Impact of Executive Functions-Based Intervention on School Engagement and Academic Self-Concept of Students with Dyslexia
JumpAR -- Augmented Reality Platformer

The puzzle and dexterity mobile game JumpAR combines well-known Jump n' Run elements with the opportunities of Augmented Reality (AR). The goal of the game is to complete a parkour of platforms with a character and to collect rewards on the way. But with one trick - the construction of the level is done by the player themselves. The bases of the different platforms and obstacles are placed in the real-world surroundings of the player. These then get augmented on the mobile device and form a parkour through which the player must maneuver the character to successfully reach the exit platform. This highly interactive game leads to individual creative game areas wherever the player starts to build the parkour: on the floor, on a desk or on even on the lawn. It can be played either alone or in groups and allows both, a collaborative and competitive gameplay by creating challenging parkours for oneself or others.
Leveraging AI in Service Automation Modeling: From Classical AI Through Deep Learning to Combination Models

With the advent of cloud, new generations of digital services are being conceived to respond to the ever-growing demands and expectations of the market place. In parallel, automations are becoming an essential enabler for successful management of these services. With such importance being placed on digital services, automated management of these services – in particular, automated incident resolution – becomes a key issue for both the provider and the users. The challenge facing automation providers lies in variability and the frequently changing nature of the monitoring tickets that provide the primary input to automation. Despite the value of the correct automation at the correct time, it is also important to remember that triggering an incorrect automation may damage the smooth operation of the business. In this paper, we discuss AI modeling for automation recommendations. We describe a wide range of experiments which allowed us to conclude an optimal method with respect to accuracy and speed acceptable to service providers.
GDALR: An Efficient Model Duplication Attack on Black Box Machine Learning Models

Trained Machine learning models are core components of proprietary products. Business models are entirely built around these ML powered products. Such products are either delivered as a software package (containing the trained model) or they are deployed on cloud with restricted API access for prediction. In ML-as-a-service, users are charged per-query or per-hour basis, generating revenue for businesses. Models deployed on cloud could be vulnerable to Model Duplication attacks. Researchers found ways to exploit these services and clone the functionalities of black box models hidden in the cloud by continuously querying the provided APIs. After successful execution of attack, the attacker does not require to pay the cloud service provider. Worst case scenario, attackers can also sell the cloned model or use them in their business model.Traditionally attackers use convex optimization algorithm like Gradient Descent with appropriate hyper-parameters to train their models. In our research we propose a modification to traditional approach called as GDALR (Gradient Driven Adaptive Learning Rate) that dynamically updates the learning rate based on the gradient values. This results in stealing the target model in comparatively less number of epochs, decreasing the time and cost, hence increasing the efficiency of the attack. This shows that sophisticated attacks can be launched for stealing the black box machine learning models which increases risk for MLaaS based businesses.
The Concepts of Proof and Ground

Convinced that the concepts of proof and ground should be a main concern of proof theory as mathesis universalis, I discuss in this paper how one may try to develop Bolzano’s ideas about grounds in the light of Gentzen’s ideas about the justification of inferences. At the end, I briefly consider whether the result can give some insights about the concepts of proof and ground. I proceed in a somewhat experimental fashion, starting with proposals for how to specify grounds for propositions of various forms that stay fairly close to Bolzano’s ideas. I then apply results from present day proof theory and show how one has to depart successively from Bolzano in order to get a tenable concept of ground. The final definition of ground that I arrive at may be seen as a combination of ideas from Bolzano and Gentzen and later results in proof theory.
The application of particle swarm optimization using neural network to optimize classification of employee performance assessment

The measurement of employee value performance in an institution is very important for the future evaluation and planning. The employee performance assessment absolutely must be done to determine the accomplishments to be achieved by every employee. Policy makers are difficult to determine the optimal classification toward assessment of the performance appraisal. Selection of the best attributes weights in optimizing Neural Network for the performance appraisal that has been not optimal yet and the effect on level of accuracy that is generated. Practitioners of Particle Swarm Optimization (PSO) on Neural Network (NN) are expected to optimize the selection of the best attribute weights to the performance appraisal classification resulting in increased accuracy.
Object Detection on FPGAs and GPUs by Using Accelerated Deep Learning

Object detection and recognition is one of the main tasks in many areas such as autonomous unmanned ground vehicles, robotic and medical image processing. Recently, deep learning has been used by many researchers in these areas when the data measure is large. In particular, one of the most up-to-date structures of deep learning, Convolutional Neural Networks (CNNs) has achieved great success in this field. Real-time works related to CNNs are carried out by using GPU-Graphics Processing Units. Although GPUs provides high stability, they requires high power, energy consumption, and large computational load problems. In order to overcome this problem, it has started to used the Field Programmable Gate Arrays (FPGAs). In this article, object detection and recognition procedures were performed using the ZYNQ XC7Z020 development board including both the ARM processor and the FPGA. Real-time object recognition has been made with the Movidius USB-GPU externally plugged into the FPGA. The results are given with figures.
Image synthesis and superresolution in medical imaging

Image synthesis and superresolution have a long history of research and use in medical imaging. Recently, with the emergence of powerful machine learning methods, both processes have risen to new levels of capability. This chapter presents some historical and technical background in each of these areas and describes some of the new approaches. Magnetic resonance imaging is the primary modality considered.
Better Code Reviews With Design by Contract

Design by contract (DBC) is a technique that improves the quality of your team's code. It yields code with both a logical and a procedural nature, where the contracts state declaratively what will happen, and the implementations procedurally cause the desired effect. The team can reason either logically, by using the contracts, or procedurally, by following the code line by line, but the former allows them to reason about far larger programs. It also creates conditions for deliberate practice so developers using DBC grow their design skills faster.
Recognising Irish Sign Language Using Electromyography

Sign language is the non-verbal communication used by people with hearing and speaking impairments. The automatic recognition of sign languages is usually based on video analysis of the signer though this is difficult when considering different light levels or the surrounding environment. The work in this paper uses electromyography (EMG) and focuses on letters of the Irish Sign Language (ISL) alphabet. EMG is the recording of the electrical activity produced to stimulate movement in the skeletal muscles. We capture muscle signals and inertial movement data using the Thalmic MYO armband and, in real time, recognise the ISL alphabet. Our implementation is based on signal processing, feature extraction and machine learning. The only input required to translate the ISL gestures are EMG and movement data, thus our approach is usable in scenarios where using video for automatic recognition video is not possible.
Evaluating Fiber Detection Models Using Neural Networks

Ceramic matrix composites are resistant materials that withstand high temperatures, but quality control of such composites depends on microtomography image analysis to enable the spatial analysis of fibers, matrix cracks detection, among others. While there are several approaches for fiber detection from microtomography, materials scientists lack computational schemes to validate the accuracy of different fiber detection models. This paper proposes a set of statistical methods to analyse images of CMC in 3D and visualize respective fiber beds, including a lossless data reduction algorithm. The main contribution is our method based on a convolutional neural network that enables evaluation of results from automated fiber detection models, particularly when compared with human curated datasets. We build all the algorithms using free tools to allow full reproducibility of the experiments, and illustrate our results using algorithms designed to probe sample content from gigabyte-size image volumes with minimalistic computational infrastructure.
Enhanced Single Shot MultiBox Detector for Pedestrian Detection

Our work focus on pedestrian detection with the enhanced Single Shot MultiBox Detector (SSD) [1]. We find that the most of the missed pedestrian targets are concentrated on scenes with dense targets and small-scale objects when we regard SSD as the detector. In order to promote the detection performance for small-scale pedestrian targets, an additional feature in the third convolutional block and dense connections among convolutional blocks are added to the original SSD network structure. In addition, our study found that fine-grained sampling of the global prior box could improve the detection accuracy for multiscale pedestrians. We change the matching strategy between global prior boxes and truth target boxes, and add an extra loss that can keep more information in dense targets situation. The experimental results on PRW [2], Caltech and VOC dataset demonstrate that our enhanced SSD detector can achieve competitive detection accuracy as well as real-time detection speed on pedestrian objects.
A New Approach of Steganography Using Center Sequential Technique

Abstract The quality of steganographic images can be determined from several aspects. One of the testing techniques on steganography is cropping which is part of robustness testing. Under cropping process, it is possible that the secret message cannot be extracted from the stego-image. This happen because the pixels values on the stego-image embedded message changed and the message content on the stego-image will be lost. In this research we propose a new approach for solving that problem by using the algorithm based on Sequential Technique where the input process starts from the center of the cover. The secret information in the form of image and the cover image is also an image. By using this method, the secret image is safe although the cropping is done from left, right, above or bottom of the stego image.
Linear predictor coefficient, power spectral analysis and two-layer feed forward network for bangla speech recognition

Within this research we investigates the tools and techniques how correctly recognize Bangla (Bengali) language. For feature extraction the power spectral analysis and linear predictor coefficient method considered. Speech samples taken from bangla phoneme, word, command and sentence. Each frame of a speech signal's spectrum segmented into number of fragments to take the mean absolute value from each fragment for pattern classification. A two-layer feed forward neural network with maximum likelihood method used. Features extraction by power spectral analysis gives little more accuracy for bangla phoneme and word recognition comparing to linear predictor coefficient analysis. Both methods provide average result for bangla command and sentence recognition. The mel frequency cepstral coefficient could be another approach for future research.
Synchronization between Sensors and Cameras in Movement Data Labeling Frameworks

Obtaining labeled data for activity recognition tasks is a tremendously time consuming, tedious, and labor-intensive task. Often, ground-truth video of the activity is recorded along with sensordata recorded during the activity. The data must be synchronized with the recorded video to be useful. In this paper, we present and compare two labeling frameworks that each has a different approach to synchronization. Approach A uses time-stamped visual indicators positioned on the data loggers. The approach results in accurate synchronization between video and data but adds more overhead and is not practical when using multiple sensors, subjects, and cameras simultaneously. Also, synchronization needs to be redone for each recording session. Approach B uses Real-Time Clocks (RTCs) on the devices for synchronization, which is less accurate but has several advantages: multiple subjects can be recorded on various cameras, it becomes easier to collect more data, and synchronization only needs to be done once across multiple recording sessions. Therefore, it is easier to collect more data which increases the probability of capturing an unusual activity. The best way forward is likely a combination of both approaches.
A Hybrid Recommendation System in Co-authorship Networks

Recommender systems are an important field that aim to direct users more quickly to their needs. Scientific research studies in academic field are mostly carried out by collaboration between scientists. Finding new collaborations and analyzing the quality of them are quite complicated situations. Therefore, developing a system that recommend strong collaborations for scientists can be an invaluable tool. In this study, we present a hybrid recommendation method that uses both collaborative filtering and a network-based method by examining the structure of the co-authorship network that was created by using a real database of scientific articles in computer science. In the constructed network, the nodes represent scientists and links represent the coauthorship relationships. If two scientists in the network have been co-author of at least one article, they are considered to be connected. Many of the present recommendation methods in this area take into account only the common neighbors of the nodes, while calculating the similarity of the node pairs in the network. To overcome these limited methods, we use the local community-based similarity indexes which also take into account the similarities between the common neighbors of the nodes. When the experimental results are examined, they prove the success of the proposed method.
Big data metrics: Time sensitivity analysis of multimedia news

With the daily release of the huge amount of new information, the web information such as video, image, audio and text information is growing dramatically. Most of this vast information changes over time. It may become ineffective, obsolete and worthless, and even affect user’s understanding of web information, degrade their experience. Therefore, analyzing this temporal-sensitive information in multimedia is a vital issue. To this end, this paper analyses the temporal sensitivity of multimedia web information to find what kind of information is temporal-sensitive, how this information is responsive to time, and how to evaluate the temporal sensitivity of information. We start with four types of features, that is time, content, user behavior and related multimedia news, then set up a triple model to depict news. By establishing the energy transfer relationship between news and related and similar news, time, and user behavior, we measure the energy, and use the change ratio of energy as the temporal sensitivity of news. The data set is a multimedia news corpus, including video, image and text news. In the experiment, we take the users’ comments as the validation set. The result basically matches the validation set, and it shows our metric is reasonable.
Compact Data Structures for Shortest Unique Substring Queries

Given a string T of length n, a substring $$u = T[i..j]$$ of T is called a shortest unique substring (SUS) for an interval [s, t] if (a) u occurs exactly once in T, (b) u contains the interval [s, t] (i.e. $$i \le s \le $$ $$ t \le j$$ ), and (c) every substring v of T with $$|v| < |u|$$ containing [s, t] occurs at least twice in T. Given a query interval $$[s, t] \subset [1, n]$$ , the interval SUS problem is to output all the SUSs for the interval [s, t]. In this article, we propose a $$4n + o(n)$$ bits data structure answering an interval SUS query in output-sensitive $$O(occ)$$ time, where $$occ$$ is the number of returned SUSs. Additionally, we focus on the point SUS problem, which is the interval SUS problem for $$s = t$$ . Here, we propose a $$\lceil (\log _2{3} + 1)n \rceil + o(n)$$ bits data structure answering a point SUS query in the same output-sensitive time.
Prediction intervals based soft sensor development using fuzzy information granulation and an improved recurrent ELM

With the increasing complexity of large-scale industrial production processes, the number of variable factors is increasing. As a result, it is demanding to predict process key variables accurately. Currently, most of soft sensor models using support vector regression and artificial neural networks are based on point prediction. The soft measurement models using the technique of point prediction can only track or fit set values. It is difficult to deal with the problem of system uncertainty and to make reliability analysis using the point prediction based soft sensors. To address this problem, this paper proposes a development method of soft sensor using the technique of prediction intervals. Under this condition, the prediction intervals instead of the point prediction of the stable operation of the industrial process system are used. The interval boundaries of the trend change can be utilized to quantify and estimate the associated uncertainty. The proposed prediction intervals based soft sensor is based on fuzzy information granularity and improved recurrent extreme learning machine. First, the fuzzy information granularity is adopted to get the lower bound, trend and upper bound of the interval. Secondly, an improved recurrent extreme learning machine is built to further enhance the ability of prediction intervals. In the improved extreme learning machine model, a feedback layer is adopted to store the hidden layer output, calculate the data trend change and dynamically update the outputs of the feedback layer. Third, the comprehensive interval evaluation function is used to evaluate the rationality of the interval results. Through case studies using a University of California Irvine dataset and the purified Terephthalic acid solvent system, the provided prediction intervals method can directly generate the upper and lower bounds for process key variables with high accuracy.
ViProVoQ

This paper presents a method for developing a consensus vocabulary to describe and evaluate the visual experience of videos. As a first result, a vocabulary characterizing the specific look of cinema-type video is presented. Such a vocabulary can be used to relate perceptual features of professional high-end image and video quality of experience (QoE) with the underlying technical characteristics and settings of the video systems involved in the creative content production process. For the vocabulary elicitation, a combination of different survey techniques was applied in this work. As the first step, individual interviews were conducted with experts of the motion picture industry on image quality in the context of cinematography. The data obtained from the interviews was used for the subsequent Real-time Delphi survey, where an extended group of experts worked out a consensus on key aspects of the vocabulary specification. Here, 33 experts were supplied with the anonymized results of the other panelists, which they could use to revise their own assessment. Based on this expert panel, the attributes collected in the interviews were verified and further refined, resulting in the final vocabulary proposed in this paper. Besides an attribute-based sensory evaluation of high-quality image, video and film material, applications of the vocabulary are the development of dimension-based image and video quality models, and the analysis of the multivariate relationship between quality-relevant perceptual attributes and technical system parameters.
Multimodal Deep Learning for Solar Irradiance Prediction

It is widely recognized that solar irradiance prediction is significant in the scheduling and operations of smart grid. Traditional ways of prediction primarily rely on statistical based models that take the current and the historical data as the inputs. However, solar irradiance is not purely dependant on its own features, but also affected by many factors from other modalities. Moreover, conventional models cannot deal with the sudden changes of solar irradiance very well due to the lack of information relevant to the predictive moment. To address these limitations, in this paper, we propose two multimodal deep learning models, namely hierarchical multimodal deep learning model (H-MDL) and similar-day based wavelet decomposition multimodal deep learning model (SW-MDL), to predict the solar irradiance. In particular, H-MDL consists a two-level hierarchical deep learning structure. In the 1st-level, latent features from each single modality are extracted, then deeply fused over the space of multimodal inputs, finally learn the joint representation in the 2nd hierarchy. By contrast, SW-MDL model utilizes the wavelet decomposition module to denoise the sequential input features and takes the impacts of the frequency properties of different frequency components into consideration as well. Moreover, SW-MDL deliberately constructs a new modality of similar-day irradiance data, and use it as an additional model input on account of the periodicity and repeatability characteristics between the predictive day and similar days in history. We carry out extensive experiments on a 12-year multimodal meteorologic dataset from the U.S. National Renewable Energy Laboratory (NREL). The obtained results validate H-MDL and SW-MDL outperform the state-of-the-art baseline models by a large margin.