Reviewerly/exHarmony huggingface dataset:
Compositional Video Prediction

We present an approach for pixel-level future prediction given an input image of a scene. We observe that a scene is comprised of distinct entities that undergo motion and present an approach that operationalizes this insight. We implicitly predict future states of independent entities while reasoning about their interactions, and compose future video frames using these predicted states. We overcome the inherent multi-modality of the task using a global trajectory-level latent random variable, and show that this allows us to sample diverse and plausible futures. We empirically validate our approach against alternate representations and ways of incorporating multi-modality. We examine two datasets, one comprising of stacked objects that may fall, and the other containing videos of humans performing activities in a gym, and show that our approach allows realistic stochastic video prediction across these diverse settings. See project website (https://judyye.github.io/CVP/) for video predictions.
Future Frame Prediction Using Convolutional VRNN for Anomaly Detection

Anomaly detection in videos aims at reporting anything that does not conform the normal behaviour or distribution. However, due to the sparsity of abnormal video clips in real life, collecting annotated data for supervised learning is exceptionally cumbersome. Inspired by the practicability of generative models for semi-supervised learning, we propose a novel sequential generative model based on variational autoencoder (VAE) for future frame prediction with convolutional LSTM (ConvLSTM). To the best of our knowledge, this is the first work that considers temporal information in future frame prediction based anomaly detection framework from the model perspective. Our experiments demonstrate that our approach is superior to the state-of-the-art methods on three benchmark datasets.
Optimize TSK Fuzzy Systems for Regression Problems: Minibatch Gradient Descent With Regularization, DropRule, and AdaBound (MBGD-RDA)

Takagi–Sugeno–Kang (TSK) fuzzy systems are very useful machine learning models for regression problems. However, to our knowledge, there has not existed an efficient and effective training algorithm that ensures their generalization performance and also enables them to deal with big data. Inspired by the connections between TSK fuzzy systems and neural networks, we extend three powerful neural network optimization techniques, i.e., minibatch gradient descent (MBGD), regularization, and AdaBound, to TSK fuzzy systems, and also propose three novel techniques (DropRule, DropMF, and DropMembership) specifically for training TSK fuzzy systems. Our final algorithm, MBGD with regularization, DropRule, and AdaBound, can achieve fast convergence in training TSK fuzzy systems, and also superior generalization performance in testing. It can be used for training TSK fuzzy systems on datasets of any size; however, it is particularly useful for big datasets, on which currently no other efficient training algorithms exist.
Systematic Review of Good Teaching Practices with ICT in Spanish Higher Education. Trends and Challenges for Sustainability

Good teaching practices are developed in order to improve student learning. With technological and social development, educational practices increasingly demand the use of technology, so having good teaching practices with Information and Communication Technologies (ICT) is essential. At the same time, the Sustainable Development Agenda of the United Nations for the year 2030 is setting the principles of sustainable action for today’s society. In this context, the following objectives were proposed: to identify the good teaching practices with ICT that are being developed in Spanish higher education and to establish the relationship between good teaching practices with ICT and the Sustainable Development Goals (SDG). For this purpose, the method of systematic review of the literature based on the PRISMA protocol was used. A total of 27 documents published in the period 2008–2019 on good teaching practices with ICT in Spanish higher education were analyzed. Among the results, the emerging trend of good practices with ICT, the use of a Virtual Learning Environment (VLE) as the main technology in good practices, and the majority of good practices in the development of SDG #4 (Quality Education) are highlighted. Finally, it is a challenge for teachers to include all these aspects in their classroom teaching.
PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

This article tackles a new problem setting: reinforcement learning with pixel-wise rewards (pixelRL) for image processing. After the introduction of the deep Q-network, deep RL has been achieving great success. However, the applications of deep reinforcement learning (RL) for image processing are still limited. Therefore, we extend deep RL to pixelRL for various image processing applications. In pixelRL, each pixel has an agent, and the agent changes the pixel value by taking an action. We also propose an effective learning method for pixelRL that significantly improves the performance by considering not only the future states of the own pixel but also those of the neighbor pixels. The proposed method can be applied to some image processing tasks that require pixel-wise manipulations, where deep RL has never been applied. Besides, it is possible to visualize what kind of operation is employed for each pixel at each iteration, which would help us understand why and how such an operation is chosen. We also believe that our technology can enhance the explainability and interpretability of the deep neural networks. In addition, because the operations executed at each pixels are visualized, we can change or modify the operations if necessary. We apply the proposed method to a variety of image processing tasks: image denoising, image restoration, local color enhancement, and saliency-driven image editing. Our experimental results demonstrate that the proposed method achieves comparable or better performance, compared with the state-of-the-art methods based on supervised learning. The source code is available on https://github.com/rfuruta/pixelRL.
Sentiment Analysis of Social Media Twitter with Case of Anti-LGBT Campaign in Indonesia using Naïve Bayes, Decision Tree, and Random Forest Algorithm

People use social media as a means to express their thoughts, interests, and opinions on various things. Thousands of submissions occur every day on every social media. Everyone can express their opinions through social media freely. These opinions contain positive, negative and neutral sentiments on a topic. The case study taken by researchers is the Anti-LGBT campaign in Indonesia. The case was taken because the Anti-LGBT campaign was widely discussed by the Indonesian people on Twitter’s social media. If you want to know the tendency of public comments on the Anti-LGBT campaign in Indonesia, is it positive, negative, or neutral, then a sentiment analysis is conducted. The algorithm used in conducting sentiment analysis is Naïve Bayes because it has a high degree of accuracy in classifying sentiment analysis. The stages in conducting sentiment analysis in this study are preprocessing data, processing data, classification, and evaluation. The sentiment analysis obtained in this study shows that Twitter users in Indonesia give more neutral comments. In this study, an accuracy of 86.43% was obtained from testing data using Naïve Bayes Algorithm in RapidMiner tools, where the accuracy is higher than the other algorithms, Decision Tree and Random Forest which is 82.91%.
Blockchain-Based Charging Coordination Mechanism for Smart Grid Energy Storage Units

Energy storage units (ESUs) enable several attractive features of modern smart grids such as enhanced grid resilience, effective demand response, and reduced bills. However, uncoordinated charging of ESUs stresses the power system and can lead to a blackout. In this paper, we leverage the blockchain and smart contracts to build a decentralized charging coordination mechanism without the need for a centralized charging coordinator. First ESUs should use tokens for anonymously authenticate themselves to the blockchain. Then each ESU sends a charging request that contains its State-of-Charge (SoC), Time-to-complete-charge (TCC) and amount of required charging to the smart contract address on the blockchain. The smart contract will then run the charging coordination mechanism in a self-executed manner such that ESUs with the highest priorities are charged in the present time slot while charging requests of lower priority ESUs are deferred to future time slots. In this way, each ESU can make sure that charging schedules are computed correctly. Finally, we have implemented the proposed mechanism on the Ethereum test-bed blockchain, and our analysis shows that execution cost can be acceptable in terms of gas consumption while enabling decentralized charging coordination with increased transparency, reliability, and privacy preserving.
Deep Learning for User Interest and Response Prediction in Online Display Advertising

Abstract User interest and behavior modeling is a critical step in online digital advertising. On the one hand, user interests directly impact their response and actions to the displayed advertisement (Ad). On the other hand, user interests can further help determine the probability of an Ad viewer becoming a buying customer. To date, existing methods for Ad click prediction, or click-through rate prediction, mainly consider representing users as a static feature set and train machine learning classifiers to predict clicks. Such approaches do not consider temporal variance and changes in user behaviors, and solely rely on given features for learning. In this paper, we propose two deep learning-based frameworks, $${\hbox {LSTM}}_{\mathrm{cp}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msub><mml:mtext>LSTM</mml:mtext><mml:mi>cp</mml:mi></mml:msub></mml:math> and $${\hbox {LSTM}}_{\mathrm{ip}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msub><mml:mtext>LSTM</mml:mtext><mml:mi>ip</mml:mi></mml:msub></mml:math> , for user click prediction and user interest modeling. Our goal is to accurately predict (1) the probability of a user clicking on an Ad and (2) the probability of a user clicking a specific type of Ad campaign. To achieve the goal, we collect page information displayed to the users as a temporal sequence and use long short-term memory (LSTM) network to learn features that represents user interests as latent features. Experiments and comparisons on real-world data show that, compared to existing static set-based approaches, considering sequences and temporal variance of user requests results in improvements in user Ad response prediction and campaign specific user Ad click prediction.
Multi-View Multi-Label Learning With Sparse Feature Selection for Image Annotation

In image analysis, image samples are always represented by multiple view features and associated with multiple class labels for better interpretation. However, multiple view data may include noisy, irrelevant and redundant features, while multiple class labels can be noisy and incomplete. Due to the special data characteristic, it is hard to perform feature selection on multi-view multi-label data. To address these challenges, in this paper, we propose a novel multi-view multi-label sparse feature selection (MSFS) method, which exploits both view relations and label correlations to select discriminative features for further learning. Specifically, the multi-labeled information is decomposed into a reduced latent label representation to capture higher level concepts and correlations among multiple labels. Multiple local geometric structures are constructed to exploit visual similarities and relations for different views. By taking full advantage of the latent label representation and multiple local geometric structures, the sparse regression model with an l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm and an Frobenius norm (F-norm) penalty terms is utilized to perform hierarchical feature selection, where the F-norm penalty performs high-level (i.e., view-wise) feature selection to preserve the informative views and the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm penalty conducts low-level (i.e., row-wise) feature selection to remove noisy features. To solve the proposed formulation, we also devise a simple yet efficient iterative algorithm. Experiments and comparisons on real-world image datasets demonstrate the effectiveness and potential of MSFS.
Privacy-Preserving Distributed Averaging via Homomorphically Encrypted Ratio Consensus

In this article, we develop distributed iterative algorithms that enable the components of a multicomponent system, each with some integer initial value, to asymptotically compute the average of their initial values, without having to reveal to other components the specific value they contribute to the average calculation. We assume a communication topology captured by an arbitrary strongly connected digraph, in which certain nodes (components) might be curious but not malicious (i.e., they execute the distributed protocol correctly, but try to identify the initial values of other nodes). We first develop a variation of the so-called ratio consensus algorithm that operates exclusively on integer values and can be used by the nodes to asymptotically obtain the average of their initial (integer) values, by taking the ratio of two integer values they maintain and iteratively update. Assuming the presence of a trusted node (i.e., a node that is not curious and can be trusted to set up a cryptosystem and not reveal any decrypted values of messages it receives), we describe how this algorithm can be adjusted using homomorphic encryption to allow the nodes to obtain the average of their initial values while ensuring their privacy (i.e., without having to reveal their initial value). We also extend the algorithm to handle situations where multiple nodes set up cryptosystems and privacy is preserved as long as one of these nodes can be trusted (i.e., the ratio of trusted nodes over the nodes that set up cryptosystems decreases).
Design methodology for 360° immersive video applications: the case study of a cultural heritage virtual tour

Abstract Three hundred sixty–degree (360°) immersive video applications for Head Mounted Display (HMD) devices offer great potential in providing engaging forms of experiential media solutions especially in Cultural Heritage education. Design challenges emerge though by this new kind of immersive media due to the 2D form of resources used for their construction, the lack of depth, the limited interaction and the need to address the sense of presence. In addition, the use of Virtual Reality (VR) headsets often causes nausea, or motion sickness effects imposing further implications in moderate motion design tasks. This paper introduces a methodological categorisation of tasks and techniques for the design of 360° immersive video applications. Following the design approach presented, a testbed application has been created as an immersive interactive virtual tour at the historical centre of the city of Rethymno in Crete, Greece, which has undergone user trials. Based on the analysis of the results of this study, a set of design guidelines for the implementation of 360° immersive video virtual tours is proposed.
ALDONAr: A hybrid solution for sentence-level aspect-based sentiment analysis using a lexicalized domain ontology and a regularized neural attention model

Aspect-based sentiment analysis allows one to compute the sentiment for an aspect in a certain context. One problem in this analysis is that words possibly carry different sentiments for different aspects. Moreover, an aspect's sentiment might be highly influenced by the domain-specific knowledge. In order to tackle these issues, in this paper, we propose a hybrid solution for sentence-level aspect-based sentiment analysis using A Lexicalized Domain Ontology and a Regularized Neural Attention model (ALDONAr). The bidirectional context attention mechanism is introduced to measure the influence of each word in a given sentence on an aspect's sentiment value. The classification module is designed to handle the complex structure of a sentence. The manually created lexicalized domain ontology is integrated to utilize the field-specific knowledge. Compared to the existing ALDONA model, ALDONAr uses BERT word embeddings, regularization, the Adam optimizer, and different model initialization. Moreover, its classification module is enhanced with two 1D CNN layers providing superior results on standard datasets.
SACPC: A framework based on probabilistic linguistic terms for short text sentiment analysis

Short text sentiment analysis is challenging because short texts are limited in length and lack context. Short texts are usually rather ambiguous because of polysemy and the typos these texts contain. Polysemy is the coexistence of multiple word meanings and commonly appears in every language. Various uses of a word may assign the word both positive and negative meanings. In previous studies, the variability of words is often ignored, which may cause analysis errors. In this study, to resolve this problem, we proposed a novel text representation model named Word2PLTS for short text sentiment analysis by introducing probabilistic linguistic terms sets (PLTSs) and the relevant theory. In this model, every word is represented as a PLTS that fully describes the possibilities for the sentiment polarity of the word. Then, by using support vector machines (SVM), a novel sentiment analysis and polarity classification framework named SACPC is obtained. This framework is a technique that combines supervised learning and unsupervised learning. We compare SAPCP to lexicon-based approaches and machine learning approaches on three benchmark datasets. A noticeable improvement in performance is achieved. To further verify the superiority of SAPCP, state of the art performance comparisons are conducted, and the results are impressive.
Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints

Non-negative matrix factorization (NMF) has attracted sustaining attention in multi-view clustering, because of its ability of processing high-dimensional data. In order to learn the desired dimensional-reduced representation, a natural scheme is to add constraints to traditional NMF. Motivated by that the clustering performance is affected by the orthogonality of inner vectors of both the learned basis matrices and the representation matrices, a novel NMF model with co-orthogonal constraints is designed to deal with the multi-view clustering problem in this paper. For solving the proposed model, an efficient iterative updating algorithm is derived. And the corresponding convergence is proved, together with the analysis to its computational complexity. Experiments on five datasets are performed to present the advantages of the proposed algorithm against the state-of-the-art methods.
A RUGD Dataset for Autonomous Navigation and Visual Perception in Unstructured Outdoor Environments

Research in autonomous driving has benefited from a number of visual datasets collected from mobile platforms, leading to improved visual perception, greater scene understanding, and ultimately higher intelligence. However, this set of existing data collectively represents only highly structured, urban environments. Operation in unstructured environments, e.g., humanitarian assistance and disaster relief or off-road navigation, bears little resemblance to these existing data. To address this gap, we introduce the Robot Unstructured Ground Driving (RUGD) dataset with video sequences captured from a small, unmanned mobile robot traversing in unstructured environments. Most notably, this data differs from existing autonomous driving benchmark data in that it contains significantly more terrain types, irregular class boundaries, minimal structured markings, and presents challenging visual properties often experienced in off road navigation, e.g., blurred frames. Over 7, 000 frames of pixel-wise annotation are included with this dataset, and we perform an initial benchmark using state-of-the-art semantic segmentation architectures to demonstrate the unique challenges this data introduces as it relates to navigation tasks.
Scene-Adaptive Remote Sensing Image Super-Resolution Using a Multiscale Attention Network

Remote sensing image super-resolution has always been a major research focus, and many deep-learning-based algorithms have been proposed in recent years. However, since the structure of remote sensing images tends to be much more complex than that of natural images, several difficulties still remain for remote sensing images super-resolution. First, it is difficult to depict the nonlinear mapping between high-resolution (HR) and low-resolution (LR) images of different scenes with the same model. Second, the wide range of scales within the ground objects in remote sensing images makes it difficult for single-scale convolution to effectively extract features of various scales. To address the above-mentioned issues, we propose a multiscale attention network (MSAN) to extract the multilevel features of remote sensing images. The basic component of MSAN is the multiscale activation feature fusion block (MAFB). In addition, a scene-adaptive super-resolution strategy for remote sensing images is employed to more accurately describe the structural characteristics of different scenes. The experiments undertaken on several data sets confirm that the proposed algorithm outperforms the other state-of-the-art algorithms, in both evaluation indices and visual results.
One-Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking

Deformable image registration is a very important field of research in medical imaging. Recently multiple deep learning approaches were published in this area showing promising results. However, drawbacks of deep learning methods are the need for a large amount of training datasets and their inability to register unseen images different from the training datasets. One shot learning comes without the need of large training datasets and has already been proven to be applicable to 3D data. In this work we present a one shot registration approach for periodic motion tracking in 3D and 4D datasets. When applied to a 3D dataset the algorithm calculates the inverse of the registration vector field simultaneously. For registration we employed a U-Net combined with a coarse to fine approach and a differential spatial transformer module. The algorithm was thoroughly tested with multiple 4D and 3D datasets publicly available. The results show that the presented approach is able to track periodic motion and to yield a competitive registration accuracy. Possible applications are the use as a stand-alone algorithm for 3D and 4D motion tracking or in the beginning of studies until enough datasets for a separate training phase are available.