Reviewerly/exHarmony huggingface dataset:
A Multi-Attentive Pyramidal Model for Visual Sentiment Analysis

Visual sentiment analysis aims to recognize emotions from visual contents. It is a very useful yet challenging task, especially when fine-grained emotions (such as love, joy, surprise, sadness, fear, anger, disgust, and anxiety) are analyzed. Existing methods based on convolutional neural networks learn sentiment representations based on global visual features, while ignoring the fact that both the local regions of the images and their relationships can have impact on sentiment representation learning. To address this limitation, in this paper, we propose a new MultiAttentive Pyramidal model (MAP) for visual sentiment analysis. The model performs pyramidal segmentation and pooling upon the visual feature blocks obtained from a fully convolutional network, aiming to extract local visual features from multiple local regions at different scales of the global image. It then implants a self-attention mechanism to mine the associations between local visual features, and achieves the final sentiment representation. Extensive experiments on six benchmark datasets show the proposed MAP model outperforms the state-of-the-art methods in visual sentiment analysis.
A computational approach to lexical polysemy in Ancient Greek

Abstract Language is a complex and dynamic system. If we consider word meaning, which is the scope of lexical semantics, we observe that some words have several meanings, thus displaying lexical polysemy. In this article, we present the first phase of a project that aims at computationally modelling Ancient Greek semantics over time. Our system is based on Bayesian learning and on the Diorisis Ancient Greek corpus, which we have built for this purpose. We illustrate preliminary results in light of expert annotation, and take this opportunity to discuss the role of computational systems and human analysis in a complex research area like historical semantics. On the one hand, computational approaches allow us to model large corpora of texts. On the other hand, a long and rich scholarly tradition in Ancient Greek has provided us with valuable insights into the mechanisms of semantic change (cf. e.g. Leiwo, M. (2012). Introduction: variation with multiple faces. In Leiwo, M., Halla-aho, H., and Vierros, M. (eds), Variation and Change in Greek and Latin, Helsinki: Suomen Ateenan-instituutin säätiö, pp. 1–11.). In this article, we show that these qualitative analyses can be leveraged to support and complement the computational modelling.
LSC: Online auto-update smart contracts for fortifying blockchain-based log systems

Smart contracts allow verifiable operations to be executed in blockchains, bringing new possibilities for trust establishment in trustless scenarios. However, smart contracts are cumbersome when used as security mechanisms in security scenarios due to two reasons: they have limited power and are inert to changes. In order to mitigate the two problems of employed smart contracts, we propose LSC, a framework for online auto-update smart contracts in blockchain-based log systems, to enable self-adaptive log anomaly detection via smart contracts. Time-varying log anomaly detection patterns are extracted by self-adaptive machine learning log anomaly analysis and are continuously fed to the contracts. The framework allows smart contracts to be automatically updated to express the patterns in low-cost ways. The anomaly detection strategies for audit log systems are shared and collaboratively enforced amongst network nodes to defend against targeted detection evasion. We provide a plain prototype as a proof of the feasibility and efficiency of LSC in log systems.
A Method for Extending Ontologies with Application to the Materials Science Domain

In the materials science domain the data-driven science paradigm has become the focus since the beginning of the 2000s. A large number of research groups and communities are building and developing data-driven workflows. However, much of the data and knowledge is stored in different heterogeneous data sources maintained by different groups. This leads to a reduced availability of the data and poor interoperability between systems in this domain. Ontology-based techniques are an important way to reduce these problems and a number of efforts have started. In this paper we investigate efforts in the materials science, and in particular in the nanotechnology domain, and show how such ontologies developed by domain experts, can be improved. We use a phrase-based topic model approach and formal topical concept analysis on unstructured text in this domain to suggest additional concepts and axioms for the ontology that should be validated by a domain expert. We describe the techniques and show the usefulness of the approach through an experiment where we extend two nanotechnology ontologies using approximately 600 titles and abstracts.
Centered kernel alignment inspired fuzzy support vector machine

Support vector machine (SVM) is a theoretically well motivated algorithm developed from statistical learning theory which has shown impressive performance in many fields. In spite of its success, it still suffers from the noise sensitivity problem originating from the assumption that each training point has equal importance or weight in the training process. To relax this problem, the SVM was extended to the fuzzy SVM (FSVM) by applying a fuzzy membership to each training point such that different training points can make different contributions to the learning of the decision surface. Although well-determined fuzzy memberships can improve classification performance, there are no general guidelines for their construction. In this paper, inspired by the centered kernel alignment (CKA), which measures the degree of similarity between two kernels (or kernel matrices), we propose a new fuzzy membership function calculation method in which a heuristic function derived from the CKA is used to calculate the dependence between a data point and its associated label. Although the CKA induced FSVM is similar to the kernel target alignment (KTA) induced FSVM, there is actually a critical difference. Without that centering, the definition of alignment does not correlate well with the performance of learning machines. Extensive experiments are performed on real-world data sets from the UCI benchmark repository and the application domain of computational biology which validate the superiority of the proposed FSVM model in terms of several classification performance measures.
MPS-Net: Learning to recover surface normal for multispectral photometric stereo

Multispectral Photometric Stereo (MPS) estimates per-pixel surface normals from one single image captured under three colored (red, green and blue) light sources. Unlike traditional Photometric Stereo, MPS can therefore be used in dynamic scenes for single frame reconstruction. However, MPS is challenging due to the tangle of the illumination, surface reflectance and camera response, causing inaccurate estimation of surface normal. Existing approaches rely on either extra depth information or materials calibration strategies, thus limiting its usage in practical applications. In this paper, we propose a Multispectral Photometric Stereo Network (MPS-Net) to solve this under-determined system. The MPS-Net takes the single multispectral image and an initial surface normal estimation obtained from this image itself, and outputs an accurate surface normal map, where no extra depth or materials calibration information is required. We show that the MPS-Net is not constrained to Lambertian surfaces and can be applied to surfaces with complex reflectance. We evaluated the MPS-Net using both synthetic and real objects of various materials. Our experiment results show that the MPS-Net outperforms the state-of-the-art approaches.
Transparent Machine Education of Neural Networks for Swarm Shepherding Using Curriculum Design

Swarm control is a difficult problem due to the need to guide a large number of agents simultaneously. We cast the problem as a shepherding problem, similar to biological dogs guiding a group of sheep towards a goal. The shepherd needs to deal with complex and dynamic environments and make decisions in order to direct the swarm from one location to another. In this paper, we design a novel curriculum to teach an artificial intelligence empowered agent to shepherd in the presence of the large state space associated with the shepherding problem and in a transparent manner. The results show that a properly designed curriculum could indeed enhance the speed of learning and the complexity of learnt behaviours.
An Efficiently Searchable Encrypted Data Structure for Range Queries

At CCS 2015 Naveed et al. presented first attacks on efficiently searchable encryption, such as deterministic and order-preserving encryption. These plaintext guessing attacks have been further improved in subsequent work, e.g. by Grubbs et al. in 2016. Such cryptanalysis is crucially important to sharpen our understanding of the implications of security models. In this paper we present an order-preserving encryption scheme in the form of an efficiently searchable, encrypted data structure that is provably secure against these and even more powerful chosen plaintext attacks. Our data structure supports logarithmic-time search with linear space complexity. The indices of our data structure can be used to search by standard comparisons and hence allow easy retrofitting to existing database management systems. We implemented our scheme and show that its search time overhead is only 10 ms compared to non-secure search on a database with 1 million entries.
Wearable technology for infant health monitoring: a survey

Continuous health monitoring for an infant is crucial for detecting and preventing several diseases. Recent advancement in wearable technology has given rise to the development of wearable infant health monitoring systems (WIHMSs). These systems provide an edge over conventional infant health monitoring systems which are not only bulky and uncomfortable for the infants but are also limited to the clinical settings. This study reviews some state-of-the-art WIHMS, mentions the advantages, limitations and the challenges faced by such systems. This study, for the first time, contributes an appraisal of research prototypes and commercially available systems for WIHMS, which is the need of the day.
Self-Concept Clarity and the Bodily Self: Malleability Across Modalities

The self has fascinated scholars for centuries. Although theory suggests that the self-concept (cognitive self-understanding) and bodily self (pre-reflective awareness of one’s body) are related, little work has examined this notion. To this end, in Study 1, participants reported on self-concept clarity (SCC) and completed the rubber hand illusion (RHI), a paradigm in which synchronous (vs. asynchronous) stimulation between a prosthetic hand and one’s own hand leads one to “embody” the prosthetic hand. Whereas participants were equally susceptible to the RHI during synchronous stroking, low-SCC individuals were more vulnerable to the illusion during asynchronous stroking, when the effect is unwarranted. Conceptually replicating and extending this finding, in Study 2, low-SCC individuals were more susceptible to the body-swap illusion—the impression that another person’s body is one’s own. These findings suggest that a clear sense of self implies clarity and stability of both the self-concept and the bodily self.
Visual-Texual Emotion Analysis With Deep Coupled Video and Danmu Neural Networks

User emotion analysis toward videos is to automatically recognize the general emotional status of viewers from the multimedia content embedded in the online video stream. Existing works fall into two categories: 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">visual-based</i> methods, which focus on visual content and extract a specific set of features of videos. However, it is generally hard to learn a mapping function from low-level video pixels to high-level emotion space due to great intra-class variance. 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">textual-based</i> methods, which focus on the investigation of user-generated comments associated with videos. The learned word representations by traditional linguistic approaches typically lack emotion information and the global comments usually reflect viewers’ high-level understandings rather than instantaneous emotions. To address these limitations, in this paper, we propose to jointly utilize video content and user-generated texts simultaneously for emotion analysis. In particular, we introduce exploiting a new type of user-generated texts, i.e., “danmu,” which are real-time comments floating on the video and contain rich information to convey viewers’ emotional opinions. To enhance the emotion discriminativeness of words in textual feature extraction, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Emotional Word Embedding</i> (EWE) to learn text representations by jointly considering their semantics and emotions. Afterward, we propose a novel visual-textual emotion analysis model with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep Coupled Video and Danmu Neural networks</i> (DCVDN), in which visual and textual features are synchronously extracted and fused to form a comprehensive representation by deep-canonically-correlated-autoencoder-based multi-view learning. Through extensive experiments on a self-crawled real-world video-danmu dataset, we prove that DCVDN significantly outperforms the state-of-the-art baselines.
Combination of Active Learning and Semi-Supervised Learning under a Self-Training Scheme

One of the major aspects affecting the performance of the classification algorithms is the amount of labeled data which is available during the training phase. It is widely accepted that the labeling procedure of vast amounts of data is both expensive and time-consuming since it requires the employment of human expertise. For a wide variety of scientific fields, unlabeled examples are easy to collect but hard to handle in a useful manner, thus improving the contained information for a subject dataset. In this context, a variety of learning methods have been studied in the literature aiming to efficiently utilize the vast amounts of unlabeled data during the learning process. The most common approaches tackle problems of this kind by individually applying active learning or semi-supervised learning methods. In this work, a combination of active learning and semi-supervised learning methods is proposed, under a common self-training scheme, in order to efficiently utilize the available unlabeled data. The effective and robust metrics of the entropy and the distribution of probabilities of the unlabeled set, to select the most sufficient unlabeled examples for the augmentation of the initial labeled set, are used. The superiority of the proposed scheme is validated by comparing it against the base approaches of supervised, semi-supervised, and active learning in the wide range of fifty-five benchmark datasets.
Augur: Modeling the Resource Requirements of ConvNets on Mobile Devices

Convolutional Neural Networks (ConvNets/CNNs) have revolutionized the research in computer vision, due to their ability to capture complex patterns, resulting in high inference accuracies. However, the increasingly complex nature of these neural networks means that they are particularly suited for server computers with powerful GPUs. We envision that deep learning applications will be eventually widely deployed on mobile devices, e.g., smartphones, self-driving cars, and drones. Therefore, in this paper, we aim to understand the resource requirements of CNNs on mobile devices in terms of compute time, memory, and power. First, by deploying several popular CNNs on different mobile CPUs and GPUs, we measure and analyze the performance and resource usage for the CNNs on a layerwise granularity. Our findings point out the potential ways of optimizing the CNN pipelines on mobile devices. Second, we model resource requirements of core computations of CNNs. Finally, based on the measurement and modeling, we build and evaluate our modeling tool, Augur, which takes a CNN configuration (descriptor) as the input and estimates the compute time, memory, and power requirements of the CNN, to give insights about whether and how efficiently a CNN can be run on a given mobile platform.
Universal Approximation of Fuzzy Relation Models by Semitensor Product

A universal approximation of multi-input multi-output (MIMO) fuzzy systems is proposed in this article based on a fuzzy relation matrix (FRM) method. The fuzzy reasoning operation in FRM is realized by the semitensor product (STP) technique. The theoretical proof is provided to estimate approximation accuracy of the FRM models for MIMO systems. Its effectiveness is verified by simulation. Simulation results show that the proposed fuzzy design technology is efficient for fuzzy systems modeling, and the general universal approximation theory can be extended to the MIMO fuzzy systems. The proposed uniform matrix expression can be used to design fuzzy systems with FRM models and to approximate nonlinear functions with required accuracy.
Deriving the Variance of the Discrete Fourier Transform Test Using Parseval’s Theorem

The discrete Fourier transform test is a randomness test included in NIST SP800-22. However, the variance of the test statistic is smaller than expected and the theoretical value of the variance is not known. Hitherto, the mechanism explaining why the former variance is smaller than expected has been qualitatively explained based on Parseval's theorem. In this paper, we explore this quantitatively and derive the variance using Parseval's theorem under particular assumptions. Numerical experiments are then used to show that this derived variance is robust.
Consistent Brain Ageing Synthesis

Brain ageing is associated with morphological changes and cognitive degeneration, and can be affected by neurodegenerative diseases which can accelerate the ageing process. The ability to separate accelerated from healthy ageing is useful from a diagnostic perspective and towards developing subject-specific models of progression. In this paper we start with the 'simpler' problem of synthesising age-progressed 2D slices. We adopt adversarial training to learn the joint distribution of brain images and ages, and simulate aged images by a network conditioned on age (a continuous variable) encoded as an ordinal embedding vector. We introduce a loss to help preserve subject identity despite that we train with cross-sectional (unpaired) data. To evaluate the quality of aged images, a pre-trained age predictor is used to estimate an apparent age. We show qualitatively and quantitatively that our method can progressively synthesise realistic brain images of different target ages.
Integrating 3D Geometry of Organ for Improving Medical Image Segmentation

Prior knowledge of organ shape and location plays an important role in medical imaging segmentation. However, traditional 2D/3D segmentation methods usually operate as pixel-wise/voxel-wise classifiers where their training objectives are not able to incorporate the 3D shape knowledge explicitly. In this paper, we proposed an efficient deep shape-aware network to learn 3D geometry of the organ. More specifically, the network uses a 3D mesh representation in a graph-based CNN which can handle the shape inference and accuracy propagation effectively. After integrating the shape-aware module into the backbone FCNs and jointly training the full model in the multi-task framework, the discriminative capability of intermediate feature representations is increased for both geometry and segmentation regularizations on disentangling subtly correlated tasks. Experimental results show that the proposed network can not only output accurate segmentation, but also generate smooth 3D mesh simultaneously which can be used for further 3D shape analysis.
Encoding Histopathological WSIs Using GNN for Scalable Diagnostically Relevant Regions Retrieval

The research on content-based histopathological image retrieval (CBHIR) has become popular in recent years. CBHIR systems provide auxiliary diagnosis information for pathologists by searching for and returning regions that are contently similar to the region of interest (ROI) from a pre-established database. To retrieve diagnostically relevant regions from a database that consists of histopathological whole slide images (WSIs) for query ROIs is challenging and yet significant for clinical applications. In this paper, we propose a novel CBHIR framework for regions retrieval from WSI-database based on hierarchical graph neural networks (GNNs). Compared to the present CBHIR framework, the structural information of WSI is preserved by the proposed model, which makes the retrieval framework more sensitive to regions that are similar in tissue distribution. Moreover, benefited from the hierarchical GNN structures, the proposed framework is scalable for both the size and shape variation of ROIs. It allows the pathologist defining the query region using free curves. Thirdly, the retrieval is achieved by binary codes and hashing methods, which makes it very efficient and thereby adequate for practical large-scale WSI-database. The proposed method is validated on a lung cancer dataset and compared to the state-of-the-art methods. The proposed method achieved precisions above 82.4% in the irregular region retrieval task, which are superior to the state-of-the-art methods. The average time of retrieval is 0.514 ms.
Affect Recognition in Code Review: An In-situ Biometric Study of Reviewer’s Affect

Code reviews are an important practice in software development that increases team productivity and improves product quality. They are also examples of remote, computer-mediated asynchronous communications which are prone to the loss of affective information. Prior research has focused on sentiment analysis in source codes, as positive affect has been linked to developer productivity. Although methods of sentiment analysis have advanced, challenges remain due to numerous domain-specific expressions, subtle nuance, and indications of sentiment. In this paper, we uncover the potential for 1) nonverbal behavioral signals such as conventional typing, and 2) indirect physiological measures (eye gaze, GSR, touch pressure) to reveal genuine affective states in in situ code review in a large software company. Nonverbal behavioral signals of 33 professional software developers were recorded unobtrusively while they worked on their daily code reviews. After analyzing these signals using Linear Mixed Effect Models, we observe that affect presented in the written comments is associated with prolonged typing duration. Using physiological features, a trained Random Forest classifier can predict post-task valence with 90.0% accuracy (F1-score = 0.937) and arousal with 83.9% accuracy (F1-score = 0.856). The results show promise for the creation of intelligent affect-aware interfaces for code review.