Reviewerly/exHarmony huggingface dataset:
Training Passive Photonic Reservoirs With Integrated Optical Readout

As Moore's law comes to an end, neuromorphic approaches to computing are on the rise. One of these, passive photonic reservoir computing, is a strong candidate for computing at high bitrates (>10 Gb/s) and with low energy consumption. Currently though, both benefits are limited by the necessity to perform training and readout operations in the electrical domain. Thus, efforts are currently underway in the photonic community to design an integrated optical readout, which allows to perform all operations in the optical domain. In addition to the technological challenge of designing such a readout, new algorithms have to be designed in order to train it. Foremost, suitable algorithms need to be able to deal with the fact that the actual on-chip reservoir states are not directly observable. In this paper, we investigate several options for such a training algorithm and propose a solution in which the complex states of the reservoir can be observed by appropriately setting the readout weights, while iterating over a predefined input sequence. We perform numerical simulations in order to compare our method with an ideal baseline requiring full observability as well as with an established black-box optimization approach (CMA-ES).
Moving Object Detection in Video via Hierarchical Modeling and Alternating Optimization

In conventional wisdom of video modeling, the background is often treated as the primary target and foreground is derived using the technique of background subtraction. Based on the observation that foreground and background are two sides of the same coin, we propose to treat them as peer unknown variables and formulate a joint estimation problem, called Hierarchical modeling and Alternating Optimization (HMAO). The motivation behind our hierarchical extensions of background and foreground models is to better incorporate a priori knowledge about the disparity between background and foreground. For background, we decompose it into temporally low-frequency and high-frequency components for the purpose of better characterizing the class of video with dynamic background; for foreground, we construct a Markov random field prior at a spatially low resolution as the pivot to facilitate the noise-resilient refinement at higher resolutions. Built on hierarchical extensions of both models, we show how to successively refine their joint estimates under a unified framework known as alternating direction multipliers method. Experimental results have shown that our approach produces more discriminative background and demonstrates better robustness to noise than other competing methods. When compared against current state-of-the-art techniques, HMAO achieves at least comparable and often superior performance in terms of F-measure scores, especially for video containing dynamic and complex background.
Generation of Patient-Specific Cardiac Vascular Networks: A Hybrid Image-Based and Synthetic Geometric Model

Objective: In this paper, we propose an algorithm for the generation of a patient-specific cardiac vascular network starting from segmented epicardial vessels down to the arterioles. Method: We extend a tree generation method based on satisfaction of functional principles, named constrained constructive optimization, to account for multiple, competing vascular trees. The algorithm simulates angiogenesis under vascular volume minimization with flow-related and geometrical constraints adapting the simultaneous tree growths to patient priors. The generated trees fill the entire left ventricle myocardium up to the arterioles. Results: From actual vascular tree models segmented from CT images, we generated networks with 6000 terminal segments for six patients. These networks contain between 33 and 62 synthetic trees. All vascular models match morphometry properties previously described. Conclusion and significance: Image-based models derived from CT angiography are being used clinically to simulate blood flow in the coronary arteries of individual patients to aid in the diagnosis of disease and planning treatments. However, image resolution limits vessel segmentation to larger epicardial arteries. The generated model can be used to simulate the blood flow and derived quantities from the aorta into the myocardium. This is an important step for diagnosis and treatment planning of coronary artery disease.
Learning Object Context for Dense Captioning

Dense captioning is a challenging task which not only detects visual elements in images but also generates natural language sentences to describe them. Previous approaches do not leverage object information in images for this task. However, objects provide valuable cues to help predict the locations of caption regions as caption regions often highly overlap with objects (i.e. caption regions are usually parts of objects or combinations of them). Meanwhile, objects also provide important information for describing a target caption region as the corresponding description not only depicts its properties, but also involves its interactions with objects in the image. In this work, we propose a novel scheme with an object context encoding Long Short-Term Memory (LSTM) network to automatically learn complementary object context for each caption region, transferring knowledge from objects to caption regions. All contextual objects are arranged as a sequence and progressively fed into the context encoding module to obtain context features. Then both the learned object context features and region features are used to predict the bounding box offsets and generate the descriptions. The context learning procedure is in conjunction with the optimization of both location prediction and caption generation, thus enabling the object context encoding LSTM to capture and aggregate useful object context. Experiments on benchmark datasets demonstrate the superiority of our proposed approach over the state-of-the-art methods.
Residual Attribute Attention Network for Face Image Super-Resolution

Facial prior knowledge based methods recently achieved great success on the task of face image super-resolution (SR). The combination of different type of facial knowledge could be leveraged for better super-resolving face images, e.g., facial attribute information with texture and shape information. In this paper, we present a novel deep end-to-end network for face super resolution, named Residual Attribute Attention Network (RAAN), which realizes the efficient feature fusion of various types of facial information. Specifically, we construct a multi-block cascaded structure network with dense connection. Each block has three branches: Texture Prediction Network (TPN), Shape Generation Network (SGN) and Attribute Analysis Network (AAN). We divide the task of face image reconstruction into three steps: extracting the pixel level representation information from the input very low resolution (LR) image via TPN and SGN, extracting the semantic level representation information by AAN from the input, and finally combining the pixel level and semantic level information to recover the high resolution (HR) image. Experiments on benchmark database illustrate that RAAN significantly outperforms state-of-the-arts for very low-resolution face SR problem, both quantitatively and qualitatively.
Argumentation for Explainable Scheduling

Mathematical optimization offers highly-effective tools for finding solutions for problems with well-defined goals, notably scheduling. However, optimization solvers are often unexplainable black boxes whose solutions are inaccessible to users and which users cannot interact with. We define a novel paradigm using argumentation to empower the interaction between optimization solvers and users, supported by tractable explanations which certify or refute solutions. A solution can be from a solver or of interest to a user (in the context of ‘what-if’ scenarios). Specifically, we define argumentative and natural language explanations for why a schedule is (not) feasible, (not) efficient or (not) satisfying fixed user decisions, based on models of the fundamental makespan scheduling problem in terms of abstract argumentation frameworks (AFs). We define three types of AFs, whose stable extensions are in one-to-one correspondence with schedules that are feasible, efficient and satisfying fixed decisions, respectively. We extract the argumentative explanations from these AFs and the natural language explanations from the argumentative ones.
Action Knowledge Transfer for Action Prediction with Partial Videos

Predicting action class from partially observed videos, which is known as action prediction, is an important task in computer vision field with many applications. The challenge for action prediction mainly lies in the lack of discriminative action information for the partially observed videos. To tackle this challenge, in this work, we propose to transfer action knowledge learned from fully observed videos for improving the prediction of partially observed videos. Specifically, we develop a two-stage learning framework for action knowledge transfer. At the first stage, we learn feature embeddings and discriminative action classifier from full videos. The knowledge in the learned embeddings and classifier is then transferred to the partial videos at the second stage. Our experiments on the UCF-101 and HMDB-51 datasets show that the proposed action knowledge transfer method can significantly improve the performance of action prediction, especially for the actions with small observation ratios (e.g., 10%). We also experimentally illustrate that our method outperforms all the state-of-the-art action prediction systems.
SCNN: A General Distribution Based Statistical Convolutional Neural Network with Application to Video Object Detection

Various convolutional neural networks (CNNs) were developed recently that achieved accuracy comparable with that of human beings in computer vision tasks such as image recognition, object detection and tracking, etc. Most of these networks, however, process one single frame of image at a time, and may not fully utilize the temporal and contextual correlation typically present in multiple channels of the same image or adjacent frames from a video, thus limiting the achievable throughput. This limitation stems from the fact that existing CNNs operate on deterministic numbers. In this paper, we propose a novel statistical convolutional neural network (SCNN), which extends existing CNN architectures but operates directly on correlated distributions rather than deterministic numbers. By introducing a parameterized canonical model to model correlated data and defining corresponding operations as required for CNN training and inference, we show that SCNN can process multiple frames of correlated images effectively, hence achieving significant speedup over existing CNN models. We use a CNN based video object detection as an example to illustrate the usefulness of the proposed SCNN as a general network model. Experimental results show that even a nonoptimized implementation of SCNN can still achieve 178% speedup over existing CNNs with slight accuracy degradation.
Linear discriminant analysis guided by unsupervised ensemble learning

The high dimensionality and sparsity of data often increase the complexity of clustering; these factors occur simultaneously in unsupervised learning. Clustering and linear discriminant analysis (LDA) are methods to reduce the dimensionality and sparsity of data. In this study, the similarity of clustering and LDA are investigated based on their objective functions. Subsequently, their objective functions are integrated, and an LDA guided by an unsupervised ensemble learning (LDA–UEL) model is proposed. To create the proposed model, fuzziness F is designed to measure the confidence of unsupervised learning and the inference of the proposed model is illustrated. Furthermore, a corresponding algorithm for the inference is designed. Finally, extensive experiments are designed, and the results thus obtained demonstrate the effectiveness and high performance of the LDA–UEL model.
QoE-Oriented Rate Adaptation for DASH With Enhanced Deep Q-Learning

With the popularity of handheld devices, the development of wireless communication technology and the proliferation of multimedia resources, mobile video has become the main business in LTE networks with explosive traffic demands.How to improve the quality of experience (QoE) of mobile video in the dynamic and complex network environment has become a research focus.Dynamic adaptive streaming over HTTP technology introduces adaptive bitrate (ABR) requests at the client side to improve video QoE and various rate adaptation algorithms are also constantly proposed.In view of the limitations of the existing heuristic or learning-based ABR methods, we propose redirecting enhanced Deep Q-learning toward DASH video QoE (RDQ), a QoE-oriented rate adaptation framework based on enhanced deep Q-learning.First, we establish a chunkwise subjective QoE model and utilize it as the reward function in reinforcement learning so that the strategy can converge toward the direction of maximizing the subjective QoE score.Then, we apply several effective improvements of deep Q-learning to the RDQ agent's neural network architecture and learning mechanism to achieve faster convergence and higher average reward than other learning-based methods.The proposed RDQ agent has been thoroughly evaluated using trace-based simulation on the realtime LTE network data.For disparate network scenarios and different video contents, the RDQ agent can outperform the existing methods in terms of the QoE score.The breakdown analysis shows that RDQ can suppress the number and the duration of the stalling events to the minimum while maintaining high video bitrate, thus achieving better QoE performance than other methods.
Pool Strategies Selection in PoW-Based Blockchain Networks: Game-Theoretic Analysis

In proof-of-work-based (PoW-based) blockchain networks, the miners participate in a crypto-puzzle solving competition to win the reward by publishing a new block.Open mining pools attract a large number of miners for solving difficult problems together.Although the open strategy is likely to be more efficient, it makes pools susceptible to attack at the same time.In this paper, we present a gametheoretic analysis of mining pool strategy selection in order to explore the trade-off between the efficiency of openness and the vulnerability of attacks in a PoW-based blockchain network.We first model the pool mining process as a two-stage game, wherein the pools might decide whether to open or not and to attack or not.Based on the two-stage game model, we analyze the Nash equilibrium and the evolutionary stability of the mining games among pools, which uncovers the pool selection dynamics of PoW-based blockchain networks.In particular, we find that the attack behavior is the norm for a weak pool and triggers lower expected utilities when punishing the attacks more severely.Numerical simulations also support our theoretical findings as well as demonstrate the stability of the pools' strategy selection.
Same Fuzzy Logic Controller for Two-Wheeled Mobile Robot Navigation in Strange Environments

For any mobile device, the ability to navigate smoothly in its environment is of paramount importance, which justifies researchers’ continuous work on designing new techniques to reach this goal. In this work, we briefly present a description of a hard work on designing a Same Fuzzy Logic Controller (S.F.L.C.) of the two reactive behaviors of the mobile robot, namely, “go to goal obstacle avoidance” and “wall following,” in order to solve its navigation problems. This new technique allows an optimal motion planning in terms of path length and travelling time; it is meant to avoid collisions with convex and concave obstacles and to achieve the shortest path followed by the mobile robot. The efficiency of employing the proposed navigational controller is validated when compared to the results from other intelligent approaches; its qualities make of it an efficient alternative method for solving the path planning problem of the mobile robot.
Hardware-software co-design of slimmed optical neural networks

Optical neural network (ONN) is a neuromorphic computing hardware based on optical components. Since its first on-chip experimental demonstration, it has attracted more and more research interests due to the advantages of ultra-high speed inference with low power consumption. In this work, we design a novel slimmed architecture for realizing optical neural network considering both its software and hardware implementations. Different from the originally proposed ONN architecture based on singular value decomposition which results in two implementation-expensive unitary matrices, we show a more area-efficient architecture which uses a sparse tree network block, a single unitary block and a diagonal block for each neural network layer. In the experiments, we demonstrate that by leveraging the training engine, we are able to find a comparable accuracy to that of the previous architecture, which brings about the flexibility of using the slimmed implementation. The area cost in terms of the Mach-Zehnder interferometers, the core optical components of ONN, is 15%-38% less for various sizes of optical neural networks.
MAPNet: Multi-modal attentive pooling network for RGB-D indoor scene classification

RGB-D indoor scene classification is an essential and challenging task. Although convolutional neural network (CNN) achieves excellent results on RGB-D object recognition, it has several limitations when extended towards RGB-D indoor scene classification. 1) The semantic cues such as objects of the indoor scene have high spatial variabilities. The spatially rigid global representation from CNN is suboptimal. 2) The cluttered indoor scene has lots of redundant and noisy semantic cues; thus discerning discriminative information among them should not be ignored. 3) Directly concatenating or summing global RGB and Depth information as presented in popular methods cannot fully exploit the complementarity between two modalities for complicated indoor scenarios. To address the above problems, we propose a novel unified framework named Multi-modal Attentive Pooling Network (MAPNet) in this paper. Two orderless attentive pooling blocks are constructed in MAPNet to aggregate semantic cues within and between modalities meanwhile maintain the spatial invariance. The Intra-modality Attentive Pooling (IAP) block aims to mine and pool discriminative semantic cues in each modality. The Cross-modality Attentive Pooling (CAP) block is extended to learn different contributions across two modalities, which further guides the pooling of the selected discriminative semantic cues of each modality. We further show that the proposed model is interpretable, which helps to understand mechanisms of both scene classification and multi-modal fusion in MAPNet. Extensive experiments and analysis on SUN RGB-D Dataset and NYU Depth Dataset V2 show the superiority of MAPNet over current state-of-the-art methods.
Cross-modal hashing with semantic deep embedding

Cross-modal hashing has demonstrated advantages on fast retrieval tasks. It improves the quality of hash coding by exploiting semantic correlation across different modalities. In supervised cross-modal hashing, the learning of hash function replies on the quality of extracted features, for which deep learning models have been adopted to replace the traditional models based on handcraft features. All deep methods, however, have not sufficiently explored semantic correlation of modalities for the hashing process. In this paper, we introduce a novel end-to-end deep cross-modal hashing framework which integrates feature and hash-code learning into the same network. We take both between and within modalities data correlation into consideration, and propose a novel network structure and a loss function with dual semantic supervision for hash learning. This method ensures that the generated binary codes keep the semantic relationship of the original data points. Cross-modal retrieval experiments on commonly used benchmark datasets show that our method yields substantial performance improvement over several state-of-the-art hashing methods.
Multi-atlas segmentation and correction model with level set formulation for 3D brain MR images

We present an efficient multi-atlas segmentation and correction model with level set formulation for 3D brain MR images in this paper. We define a new energy functional by combining a weighted label fusion term, a bias field based image information fitting term and a regularization term together. More image information is taken into consideration in the new image data term to substantially improve the segmentation accuracy, especially when serious inhomogeneity and bias field exist in regions of interest in MR images. We introduce a spatially weight function and incorporate it into the label fusion term to increase the robustness of our segmentation algorithm to atlases with different registration accuracy. The new energy functional is in the form of L1 regularization problems, and we minimize it with the split Bregman method to ensure the segmentation efficiency. We apply the proposed model to segment six tissues in 3D brain MR images, including the amygdala, caudate, hippocampus, pallidum, putamen and thalamus. Experimental results have shown that our model can segment regions of interest accurately and eliminate bias field simultaneously. Quantitative comparisons with related methods have demonstrated the superiority of our model in terms of accuracy, efficiency and robustness.
A comparative study of sentiment analysis using SVM and SentiWordNet

&lt;p&gt;&lt;span&gt;Sentiment analysis has grown rapidly which impact on the number of services using the internet popping up in Indonesia. In this research, the sentiment analysis uses the rule-based method with the help of SentiWordNet and Support Vector Machine (SVM) algorithm with Term Frequency–Inverse Document Frequency (TF-IDF) as feature extraction method. Since the number of sentences in positive, negative and neutral classes is imbalanced, the oversampling method is implemented. For imbalanced dataset, the rule-based SentiWordNet and SVM algorithm achieve accuracies of 56% and 76%, respectively. However, for the balanced dataset, the rule-based SentiWordNet and SVM algorithm achieve accuracies of 52% and 89%, respectively.&lt;/span&gt;&lt;/p&gt;
Cost-Based Goal Recognition in Navigational Domains

&#x0D; &#x0D; &#x0D; Goal recognition is the problem of determining an agent's intent by observing her behaviour. Contemporary solutions for general task-planning relate the probability of a goal to the cost of reaching it. We adapt this approach to goal recognition in the strict context of path-planning. We show (1) that a simpler formula provides an identical result to current state-of-the-art in less than half the time under all but one set of conditions. Further, we prove (2) that the probability distribution based on this technique is independent of an agent's past behaviour and present a revised formula that achieves goal recognition by reference to the agent's starting point and current location only. Building on this, we demonstrate (3) that a Radius of Maximum Probability (i.e., the distance from a goal within which that goal is guaranteed to be the most probable) can be calculated from relative cost-distances between the candidate goals and a start location, without needing to calculate any actual probabilities. In this extended version of earlier work, we generalise our framework to the continuous domain and discuss our results, including the conditions under which our findings can be generalised back to goal recognition in general task-planning.&#x0D; &#x0D; &#x0D;