Reviewerly/exHarmony huggingface dataset:
Virtual view synthesis for 3D light-field display based on feature reprojection and fusion

Three-dimensional light field display has achieved an impressive result and is a great potential display method. To provide real-world dense views efficiently, many virtual view synthesis approaches have been proposed. However, previous methods are not robust enough when the input view poses with a wider baseline or large rotation. Here, a virtual view synthesis method based on feature reprojection and fusion is presented, which can robustly generate high-quality dense virtual views for 3D light-field display. The target view depth map is generated with our proposed unsupervised network, which provides the 3D scene distribution priori. Then input views' feature maps are reprojected to the target view position. To fuse these reprojected feature maps effectively, a feature fusion Transformer network is designed, which encodes all input views' information and generates a fused feature map to represent the target view implicitly. The target view color map can be synthesized by rendering the fused feature map. Experimental results demonstrate the effectiveness of the proposed method, which shows outstanding performances when the input view poses are arranged unregularly. We believe the proposed method relieves the requirements of input sparse views acquisition and contributes to the wide applications of 3D light-field display in the future.
Preliminary exploration of topic modelling representations for Electronic Health Records coding according to the International Classification of Diseases in Spanish

In this work, we cope with the classification of Electronic Health Records (EHR) in Spanish according to the International Classification of Diseases (ICD). We employ Topic Models representing each document as a probabilistic distribution over topics, offering a low-dimensional representation of documents. The trend is to turn to an embedding text representation, but these approaches require large amounts of textual data. We found Topic Models as a suitable alternative approach to deal with the few resources available for Spanish clinical text mining. Besides, they are interpretable and aid the explainability in artificial intelligence (XAI). We explored two different methods, known as Latent Dirichlet Allocation (LDA) and Partially Labelled Latent Dirichlet Allocation (PLDA), the supervised approach of the former. We assessed the results attained in Spanish with an analogous task in English as a reference. Evaluation methods were applied directly to the representation, with metrics to determine topic coherence and the relationship between topics and ICD labels. We learned that PLDA was able to discover topics associated with the ICD. This finding means that this representation itself can reveal ICD codes previous to classification. Also, this representation was used as predictive features to feed a conventional classifier to show their competence in a downstream task. We conclude that in a context with a lack of big data availability, PLDA emerges as a versatile candidate, able to offer a competitive representation of EHRs. While other works are primarily concerned with supervised categorization and do not pay attention to the representation, LDA and PLDA offer an interpretable approach that can be associated with ICDs. Moreover, compared with those that employ LDA, we demonstrate how its’ supervised version, PLDA, can be more intuitive as it shows a closer relation with the ICDs.
Lightweight Secure Integer Comparison

We solve the millionaires problem in the semi-trusted model with homomorphic encryption without using intermediate decryptions. This leads to the computationally least expensive solution with homomorphic encryption so far, with a low bandwidth and very low storage complexity. The number of modular multiplications needed is less than the number of modular multiplications needed for one Pallier encryption. The output of the protocol can be either publicly known, encrypted, or secret-shared. The private input of the first player is computationally secure towards the second player, and the private input of the second player is even unconditionally secure towards the first player. We also introduce an efficient client-server solution for the millionaires problem with similar security properties.
ExSpliNet: An interpretable and expressive spline-based neural network

In this paper we present ExSpliNet, an interpretable and expressive neural network model. The model combines ideas of Kolmogorov neural networks, ensembles of probabilistic trees, and multivariate B-spline representations. We give a probabilistic interpretation of the model and show its universal approximation properties. We also discuss how it can be efficiently encoded by exploiting B-spline properties. Finally, we test the effectiveness of the proposed model on synthetic approximation problems and classical machine learning benchmark datasets.
KEADA: Identifying Key Classes in Software Systems Using Dynamic Analysis and Entropy-Based Metrics

Software maintenance is indispensable in the software development process. Developers need to spend a lot of time and energy to understand the software when maintaining the software, which increases the difficulty of software maintenance. It is a feasible method to understand the software through the key classes of the software. Identifying the key classes of the software can help developers understand the software more quickly. Existing techniques on key class identification mainly use static analysis techniques to extract software structure information. Such structure information may contain redundant relationships that may not exist when the software runs and ignores the actual interaction times between classes. In this paper, we propose an approach based on dynamic analysis and entropy-based metrics to identify key classes in the Java GUI software system, called KEADA (identifying KEy clAsses based on Dynamic Analysis and entropy-based metrics). First, KEADA extracts software structure information by recording the calling relationship between classes during the software running process; such structure information takes into account the actual interaction of classes. Second, KEADA represents the structure information as a weighted directed network and further calculates the importance of each node using an entropy-based metric OSE (One-order Structural Entropy). Third, KEADA ranks classes in descending order according to their OSE values and selects a small number of classes as the key class candidates. In order to verify the effectiveness of our approach, we conducted experiments on three Java GUI software systems and compared them with seven state-of-the-art approaches. We used the Friedman test to evaluate all approaches, and the results demonstrate that our approach performs best in all software systems.
A Novel Faster RCNN with ODN-Based Rain Removal Technique

During rainy times, the impact of outdoor vision systems gets considerably decreased owing to the visibility barrier, distortion, and blurring instigated by raindrops. So, it is essential to eradicate it from the rainy images for ensuring the reliability of outdoor vision system. To achieve this, several rain removal studies have been performed in recent days. In this view, this paper presents a new Faster Region Convolutional Neural Network (Faster RCNN) with Optimal Densely Connected Networks (DenseNet)-based rain removal technique called FRCNN-ODN. The presented involves weighted mean filtering (WMF) is applied as a denoising technique, which helps to boost the quality of the input image. In addition, Faster RCNN technique is used for rain detection that comprises region proposal network (RPN) and Fast RCNN model. The RPN generates high quality region proposals that are exploited by the Faster RCNN to detect rain drops. Also, the DenseNet model is utilized as a baseline network to generate the feature map. Moreover, sparrow search optimization algorithm (SSOA) is applied to choose the hyperparameters of the DenseNet model namely learning rate, batch size, momentum, and weight decay. An extensive experimental validation process is performed to highlight the effectual outcome of the FRCNN-ODN model and investigated the results with respect to several dimensions. The FRCNN-ODN method produced a higher UIQI of 0.981 for the applied image 1. Furthermore, on the applied image 2, the FRCNN-ODN model achieved a maximum UIQI of 0.982. Furthermore, the FRCNN-ODN algorithm produced a higher UIQI of 0.998 on the applied image 3. The simulation outcome showcased the superior outcome of the FRCNN-ODN (Optimal Densely Connected Networks) model with existing methods in terms of distinct measures.
A classification and review of tools for developing and interacting with machine learning systems

In this paper we aim to bring some order to the myriad of tools that have emerged in the field of Artificial Intelligence (AI), focusing on the field of Machine Learning (ML). For this purpose, we suggest a classification of the tools in which the categories are organized following the development lifecycle of an ML system and we make a review of the existing tools within each section of the classification. We believe this will help to better understand the ecosystem of tools currently available and will also allow us to identify niches in which to develop new tools to aid in the development of AI and ML systems. After reviewing the state-of-the-art of the tools, we have identified three trends in them: the incorporation of humans into the loop of the machine learning process, the movement from ad-hoc and experimental approaches to a more engineering perspective and the ability to make it easier to develop intelligent systems for people without an educational background in the area, in order to move the focus from the technical environment to the domain-specific problem.
Segment-Wise Time-Varying Dynamic Bayesian Network with Graph Regularization

Time-varying dynamic Bayesian network (TVDBN) is essential for describing time-evolving directed conditional dependence structures in complex multivariate systems. In this article, we construct a TVDBN model, together with a score-based method for its structure learning. The model adopts a vector autoregressive (VAR) model to describe inter-slice and intra-slice relations between variables. By allowing VAR parameters to change segment-wisely over time, the time-varying dynamics of the network structure can be described. Furthermore, considering some external information can provide additional similarity information of variables. Graph Laplacian is further imposed to regularize similar nodes to have similar network structures. The regularized maximum a posterior estimation in the Bayesian inference framework is used as a score function for TVDBN structure evaluation, and the alternating direction method of multipliers (ADMM) with L-BFGS-B algorithm is used for optimal structure learning. Thorough simulation studies and a real case study are carried out to verify our proposed method’s efficacy and efficiency.
An adaptable, high-performance relation extraction system for complex sentences

The rapid proliferation of text data has lead to an increase in the use of Information Extraction (IE) techniques to automatically extract key information in a fast and effective manner. Relation Extraction (RE), a sub-task of IE focuses on extracting semantic relations from free natural language text and is crucial for further applications including Question Answering, Information Retrieval, Knowledge Base construction, Text Summarization, etc. Literature shows that supervised learning approaches were widely used in RE. However, the performance of supervised methodologies depend on the availability of domain-specific annotated datasets which is not viable for many of the domains including legal, financial, insurance etc. In recent times, Open Information Extraction (OIE) techniques address this issue, by facilitating domain-independent extraction of relations from large text corpora with no demand for domain-specific tagged data and predefined relation classes. Even though OIE systems are fast and simple to implement, they are less effective in handling complex sentences, and often produce redundant extractions. This paper proposes an efficient RE system to extract domain-specific relations from natural language text, consisting of Knowledge-based and Semi-supervised learning systems, integrated with domain ontology. We evaluated the performance of proposed work on 'judicial domain" as a use case and found that it overcomes the flaws and limitations of existing RE approaches, by achieving better results in terms of precision and recall. On further analysis, we found that the proposed system outperforms existing cutting-edge OIE systems on varying sentence length and complexity.
Essay Scoring Model Based on Gated Recurrent Unit Technique

Educational evaluation is a major factor in determining students’ learning aptitude and academic performance. The scoring technique that relies solely on human labour is time consuming, costly, and logistically challenging as this rating is usually based on the opinion of “biased” human. Several studies have considered using machine learning techniques with feature extraction based on Term Frequency (TF) - Part of Speech (POS) Tagging without consideration to global vectorization (GloVe). These solutions require the process of selecting deterministic features that are directly related to essay quality which is time-consuming and needs a great deal of linguistic knowledge. Gated Recurrent Unit (a variation of Recurrent Neural Network) deep learning technique with focus on morphological analysis of essays for content-based assessment has therefore shown the capability of addressing the challenges posed by other AES techniques by building more abstract and complete linkages among features. Deep learning algorithms such as Multilayer Perceptron (MLP), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU) were used to learn the model with performance evaluation on metrics such as validation accuracy, training time, loss function, and Quadratic Weighted Kappa. The performance results showed that MLP, LSTM and GRU had average Quadratic Weighted Kappa (QWK) values of 0.65, 0.86 and 0.88 respectively with each algorithm having an average training time of 61.4, 62.68 and 67.86 seconds respectively. The loss functions for MLP, LSTM and GRU were 0.296, 0.24 and 0.126. This meant that GRU had the best estimate of the difference between the actual and forecasted scores. MLP, LSTM, and GRU had average validation accuracy of 0.48, 0.537, and 0.511 respectively. GRU was shown to be the optimal classifier and was used in the development of the essay scoring model.
Graph fusion network for multi-oriented object detection

In object detection, non-maximum suppression (NMS) methods are extensively adopted to remove horizontal duplicates of detected dense boxes for generating final object instances. However, due to the degraded quality of dense detection boxes and not explicit exploration of the context information, existing NMS methods via simple intersection-over-union (IoU) metrics tend to underperform on multi-oriented and long-size objects detection. Distinguishing with general NMS methods via duplicate removal, we propose a novel graph fusion network, named GFNet, for multi-oriented object detection. Our GFNet is extensible and adaptively fuse dense detection boxes to detect more accurate and holistic multi-oriented object instances. Specifically, we first adopt a locality-aware clustering algorithm to group dense detection boxes into different clusters. We will construct an instance sub-graph for the detection boxes belonging to one cluster. Then, we propose a graph-based fusion network via Graph Convolutional Network (GCN) to learn to reason and fuse the detection boxes for generating final instance boxes. Extensive experiments both on public available multi-oriented text datasets (including MSRA-TD500, ICDAR2015, ICDAR2017-MLT) and multi-oriented object datasets (DOTA) verify the effectiveness and robustness of our method against general NMS methods in multi-oriented object detection.
Learning on streaming graphs with experience replay

Graph Neural Networks (GNNs) have recently achieved good performance in many predictive tasks involving graph-structured data. However, the majority of existing models consider static graphs only and do not support training on graph streams. While inductive representation learning can generate predictions for unseen vertices, these are only accurate if the learned graph structure and properties remain stable over time. In this paper, we study the problem of employing experience replay to enable continuous graph representation learning in the streaming setting. We propose two online training methods, Random-Based Rehearsal-RBR, and Priority-Based Rehearsal-PBR, which avoid retraining from scratch when changes occur. Our algorithms are the first streaming GNN models capable of scaling to million-edge graphs with low training latency and without compromising accuracy. We evaluate the accuracy and training performance of these experience replay methods on the node classification problem using real-world streaming graphs of various sizes and domains. Our results demonstrate that PBR and RBR achieve orders of magnitude faster training as compared to offline methods while providing high accuracy and resiliency to concept drift.
GssMILP for anomaly classification in surveillance videos

Multiple Instance Learning (MIL) is widely used to locate abnormal video frames in surveillance videos due to its ability to work with weakly-labeled data. On the other hand, graph-based semi-supervised approaches are employed to handle label sparsity-related issues such as Low Degree-of-Supervision (LDoS) and High Class-Imbalance (HCI). However, the application of the MIL paradigm in anomaly classification of surveillance videos faces significant challenges such as (i) LDoS, (ii) HCI, and (iii) A multitude of types of abnormal events and actions instead of a unique pattern. The current work proposes a novel anomaly classifier to address these issues with a new objective function (GssMILP) by leveraging the benefits from graph-based semi-supervised and multiple-instance learning approaches. In this regard, our contributions are six-fold. First, a deep hierarchical architecture (C3D+3DCNN BICLSTM) to extract generic video descriptors. These descriptors represent the multitude of types of abnormal events and actions by incorporating the local–global spatial information and long-short-term temporal information related to the object motion, human actions, and behavioral features. Second, we develop a learning objective (GssMILP) that extends the MIL-based instance-level labeler to graph-based semi-supervising. Third, we use three different regularizer terms, namely graph-based regularizer, non-convex L2-regularizer, and temporal-smoothness regularizer. Fourth, we present an elaborate experimental study comparing the performance of the proposed approach with six similar baseline approaches. The results indicate that at degree-of-supervision 1.0%, the proposed method improves the f1-score by 45.26%, 18.19%, 45.28%, and 9.22% in UCSD-ped1, UCSD-ped2, MED, and RWAD datasets, respectively. Fifth, we present an ablation study on the impact of different features on the performance of an anomaly classifier. Sixth, we provide temporal annotations for a label resource of 1900 videos in the RWAD dataset.
Assessing Fuzzy Cognitive Mapping as a participatory and interdisciplinary approach to explore marine microfiber pollution

Fuzzy Cognitive Mapping (FCM) is a participatory modelling tool used to explore complex systems by facilitating interdisciplinary cooperation and integrating a variety of knowledge systems. Here FCM was used to explore marine microfiber pollution. Through individual interviews with representatives from the research, industry, water and environmental sectors, five stakeholder FCMs were developed and used to produce an aggregated community FCM in a stakeholder workshop. Stakeholder FCMs and the revised community FCM were used to compute how the modelled system reacted to changes under two scenarios developed during the stakeholder workshop; (i) Green Shift and (ii) increased textile consumption and production. Significant differences were observed in scenario results from the stakeholder-based models and the community-based model. For societal challenges characterized by unknowns around the problem and potential solutions, inclusion of a variety of knowledge systems through FCM and deliberation processes contribute to a more holistic picture of the system and its uncertainties.
Construction of a Six-Pronge Intelligent Physical Education Classroom Model in Colleges and Universities

A main form of information construction in colleges and universities is smart campus, which is a new model based on the improvement and perfection of digital campus construction. The development of smart campus is inseparable from the support of technologies such as Internet of Things (IoTs), cloud computing, and big data. By perceiving the teaching environment and intelligently identifying the characteristics of teachers and students, it can be used as a basis to improve the physical education, teaching environment, and digital teaching space, that is, an intelligent and diversified teaching environment for college teachers and students. In this study, we introduce an intelligent teaching mode into physical education classroom, construct the “six-pronge” intelligent classroom model, and apply the model by setting up a control and an experimental group to study the application effects of the proposed model in physical education teaching. The control group consists of the students who do not adopt the teaching mode, while students in the experimental group use the intelligent physical education teaching model. Through a comparative analysis, students’ physical education classroom teaching atmosphere, students’ learning interest and learning attitude, the proposed intelligent physical education classroom model are tested and verified. The analysis results show that after implementation of the “six-pronged” intelligent physical education classroom model, the physical education classroom atmosphere is active, and students have strong interest in learning and correct learning attitude. We believe that the model can help students complete pre-class preparation, in class teaching and after class practice supervision, and obtain a lot of knowledge about physical training. In addition, the model can help to cultivate students’ lifelong physical education learning and exercise habits, and greatly improve the quality of physical education classroom teaching in colleges.
Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video

Automated detection and validation of fine-grained human activities from egocentric vision has gained increased attention in recent years due to the rich information afforded by RGB images. However, it is not easy to discern how much rich information is necessary to detect the activity of interest reliably. Localization of hands and objects in the image has proven helpful to distinguishing between hand-related fine-grained activities. This paper describes the design of a hand-object-based mask obfuscation method (HOBM) and assesses its effect on automated recognition of fine-grained human activities. HOBM masks all pixels other than the hand and object in-hand, improving the protection of personal user information (PUI). We test a deep learning model trained with and without obfuscation using a public egocentric activity dataset with 86 class labels and achieve almost similar classification accuracies (2% decrease with obfuscation). Our findings show that it is possible to protect PUI at smaller image utility costs (loss of accuracy).