Miracle-dz/UniFAR-FTUs huggingface dataset:
Movie Recommender Systems: Implementation and Performance Evaluation

Over the years, explosive growth in the number of items in the catalog of e-commerce businesses, such as Amazon, Netflix, Pandora, etc., have warranted the development of recommender systems to guide consumers towards their desired products based on their preferences and tastes. Some of the popular approaches for building recommender systems, for mining user, derived input datasets, are: content-based systems, collaborative filtering, latent-factor systems using Singular Value Decomposition (SVD), and Restricted Boltzmann Machines (RBM). In this project, user-user collaborative filtering, item-item collaborative filtering, content-based recommendation, SVD, and neural networks were chosen for implementation in Python to predict the user ratings of unwatched movies for each user, and their performances were evaluated and compared.
Quasars as standard candles II

A tight non-linear relation between the X-ray and the optical-ultraviolet (UV) emission has been observed in active galactic nuclei (AGN) over a wide range of redshift and several orders of magnitude in luminosity, suggesting the existence of an ubiquitous physical mechanism regulating the energy transfer between the accretion disc and the X-ray emitting corona. Recently, our group developed a method to use this relation in observational cosmology, turning quasars into standardizable candles. This work mainly seeks to investigate the potential evolution of this correction at high redshifts. We thus studied the LX − LUV relation for a sample of quasars in the redshift range 4 < z < 7, adopting the selection criteria proposed in our previous work regarding their spectral properties. The resulting sample consists of 53 type 1 (unobscured) quasars, observed either with Chandra or XMM-Newton, for which we performed a full spectral analysis, determining the rest-frame 2 keV flux density, as well as more general X-ray properties such as the estimate of photon index, and the soft (0.5–2 keV) and hard (2–10 keV) unabsorbed luminosities. We find that the relation shows no evidence for evolution with redshift. The intrinsic dispersion of the LX–LUV for a sample free of systematics/contaminants is of the order of 0.22 dex, which is consistent with previous estimates from our group on quasars at lower redshift.
Shape-Graph Based Object Recognition Using Node Context Embedding

Graphical object representation is fequently used for visual object recognition and detection methods. Since most machine learning methods requira vectorial input, significant research has been done on assigning feature vectors to graphs - a process known as graph embedding. However, when one wishes to detect objects in a larger scene, it is a more viable strategy to assign feature vectors to graph nodes, and classify them individually. In this paper, we present a graph node embedding algorithm for 3D object detection based on primitive shape graphs. Our embedding algorithm encodes the local context of the selected node into the feature vector, thus improving the classification accuracy of nodes. The method also imposes no restriction on the structure of the graphs or the weights on the nodes and edges. The method presented here will be used as part of an intelligent object pairing algorithm for Tangible Augmented Reality.
A space-consistent version of the minimum-contrast estimator for linear stochastic evolution equations

A new modification of the minimum-contrast estimator (the weighted MCE) of drift parameter in a linear stochastic evolution equation with additive fractional noise is introduced in the setting of the spectral approach (Fourier coordinates of the solution are observed). The reweighing technique, which utilizes the self-similarity property, achieves strong consistency and asymptotic normality of the estimator as number of coordinates increases and time horizon is fixed (the space consistency). In this respect, this modification outperforms the standard (non-weighted) MCE. Compared to other drift estimators studied within spectral approach (e.g., maximum likelihood, trajectory fitting), the weighted MCE is rather universal. It covers discrete time as well as continuous-time observations and it is applicable to processes with any value of Hurst index [Formula: see text]. To the author’s best knowledge, this is so far the first space-consistent estimator studied for [Formula: see text].
Preference-Based Learning for Exoskeleton Gait Optimization

This paper presents a personalized gait optimization framework for lower-body exoskeletons. Rather than optimizing numerical objectives such as the mechanical cost of transport, our approach directly learns from user prefer-ences, e.g., for comfort. Building upon work in preference-based interactive learning, we present the CoSpar algorithm. CoSpar prompts the user to give pairwise preferences between trials and suggest improvements; as exoskeleton walking is a non-intuitive behavior, users can provide preferences more easily and reliably than numerical feedback. We show that CoSpar performs competitively in simulation and demonstrate a prototype implementation of CoSpar on a lower-body exoskeleton to optimize human walking trajectory features. In the experiments, CoSpar consistently found user-preferred parameters of the exoskeleton’s walking gait, which suggests that it is a promising starting point for adapting and personalizing exoskeletons (or other assistive devices) to individual users.
Dynamic Interference Management for UAV-Assisted Wireless Networks

We investigate a transmission mechanism aiming to improve the data rate between a base station (BS) and a user equipment (UE) through deploying multiple relaying UAVs. We consider the effect of interference incurred by another established communication network, which makes our problem challenging and different from the state of the art. We aim to design the 3D trajectories and power allocation for the UAVs to maximize the data flow of the network while keeping the interference on the existing communication network below a threshold. We utilize the mobility feature of the UAVs to evade the (un)-intended interference caused by (un)-intentional interferers. To this end, we propose an alternating-maximization approach to jointly obtain the 3D trajectories and the UAVs transmission powers. We handle the 3D trajectory design by resorting to spectral graph theory and subsequently address the power allocation through convex optimization techniques. We also approach the problem from the intentional interferer’s perspective where smart jammers chase the UAVs to effectively degrade the data flow of the network. We also extend our work to the case for multiple UEs. Finally, we demonstrate the efficacy of our proposed method through extensive simulations.
Visual Servoing-based Navigation for Monitoring Row-Crop Fields

Autonomous navigation is a pre-requisite for field robots to carry out precision agriculture tasks. Typically, a robot has to navigate along a crop field multiple times during a season for monitoring the plants, for applying agrochemicals, or for performing targeted interventions. In this paper, we propose a visual-based navigation framework tailored to row-crop fields that exploits the regular crop-row structure present in fields. Our approach uses only the images from on-board cameras without the need for performing explicit localization or maintaining a map of the field. Thus, it can operate without expensive RTK-GPS solutions often used in agricultural automation systems. Our navigation approach allows the robot to follow the crop rows accurately and handles the switch to the next row seamlessly within the same framework. We implemented our approach using C++ and ROS and thoroughly tested it in several simulated fields with different shapes and sizes. We also demonstrated the system running at frame-rate on an actual robot operating on a test row-crop field. The code and data have been published.
Tyranny to Anarchy: Regimes of Organisational Influence on Directed Hierarchical Graphs

Social organisational is critical to coordinated behaviour with a diverse range of management structures. In almost all organisations, a power structure exists with managers and subordinates. Often a change in one part can cause long-term cascades throughout the organisation, leading to inefficiencies and confusion. As organisations grow in size and complexity, as well as change the way they share information and power, we analyse how their resilience to disturbances is affected. Here, we consider majority rule dynamics on organisations modelled as hierarchical directed graphs, where the direction indicates task flow. We utilise a topological measure called the trophic incoherence parameter, $q$, which effectively gauges the stratification of power structure in an organisation. This is shown to bound regimes of behaviour. There is fast consensus at low $q$ (e.g. tyranny), slow consensus at mid $q$ (e.g. democracy), and no consensus at high $q$ (e.g. anarchy). These regimes are investigated analytically and empirically with diverse case studies in the Roman Army, US Government, and a healthcare organisation. Our work has widespread application in the design of organisations as well as analysing how some become inefficient and stagnate.
Fast Fixed Dimension L2-Subspace Embeddings of Arbitrary Accuracy, With Application to L1 and L2 Tasks

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2. $ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $\Pi^T A$ and inherit its runtime and accuracy, effectively decoupling the dimension $r$ from runtime and accuracy, allowing downstream machine learning applications to benefit from both a low dimension and high accuracy (in prior embeddings higher accuracy means higher dimension). We give applications of our L2-embedding to regression, PCA and statistical leverage scores. We also give applications to L1: 1. ) An oblivious L1-embedding with dimension $d+O(d\ln^{1+\eta} d)$ and distortion $O((d\ln d)/\ln\ln d)$, with application to constructing well-conditioned bases; 2. ) Fast approximation of L1-Lewis weights using our L2 embedding to quickly approximate L2-leverage scores.
Optimizing Nondecomposable Data Dependent Regularizers via Lagrangian Reparameterization offers Significant Performance and Efficiency Gains

Data dependent regularization is known to benefit a wide variety of problems in machine learning. Often, these regularizers cannot be easily decomposed into a sum over a finite number of terms, e.g., a sum over individual example-wise terms. The F β measure, Area under the ROC curve (AUCROC) and Precision at a fixed recall (P@R) are some prominent examples that are used in many applications. We find that for most medium to large sized datasets, scalability issues severely limit our ability in leveraging the benefits of such regularizers. Importantly, the key technical impediment despite some recent progress is that, such objectives remain difficult to optimize via backpropapagation procedures. While an efficient general-purpose strategy for this problem still remains elusive, in this paper, we show that for many data-dependent nondecomposable regularizers that are relevant in applications, sizable gains in efficiency are possible with minimal code-level changes; in other words, no specialized tools or numerical schemes are needed. Our procedure involves a reparameterization followed by a partial dualization - this leads to a formulation that has provably cheap projection operators. We present a detailed analysis of runtime and convergence properties of our algorithm. On the experimental side, we show that a direct use of our scheme significantly improves the state of the art IOU measures reported for MSCOCO Stuff segmentation dataset.
MDP-based Shallow Parsing in Distantly Supervised QA Systems

Question answering systems over knowledge graphs commonly consist of multiple components such as shallow parser, entity/relation linker, query generation and answer retrieval. We focus on the first task, shallow parsing, which so far received little attention in the QA community. Despite the lack of gold annotations for shallow parsing in question answering datasets, we devise a Reinforcement Learning based model called MDP-Parser, and show that it outperforms the current state-of-the-art approaches. Furthermore, it can be easily embedded into the existing entity/relation linking tools to boost the overall accuracy.
Investigations of Interacting Quantum Systems in a Novel General Approximation Scheme (NGAS)

A novel general approximation scheme (NGAS) is proposed, which is potentially applicable for arbitrary strength of interaction in quantum theory. The scheme utilizes an input Hamiltonian, which is exactly solvable. The effects of interaction are incorporated into the input Hamiltonian through a non-linear feedback mechanism by self-consistency requirements. The method is nonperturbative, yet improvable in a new formulation of perturbation theory, designated as: mean field perturbation theory (MFPT). We apply the scheme to anharmonic interactions in one dimension and obtain uniformly accurate results by Borel summation, for the quartic-, sextic-, octic-anharmonic oscillators and to the quartic double-well oscillator (QDWO) for arbitrary strength of coupling. The results for the QDWO may be contrasted with those from the standard formulation of perturbation theory , where Borel-summation fails even for small values of the coupling strength.
How robust is MovieLens? A dataset analysis for recommender systems

Research publication requires public datasets. In recommender systems, some datasets are largely used to compare algorithms against a --supposedly-- common benchmark. Problem: for various reasons, these datasets are heavily preprocessed, making the comparison of results across papers difficult. This paper makes explicit the variety of preprocessing and evaluation protocols to test the robustness of a dataset (or lack of flexibility). While robustness is good to compare results across papers, for flexible datasets we propose a method to select a preprocessing protocol and share results more transparently.
Probabilistic Word Embeddings in Neural IR: A Promising Model That Does Not Work as Expected (For Now)

In this paper, we discuss how a promising word vector representation based on Probabilistic Word Embeddings (PWE) can be applied to Neural Information Retrieval (NeuIR). We illustrate PWE pros for text retrieval, and identify the core issues which prevent a full exploitation of their potential. In particular, we focus on the application of elliptical probabilistic embeddings, a type of PWE, to a NeuIR system (i.e., MatchPyramid). The main contributions of this paper are: (i) an analysis of the pros and cons of PWE in NeuIR; (ii) an in-depth comparison of PWE against pre-trained Word2Vec, FastText and WordNet word embeddings; (iii) an extension of the MatchPyramid model to take advantage of broader word relations information from WordNet; (iv) a topic-level evaluation of the MatchPyramid ranking models employing the considered word embeddings. Finally, we discuss some lessons learned and outline some open research problems to employ PWE in NeuIR systems more effectively.
SADHAN: Hierarchical Attention Networks to Learn Latent Aspect Embeddings for Fake News Detection

Recently false claims and misinformation have become rampant in the web, affecting election outcomes, societies and economies. Consequently, fact checking websites such as snopes.com and politifact.com are becoming popular. However, these websites require expert analysis which is slow and not scalable. Many recent works try to solve these challenges using machine learning models trained on a variety of features and a rich lexicon or more recently, deep neural networks to avoid feature engineering. In this paper, we propose hierarchical deep attention networks to learn embeddings for various latent aspects of news. Contrary to existing solutions which only apply word-level self-attention, our model jointly learns the latent aspect embeddings for classifying false claims by applying hierarchical attention. Using several manually annotated high quality datasets such as Politifact, Snopes and Fever we show that these learned aspect embeddings are strong predictors of false claims. We show that latent aspect embeddings learned from attention mechanisms improve the accuracy of false claim detection by up to 13.5% in terms of Macro F1 compared to a state-of-the-art attention mechanism guided by claim-text DeClarE. We also extract and visualize the evidence from the external articles which supports or disproves the claims
JIGSAW: Structuring Text into Tables

We present JIGSAW, an end-to-end query driven system that efficiently generates structured tables from unstructured documents. To do so, first we describe how we can quickly retrieve sentences in support of structured queries that describe the table schema. Second, we describe how we can estimate table cell values using document context where such values can not be retrieved. Third, we describe how we can link together similar rows, rank, and diversify them to generate high-quality tables. We show that JIGSAW can generate tables from 25 million documents within seconds.
A Multi-Source Collection of Event-Labeled News Documents

In this paper, we present a collection of news documents labeled at the level of crisp events. Compared to other publicly-available collections, our dataset is made of heterogeneous documents published by popular news channels on different platforms in the same temporal window and, therefore, dealing with roughly the same events and topics. The collection spans 4 months and comprises 147K news documents from 27 news streams, i.e., 9 different channels and 3 platforms: Twitter, RSS portals, and news websites. We also provide relevance labels of news documents for some selected events. These relevance judgments were collected using crowdsourcing. The collection can be useful to researchers investigating challenging news-mining tasks, such as event detection and tracking, multi-stream analysis, and temporal analysis of news publishing patterns.
Hospital volume and mortality in liver resections for colorectal metastasis using population‐based administrative data

Approximately one‐quarter of patients with colorectal carcinoma develop colorectal liver metastases (CRLM). Surgical treatment with curative intent by hepatic resection is the standard medical care. While some studies with small sample sizes have investigated the relationship between hospital procedure volume and in‐hospital mortality for this diagnosis, no population‐based study has been conducted. The present study was aimed at closing this gap.
Learning from Observations Using a Single Video Demonstration and Human Feedback

In this paper, we present a method for learning from video demonstrations by using human feedback to construct a mapping between the standard representation of the agent and the visual representation of the demonstration. In this way, we leverage the advantages of both these representations, i.e., we learn the policy using standard state representations, but are able to specify the expected behavior using video demonstration. We train an autonomous agent using a single video demonstration and use human feedback (using numerical similarity rating) to map the standard representation to the visual representation with a neural network. We show the effectiveness of our method by teaching a hopper agent in the MuJoCo to perform a backflip using a single video demonstration generated in MuJoCo as well as from a real-world YouTube video of a person performing a backflip. Additionally, we show that our method can transfer to new tasks, such as hopping, with very little human feedback.
Re-learning of Child Model for Misclassified data by using KL Divergence in AffectNet: A Database for Facial Expression

AffectNet contains more than 1,000,000 facial images which manually annotated for the presence of eight discrete facial expressions and the intensity of valence and arousal. Adaptive structural learning method of DBN (Adaptive DBN) is positioned as a top Deep learning model of classification capability for some large image benchmark databases. The Convolutional Neural Network and Adaptive DBN were trained for AffectNet and classification capability was compared. Adaptive DBN showed higher classification ratio. However, the model was not able to classify some test cases correctly because human emotions contain many ambiguous features or patterns leading wrong answer which includes the possibility of being a factor of adversarial examples, due to two or more annotators answer different subjective judgment for an image. In order to distinguish such cases, this paper investigated a re-learning model of Adaptive DBN with two or more child models, where the original trained model can be seen as a parent model and then new child models are generated for some misclassified cases. In addition, an appropriate child model was generated according to difference between two models by using KL divergence. The generated child models showed better performance to classify two emotion categories: ‘Disgust’ and ‘Anger’.
Efficient scheme to eliminate spin contamination in magnetic dipolar coupling: Rectifying the zero-field splitting of NV$^-$ centers

An accurate description of the two-electron density, crucial for magnetic coupling in spin systems, provides in general a major challenge for density functional theory (DFT) calculations. It affects, e.g., the calculated zero-field splitting (ZFS) energies of spin qubits in semiconductors that frequently deviate significantly from experiment. In the present work (i) this discrepancy is demonstrated to arise from spin contamination, (ii) an efficient scheme to correct for spin contamination in both extended periodic and finite-size systems is devised, and (iii) a systematic study of the zero-field splitting of NV$^-$ centers in diamond and $3C$, $4H$, and $6H$ polytypes of silicon carbide is presented, showing excellent agreement between experiment and spin-contamination corrected theory.
Incremental processing of noisy user utterances in the spoken language understanding task

The state-of-the-art neural network architectures make it possible to create spoken language understanding systems with high quality and fast processing time. One major challenge for real-world applications is the high latency of these systems caused by triggered actions with high executions times. If an action can be separated into subactions, the reaction time of the systems can be improved through incremental processing of the user utterance and starting subactions while the utterance is still being uttered. In this work, we present a model-agnostic method to achieve high quality in processing incrementally produced partial utterances. Based on clean and noisy versions of the ATIS dataset, we show how to create datasets with our method to create low-latency natural language understanding components. We get improvements of up to 47.91 absolute percentage points in the metric F1-score.