Reviewerly/exHarmony huggingface dataset:
Watch Hours in Minutes: Summarizing Videos with User Intent

With the ever increasing growth of videos, automatic video summarization has become an important task which has attracted lot of interest in the research community. One of the challenges which makes it a hard problem to solve is presence of multiple ‘correct answers’. Because of the highly subjective nature of the task, there can be different “ideal” summaries of a video. Modelling user intent in the form of queries has been posed in literature as a way to alleviate this problem. The query-focused summary is expected to contain shots which are relevant to the query in conjunction with other important shots. For practical deployments in which very long videos need to be summarized, this need to capture user’s intent becomes all the more pronounced. In this work, we propose a simple two stage method which takes user query and video as input and generates a query-focused summary. Specifically, in the first stage, we employ attention within a segment and across all segments, combined with the query to learn the feature representation of each shot. In the second stage, such learned features are again fused with the query to learn the score of each shot by regressing through fully connected layers. We then assemble the summary by arranging the top scoring shots in chronological order. Extensive experiments on a benchmark query-focused video summarization dataset for long videos give better results as compared to the current state of the art, thereby demonstrating the effectiveness of our method even without employing computationally expensive architectures like LSTMs, variational autoencoders, GANs or reinforcement learning, as done by most past works.
Probing the topological Anderson transition with quantum walks

We consider one-dimensional quantum walks in optical linear networks with synthetically introduced disorder and tunable system parameters allowing for the engineered realization of distinct topological phases. The option to directly monitor the walker's probability distribution makes this optical platform ideally suited for the experimental observation of the unique signatures of the one-dimensional topological Anderson transition. We analytically calculate the probability distribution describing the quantum critical walk in terms of a (time staggered) spin polarization signal and propose a concrete experimental protocol for its measurement. Numerical simulations back the realizability of our blueprint with current date experimental hardware.
Emojis predict dropouts of remote workers: An empirical study of emoji usage on GitHub

Emotions at work have long been identified as critical signals of work motivations, status, and attitudes, and as predictors of various work-related outcomes. When more and more employees work remotely, these emotional signals of workers become harder to observe through daily, face-to-face communications. The use of online platforms to communicate and collaborate at work provides an alternative channel to monitor the emotions of workers. This paper studies how emojis, as non-verbal cues in online communications, can be used for such purposes and how the emotional signals in emoji usage can be used to predict future behavior of workers. In particular, we present how the developers on GitHub use emojis in their work-related activities. We show that developers have diverse patterns of emoji usage, which can be related to their working status including activity levels, types of work, types of communications, time management, and other behavioral patterns. Developers who use emojis in their posts are significantly less likely to dropout from the online work platform. Surprisingly, solely using emoji usage as features, standard machine learning models can predict future dropouts of developers at a satisfactory accuracy. Features related to the general use and the emotions of emojis appear to be important factors, while they do not rule out paths through other purposes of emoji use.
Analysis iron distribution methods in fine sand- and silt-sized soil particles

Iron is prone to change its form and speciation in phases. Thus, several methods have been developed to estimate iron partitioning in the mineral phases of soils. However, the accuracy of these methods to evaluate the iron contribution from minor phases, such as actinolite, almandine, biotite, chlorite, epidote, hornblende, muscovite, and Fe-diospide, remains low. Furthermore, most of the current iron speciation research is focused on bulk samples, and only applies to soil samples that are mostly composed of clays or clay minerals, without a wide-ranging evaluation of soil particles with different grain sizes. In this study, we classified several iron phases using a mineral liberation analyzer on desert soil particles with diameters ranging from silt- to fine-sand (5–20 µm, 20–45 µm, 45–63 µm, 63–75 µm, and > 75 µm). The iron containing minor phases were identified, the modal mineral abundances were determined via matching with the standard energy dispersive spectra library, and a particle size analysis was performed using mineral processing tools on each of the examined 40,000 particles. The iron partition results were ultimately established based on the standard iron concentration in the mineral phases and the modal mineral abundances. This new method could be automated, thereby facilitating high efficiency identification of iron-containing phases that would allow, for the first time, the generation of a dataset for iron partitioning in soil particles. This method allows the identification of minor iron phases in soil particles, and permits in situ mapping of iron mineralogy in fine sand- to silt-sized soil particles. Not restricted by single mineral particles, this method considers multi-phase complex particles. Thus, it largely improves the accuracy for estimating the iron partition parameter.
A novel approach to the creation of a labelling lexicon for improving emotion analysis in text

Purpose This paper aims to describe the process used to create an emotion lexicon enriched with the emotional intensity of words and focuses on improving the emotion analysis process in texts. Design/methodology/approach The process includes setting, preparation and labelling stages. In the first stage, a lexicon is selected. It must include a translation to the target language and labelling according to Plutchik’s eight emotions. The second stage starts with the validation of the translations. Then, it is expanded with the synonyms of the emotion synsets of each word. In the labelling stage, the similarity of words is calculated and displayed using WordNet similarity. Findings The authors’ approach shows better performance to identification of the predominant emotion for the selected corpus. The most relevant is the improvement obtained in the results of the emotion analysis in a hybrid approach compared to the results obtained in a purist approach. Research limitations/implications The proposed lexicon can still be enriched by incorporating elements such as emojis, idioms and colloquial expressions. Practical implications This work is part of a research project that aids in solving problems in a digital society, such as detecting cyberbullying, abusive language and gender violence in texts or exercising parental control. Detection of depressive states in young people and children is added. Originality/value This semi-automatic process can be applied to any language to generate an emotion lexicon. This resource will be available in a software tool that implements a crowdsourcing strategy allowing the intensity to be re-labelled and new words to be automatically incorporated into the lexicon.
Eight-Channel Multispectral Image Database for Saliency Prediction

Saliency prediction is a very important and challenging task within the computer vision community. Many models exist that try to predict the salient regions on a scene from its RGB image values. Several new models are developed, and spectral imaging techniques may potentially overcome the limitations found when using RGB images. However, the experimental study of such models based on spectral images is difficult because of the lack of available data to work with. This article presents the first eight-channel multispectral image database of outdoor urban scenes together with their gaze data recorded using an eyetracker over several observers performing different visualization tasks. Besides, the information from this database is used to study whether the complexity of the images has an impact on the saliency maps retrieved from the observers. Results show that more complex images do not correlate with higher differences in the saliency maps obtained.
Understanding Over-parameterization in Generative Adversarial Networks

A broad class of {\it unsupervised} deep learning methods such as Generative Adversarial Networks (GANs) involve training of overparameterized models where the number of parameters of the model exceeds a certain threshold. Indeed, most successful GANs used in practice are trained using overparameterized generator and discriminator networks, both in terms of depth and width. A large body of work in {\it supervised} learning have shown the importance of model overparameterization in the convergence of the gradient descent (GD) to globally optimal solutions. In contrast, the unsupervised setting and GANs in particular involve non-convex concave mini-max optimization problems that are often trained using Gradient Descent/Ascent (GDA). The role and benefits of model overparameterization in the convergence of GDA to a global saddle point in non-convex concave problems is far less understood. In this work, we present a comprehensive analysis of the importance of model overparameterization in GANs both theoretically and empirically. We theoretically show that in an overparameterized GAN model with a 1-layer neural network generator and a linear discriminator, GDA converges to a global saddle point of the underlying non-convex concave min-max problem. To the best of our knowledge, this is the first result for global convergence of GDA in such settings. Our theory is based on a more general result that holds for a broader class of nonlinear generators and discriminators that obey certain assumptions (including deeper generators and random feature discriminators). Our theory utilizes and builds upon a novel connection with the convergence analysis of linear time-varying dynamical systems which may have broader implications for understanding the convergence behavior of GDA for non-convex concave problems involving overparameterized models. We also empirically study the role of model overparameterization in GANs using several large-scale experiments on CIFAR-10 and Celeb-A datasets. Our experiments show that overparameterization improves the quality of generated samples across various model architectures and datasets. Remarkably, we observe that overparameterization leads to faster and more stable convergence behavior of GDA across the board.
Self-incremental learning vector quantization with human cognitive biases

Human beings have adaptively rational cognitive biases for efficiently acquiring concepts from small-sized datasets. With such inductive biases, humans can generalize concepts by learning a small number of samples. By incorporating human cognitive biases into learning vector quantization (LVQ), a prototype-based online machine learning method, we developed self-incremental LVQ (SILVQ) methods that can be easily interpreted. We first describe a method to automatically adjust the learning rate that incorporates human cognitive biases. Second, SILVQ, which self-increases the prototypes based on the method for automatically adjusting the learning rate, is described. The performance levels of the proposed methods are evaluated in experiments employing four real and two artificial datasets. Compared with the original learning vector quantization algorithms, our methods not only effectively remove the need for parameter tuning, but also achieve higher accuracy from learning small numbers of instances. In the cases of larger numbers of instances, SILVQ can still achieve an accuracy that is equal to or better than those of existing representative LVQ algorithms. Furthermore, SILVQ can learn linearly inseparable conceptual structures with the required and sufficient number of prototypes without overfitting.
Facing Image Source Attribution on iPhone X

Most image forensics techniques rely on the analysis of traces left into the signal during the image acquisition process, which is supposed to be common among most devices. However, recent advances in visual technologies led several manufacturers to customize the acquisition pipeline in order to improve the image quality, by designing alternative coding schemes and in-camera processing. This fact threatens the effectiveness of available forensic techniques. It is thus required to study modern acquisition devices to both assess the effectiveness of available techniques and to develop new effective approaches. In this paper we focus on the source identification of images coming from one of the most spread moderm smartphones, i.e. the iPhone X. This model is significant since it comprises two main new features: the new HEIF compression standard, set as the default image format, and a brand new shooting mode, called Portrait Mode. We show that existing source identification methods are ineffective when images are acquired in Portrait Mode. We also show when and how it is possible to address this limitation by removing non unique artefacts introduced by the camera software.
Deformable MR-CT image registration using an unsupervised synthesis and registration network for neuro-endoscopic surgery

<strong>Purpose</strong>: Deep-brain stimulation via neuro-endoscopic surgery is a challenging procedure that requires accurate targeting of deep-brain structures that can undergo deformations (up to 10 mm). Conventional deformable registration methods have the potential to resolve such geometric error between preoperative MR and intraoperative CT but at the expense of long computation time. New advances in deep learning methods offer benefits to inter-modality image registration accuracy and runtime using novel similarity metrics and network architectures. <strong>Method:</strong> An unsupervised deformable registration network is reported that first generates a synthetic CT from MR using CycleGAN and then registers the synthetic CT to the intraoperative CT using an inverse-consistent registration network. Diffeomorphism of the registration is maintained using deformation exponentiation “squaring and scaling” layers. The method was trained and tested on a dataset of CT and T1-weighted MR images with randomly simulated deformations that mimic deep-brain deformation during surgery. The method was compared to a baseline method using inter-modality deep learning registration, VoxelMorph. <strong>Results</strong>: The methods were tested on 10 pairs of CT/MR images from 5 subjects. The proposed method achieved a Dice score of 0.84±0.04 for the lateral ventricles, 0.72±0.09 for the 3<sup>rd</sup> ventricle, and 0.63±0.10 for the 4<sup>th</sup> ventricle, with target registration error (TRE) of 0.95±0.54 mm. The proposed method showed statistically significant improvement in both Dice score and TRE in comparison to inter-modality VoxelMorph, while maintaining a fast runtime of less than 3 seconds for a typical MR-CT pair of volume images. <strong>Conclusion</strong>: The proposed unsupervised image synthesis and registration network demonstrates the capability for accurate volumetric deformable MR-CT registration with near real-time performance. The method will be further developed for application in intraoperative CT (or cone-beam CT) guided neurosurgery.
A critical comparison analysis between human and machine-generated tags for the Metropolitan Museum of Art's collection

Purpose Based on the highlights of The Metropolitan Museum of Art's collection, the purpose of this paper is to examine the similarities and differences between the subject keywords tags assigned by the museum and those produced by three computer vision systems. Design/methodology/approach This paper uses computer vision tools to generate the data and the Getty Research Institute's Art and Architecture Thesaurus (AAT) to compare the subject keyword tags. Findings This paper finds that there are clear opportunities to use computer vision technologies to automatically generate tags that expand the terms used by the museum. This brings a new perspective to the collection that is different from the traditional art historical one. However, the study also surfaces challenges about the accuracy and lack of context within the computer vision results. Practical implications This finding has important implications on how these machine-generated tags complement the current taxonomies and vocabularies inputted in the collection database. In consequence, the museum needs to consider the selection process for choosing which computer vision system to apply to their collection. Furthermore, they also need to think critically about the kind of tags they wish to use, such as colors, materials or objects. Originality/value The study results add to the rapidly evolving field of computer vision within the art information context and provide recommendations of aspects to consider before selecting and implementing these technologies.
Demosaicing by Differentiable Deep Restoration

A mosaic of color filter arrays (CFAs) is commonly used in digital cameras as a spectrally selective filter to capture color images. The captured raw image is then processed by a demosaicing algorithm to recover the full-color image. In this paper, we formulate demosaicing as a restoration problem and solve it by minimizing the difference between the input raw image and the sampled full-color result. This under-constrained minimization is then solved with a novel convolutional neural network that estimates a linear subspace for the result at local image patches. In this way, the result in an image patch is determined by a few combination coefficients of the subspace bases, which makes the minimization problem tractable. This approach further allows joint learning of the CFA and demosaicing network. We demonstrate the superior performance of the proposed method by comparing it with state-of-the-art methods in both settings of noise-free and noisy data.
A Neural Network MCMC Sampler That Maximizes Proposal Entropy

Markov Chain Monte Carlo (MCMC) methods sample from unnormalized probability distributions and offer guarantees of exact sampling. However, in the continuous case, unfavorable geometry of the target distribution can greatly limit the efficiency of MCMC methods. Augmenting samplers with neural networks can potentially improve their efficiency. Previous neural network-based samplers were trained with objectives that either did not explicitly encourage exploration, or contained a term that encouraged exploration but only for well structured distributions. Here we propose to maximize proposal entropy for adapting the proposal to distributions of any shape. To optimize proposal entropy directly, we devised a neural network MCMC sampler that has a flexible and tractable proposal distribution. Specifically, our network architecture utilizes the gradient of the target distribution for generating proposals. Our model achieved significantly higher efficiency than previous neural network MCMC techniques in a variety of sampling tasks, sometimes by more than an order magnitude. Further, the sampler was demonstrated through the training of a convergent energy-based model of natural images. The adaptive sampler achieved unbiased sampling with significantly higher proposal entropy than a Langevin dynamics sample. The trained sampler also achieved better sample quality.
NausicaApp: A Hybrid Decentralized Approach to Managing Covid-19 Pandemic at Campus Premises

Several contact-tracing app-based solutions have been proposed to alert app users who have previously encountered a Covid-19-positive user.In most countries, a statesponsored app has been made available to let citizens who install it on their phone monitor inter-personal contacts.The particular scenario considered in this work aims to support tracing within a university campus.For this, an alternative solution is proposed, based on a hybrid decentralized approach, that traces people who have been present in the same environment within some meaningful time interval.Thus, tracing goes beyond direct contact between persons.Our solution can also monitor crowd gathering on campus premises.The proposed Android app senses surrounding WiFi signals and uses them to obtain the user's absolute location.The app then securely sends an anonymized presence data object to the server.Thanks to data thus gathered, as soon as a user has reported herself as Covid-19-positive, all apps will be alerted by the server and receive anonymous data to determine whether their users happened to be in the same environment as the Covid-positive one.We believe our approach to be both effective, for it eschews weaknesses and limitations of Bluetooth-based solutions, and viable, for the experiments reported have proved WiFi sensingbased localization to be accurate enough.
Sentiment analysis of MOOC reviews via ALBERT-BiLSTM model

The accurate exploration of the sentiment information in comments for Massive Open Online Courses (MOOC) courses plays an important role in improving its curricular quality and promoting MOOC platform’s sustainable development. At present, most of the sentiment analyses of comments for MOOC courses are actually studies in the extensive sense, while relatively less attention is paid to such intensive issues as the polysemous word and the familiar word with an upgraded significance, which results in a low accuracy rate of the sentiment analysis model that is used to identify the genuine sentiment tendency of course comments. For this reason, this paper proposed an ALBERT-BiLSTM model for sentiment analysis of comments for MOOC courses. Firstly, ALBERT was used to dynamically generate word vectors. Secondly, the contextual feature vectors were obtained through BiLSTM pre-sequence and post-sequence, and the attention mechanism that could calculate the weight of different words in a sentence was applied together. Finally, the BiLSTM output vectors were input into Softmax for the classification of sentiments and prediction of the sentimental tendency. The experiment was performed based on the genuine data set of comments for MOOC courses. It was proved in the result that the proposed model was higher in accuracy rate than the already existing models.
Design and Evaluation of Attention Guidance Through Eye Gazing of “NAMIDA” Driving Agent

The driving agents considered thus far have aimed at navigating the driver’s attention while driving, for example, using interactions through linguistic conversations. Therefore, in this study, to investigate such a role in automatic driving from the perspective of nonverbal communication focusing on physicality (e.g., head movements and eye gaze), we constructed a driving agent called NAMIDA, along with its physical properties, as a research platform to investigate the role of nonverbal communication. We conducted a cognitive experiment on attention guidance, focusing on “gaze direction,” i.e., the movement of the eyes of NAMIDA. As a result, we confirmed that the attention of the participants is attracted by such eye-gaze movements of “NAMIDA,” which become a “cue” to exploring the surroundings.