Miracle-dz/UniFAR-FTUs huggingface dataset:
Deep multi-frame face hallucination for face identification

Face verification problem has been successfully approached in many recent papers. Unfortunately, recognition quality decreases significantly in the presence of different degradation factors inherent to real world data, for example surveillance videos, where faces can be very small in size, blurred and affected with compression artifacts. Here we present face hallucination approach based on using several sequential video frames: our neural network restores the central frame of each input sequence additionally taking into account a number of adjacent frames, warped to align with the central one. We show that our face hallucination approach can improve face identification quality compared to the single-frame baseline and to the multi-frame approach without adjacent frame warping.
Adaptive Affinity Field for Semantic Segmentation

Existing semantic segmentation methods mostly rely on per-pixel supervision, unable to capture structural regularity present in natural images. Instead of learning to enforce semantic labels on individual pixels, we propose to enforce affinity field patterns in individual pixel neighbourhoods, i.e., the semantic label patterns of whether neighbouring pixels are in the same segment should match between the prediction and the ground-truth. The affinity fields characterize geometric relationships within the image, such as "motorcycles have round wheels". We further develop a novel method for learning the optimal neighbourhood size for each semantic category, with an adversarial loss that optimizes over worst-case scenarios. Unlike the common Conditional Random Field (CRF) approaches, our adaptive affinity field (AAF) method has no extra parameters during inference, and is less sensitive to appearance changes in the image. Extensive evaluations on Cityscapes, PASCAL VOC 2012 and GTA5 datasets demonstrate the effectiveness and robustness of AAF in semantic segmentation.
Boundaries can Enhance Physical Layer Security at High Data Rates

In this paper, we study the receiver performance with physical layer security in a Poisson field of interferers. We compare the performance in two deployment scenarios: (i) the receiver is located at the corner of a quadrant, (ii) the receiver is located in the infinite plane. When the channel state information (CSI) of the eavesdropper is not available at the transmitter, we calculate the probability of secure connectivity using the Wyner coding scheme, and we show that hiding the receiver at the corner is beneficial at high rates of the transmitted codewords and detrimental at low transmission rates. When the CSI is available, we show that the average secrecy capacity is higher when the receiver is located at the corner, even if the intensity of interferers over there is four times higher than the intensity of interferers in the bulk. Therefore boundaries can also be used as a secrecy enhancement technique for high data rate applications.
Interpretable Machine Learning for Privacy-Preserving IoT and Pervasive Systems

The presence of pervasive computing in our everyday lives and emergence of the Internet of Things, such as the interaction of users with connected devices like smartphones or home appliances generate increasing amounts of traces that reflect users' behavior. A plethora of machine learning techniques enable service providers to process these traces to extract latent information about the users. While most of the existing projects have focused on the accuracy of these techniques, little work has been done on the interpretation of the inference and identification algorithms based on them. In this paper, we propose a machine learning interpretability framework for inference algorithms based on data collected through IoT and pervasive systems and we outline the open challenges in this research area. Our interpretability framework enable users to understand how the traces they generate could expose their privacy, while allowing for usable and personalized services at the same time.
Top-down Flow Transformer Networks

We study the deformation fields of feature maps across convolutional network layers under explicit top-down spatial transformations. We propose top-down flow transformer (TFT) by focusing on three transformations: translation, rotation, and scaling. We learn flow transformation generators that are able to account for the hidden layer deformations while maintaining the overall consistency across layers. The learned generators are shown to capture the underlying feature transformation processes that are independent of the particular training images. We observe favorable experimental results compared to the existing methods that tie transformations to fixed datasets. A comprehensive study on various datasets including MNIST, shapes, and natural images with both inner and inter datasets (trained on MNIST and validated in a number of datasets) evaluation demonstrates the advantages of our proposed TFT framework, which can be adopted in a variety of computer vision applications.
An answer to the Gamma question

We answer in this paper an open question (known as the "Gamma question"), related to the recent notion of coarse computability, which stems from complexity theory. The question was formulated by Andrews, Cai, Diamondstone, Jockusch and Lempp in "Asymptotic density, computable traceability and 1-randomness" [1]. The Gamma value of an oracle set measures to what extent each set computable with the oracle is approximable in the sense of density by a computable set. The closer to 1 this value is, the closer the oracle is to being computable. The Gamma question asks whether this value can be strictly in between 0 and 1/2. In this paper, we pursue some work initiated by Monin and Nies in "A unifying approach to the Gamma question" [19]. Using notions from computability theory, developed by Monin and Nies, together with some basic techniques from the field of error-correcting codes, we are able to give a negative answer to this question. The proof we give also provides an answer to a related question, asked by Denis Hirschfeldt in the expository paper "Some questions in computable mathematics" [12]. We also solve the Gamma problem for bases other than 2, answering another question of Monin and Nies.
Visual Speech Enhancement using Noise-Invariant Training

Visual speech enhancement is used on videos shot in noisy environments to enhance the voice of a visible speaker and to reduce background noise. While most existing methods use audio-only inputs, we propose an audio-visual neural network model for this purpose. The visible mouth movements are used to separate the speaker's voice from the background sounds. Instead of training our speech enhancement model on a wide range of possible noise types, we train the model on videos where other speech samples of the target speaker are used as background noise. A model trained using this paradigm generalizes well to various noise types, while also substantially reducing training time. The proposed model outperforms prior audio visual methods on two public lipreading datasets. It is also the first to be demonstrated on a general dataset not designed for lipreading. Our dataset was composed of weekly addresses of Barack Obama.
Algorithms in Bioinformatics: A Practical Introduction

This text focuses on using algorithms and discrete mathematics to solve biological problems. It systematically describes biological applications, the corresponding mathematical/computational problems, and various algorithmic solutions. The author also discusses the practical use of many algorithmic methods and describes what algorithms should be used in different situations. Each chapter contains an overview of the biological problem, a precise definition of the computational problem, a description of various methods, practical issues and further research, references to further reading, and a set of knowledge-testing exercises. Instruction and data for practical exercises are provided on the authors website.
Reducing Cascading Parsing Errors Through Fast Error Recovery

Syntax errors are generally easy to fix for humans, but not for parsers: the latter often fail to find an effective recovery, leading to a cascading chain of errors that drown out the original. More advanced recovery techniques suffer less from this problem but have seen little practical use because their typical performance was seen as poor and their worst case unbounded. In this paper we show that not only can an advanced approach run in acceptable time -- which we define as spending a maximum of 0.5s in error recovery per file -- but that we can find the complete set of minimum cost repair sequences within acceptable time. We then use the existence of the complete set of minimum cost repair sequences to further reduce the cascading error problem. We first extend Corchuelo et al.'s algorithm, before introducing a new, faster, alternative called MF. We validate our algorithms with a corpus of 200,000 real-world syntactically invalid Java programs: MF is able to repair 98.74% of files within acceptable time. By making use of the complete set of repair sequences, we show that the worst case for cascading errors is reduced by 32.27%.
An Interface between Lexical and Discourse Semantics. The case of the light verb "have"

The macro-target of discourse interpretation for computational purposes is the automatic detection of events in a text and their ordering in a temporal scale. Asher and Lascarides’ (2003) ideas on the semantics-pragmatics interface between the lexical and discourse level along with the logic mechanism for inferring rhetorical structure (often nonmonotonically) proposes an interesting way of achieving this macro-target. This talk examines the discourse behaviour of the light verb have in English, as in “John had his students walk out of class. ” and proposes an extension of Asher and Lascarides’ ideas, primarily with respect to the representation of lexical semantics and the interaction with discourse. Light have has previously been analyzed from within lexical semantics as a semantically light element which enters into a complex pred-ication with another predicate (e.g., Ritter and Rosen 1993). This talk argues for a different approach to light have by taking discourse structure into account. When one looks beyond the domain of lexical semantics, it becomes evident that light have supports the inference of discourse relations and functions as a reliable marker for discourse interpretation.
Implementation of Computer-Mediated Communication Utilizing Web Based Video Conferencing

This article reports on the use of videoconferencing as a Computer-Mediated Communication tool for pedagogical purposes in English as a Foreign Language （EFL） classes. Some of the advantages observed were heightened motivation, autonomy, and increased communication. It was found that the use of online videoconferencing was enlightening for teachers and students and could be made available for different types of classes and environments. It is suggested that web based videoconferencing has the potential to add a much needed dimension to EFL; the ability to practice speaking with native or fluent level speakers in a safe and controllable environment for a fraction of the cost.
Unsupervised Keyphrase Extraction Based on Outlier Detection

We propose a novel unsupervised keyphrase extraction approach based on outlier detection. Our approach starts by training word embeddings on the target document to capture semantic regularities among the words. It then uses the minimum covariance determinant estimator to model the distribution of non-keyphrase word vectors, under the assumption that these vectors come from the same distribution, indicative of their irrelevance to the semantics expresses by the dimensions of the learned vector representation. Candidate keyphrases are based on words that are outliers of this dominant distribution. Empirical results show that our approach outperforms state-of-the-art unsupervised keyphrase extraction methods.
Embedding Models for Episodic Memory

In recent years a number of large-scale triple-oriented knowledge graphs have been generated and various models have been proposed to perform learning in those graphs. Most knowledge graphs are static and reflect the world in its current state. In reality, of course, the state of the world is changing: a healthy person becomes diagnosed with a disease and a new president is inaugurated. In this paper, we extend models for static knowledge graphs to temporal knowledge graphs. This enables us to store episodic data and to generalize to new facts (inductive learning). We generalize leading learning models for static knowledge graphs (i.e., Tucker, RESCAL, HolE, ComplEx, DistMult) to temporal knowledge graphs. In particular, we introduce a new tensor model, ConT, with superior generalization performance. The performances of all proposed models are analyzed on two different datasets: the Global Database of Events, Language, and Tone (GDELT) and the database for Integrated Conflict Early Warning System (ICEWS). We argue that temporal knowledge graph embeddings might be models also for cognitive episodic memory (facts we remember and can recollect) and that a semantic memory (current facts we know) can be generated from episodic memory by a marginalization operation. We validate this episodic-to-semantic projection hypothesis with the ICEWS dataset.
Difficulty Controllable Question Generation for Reading Comprehension

We investigate the difficulty levels of questions, and propose a new setting called Difficulty-controllable Question Generation (DQG). Taking as input a reading comprehension paragraph and some text fragments (i.e. answers) in the paragraph that we want to ask questions about, a DQG method needs to generate questions each of which has a given text fragment as its answer, and meanwhile the generation is under the control of specified difficulty labels---the output questions should satisfy the specified difficulty as much as possible. To solve this task, we propose an end-to-end framework to generate questions of designated difficulty levels. Specifically, we explore a few intuitions: (i) In the input sentences, the nearer a word is to the answer fragment, the more likely it is used in the question; (ii) The easier a question is, the nearer its words are to the answer fragment in the sentence; (iii) Performing difficulty control could be regarded as a problem of sentence generation towards a specified attribute or style, namely difficulty level. For evaluation, we prepared the first dataset of reading comprehension questions with difficulty labels. The results show that our framework not only generates questions of better quality under the metrics like BLEU, but also has the capability to generate questions complying with the specified difficulty labels.
RGB Video Based Tennis Action Recognition Using a Deep Weighted Long Short-Term Memory

Action recognition has attracted increasing attention from RGB input in computer vision partially due to potential applications on somatic simulation and statistics of sport such as virtual tennis game and tennis techniques and tactics analysis by video. Recently, deep learning based methods have achieved promising performance for action recognition. In this paper, we propose weighted Long Short-Term Memory adopted with convolutional neural network representations for three dimensional tennis shots recognition. First, the local two-dimensional convolutional neural network spatial representations are extracted from each video frame individually using a pre-trained Inception network. Then, a weighted Long Short-Term Memory decoder is introduced to take the output state at time t and the historical embedding feature at time t-1 to generate feature vector using a score weighting scheme. Finally, we use the adopted CNN and weighted LSTM to map the original visual features into a vector space to generate the spatial-temporal semantical description of visual sequences and classify the action video content. Experiments on the benchmark demonstrate that our method using only simple raw RGB video can achieve better performance than the state-of-the-art baselines for tennis shot recognition.
Variable-bit-rate HDTV coding algorithm for ATM environments for B-ISDN

ABSTRACT A variable bit-rate HDTV coding algorithm based on motion-adaptive DCT is investigated for ATM environments in B-ISDN. Adaptive 2-layered coding, an ATM cell matrix for error correction, and a block interleave for error concealment are proposedto keep picture quality high by compensating for ATM cell loss. A new VLC and acongestion control scheme that restricts peak-rate and average-rate are also proposedfor traffic control. The proposed algorithm is shown to reduce the coding bit-rate for HDTV conference applications in ATM environments to 10-30 Mb/s. 1. INTRODUCflON HDTV services are expected to provide vivid and realistic image communication in B-ISDN. Recently, many simulation studies and prototype codecs have been reported toprovide HDTV secondary distribution services at bit-rates of 100-160 Mb/s. [1]-[4]However, some applications, for example HDTV conferences, require lower andvariable bit-rate coding algorithms to improve transmission cost and picture quality.Complex and highly efficient coding algorithms may also be able to reduce the bit-rate
Video Semantic Salient Instance Segmentation: Benchmark Dataset and Baseline

This paper pushes the envelope on salient regions in a video to decompose them into semantically meaningful components, semantic salient instances. To address this video semantic salient instance segmentation, we construct a new dataset, Semantic Salient Instance Video (SESIV) dataset. Our SESIV dataset consists of 84 high-quality video sequences with pixel-wisely per-frame ground-truth labels annotated for different segmentation tasks. We also provide a baseline for this problem, called Fork-Join Strategy (FJS). FJS is a two-stream network leveraging advantages of two different segmentation tasks, i.e., semantic instance segmentation and salient object segmentation. In FJS, we introduce a sequential fusion that combines the outputs of the two streams to have non-overlapping instances one by one. We also introduce a recurrent instance propagation to refine the shapes and semantic meanings of instances, and an identity tracking to maintain both the identity and the semantic meaning of an instance over the entire video. Experimental results demonstrated the effectiveness of our proposed FJS.
CarbonKit: a technological platform for personal carbon tracking

Ubiquitous technology platforms have been created to track and improve health and fitness; similar technologies can help individuals monitor and reduce their carbon footprints. This paper proposes CarbonKit, a platform combining technology, markets, and incentives to empower and reward people for reducing their carbon footprint. We argue that a goal-and-reward behavioral feedback loop can be combined with the Big Data available from tracked activities, apps, and social media to make CarbonKit an integral part of individuals daily lives. CarbonKit comprises five modules that link personal carbon tracking, health and fitness, social media, and economic incentives. Protocols for safeguarding security, privacy and individuals control over their own data are essential to the design of the CarbonKit. Initially CarbonKit would operate on a voluntary basis, but such a system can also serve as part of a mandatory region-wide initiative. We use the example of the British Columbia to illustrate the regulatory framework and participating stakeholders that would be required to support the CarbonKit in specific jurisdictions.
Fast and Differentially Private Algorithms for Decentralized Collaborative Machine Learning

Consider a set of agents in a peer-to-peer communication network, where each agent has a personal dataset and a personal learning objective. The main question addressed in this paper is: how can agents collaborate to improve upon their locally learned model without leaking sensitive information about their data? Our first contribution is to reformulate this problem so that it can be solved by a block coordinate descent algorithm. We obtain an efficient and fully decentralized protocol working in an asynchronous fashion. Our second contribution is to make our algorithm differentially private to protect against the disclosure of any information about personal datasets. We prove convergence rates and exhibit the trade-off between utility and privacy. Our experiments show that our approach dramatically outperforms previous work in the non-private case, and that under privacy constraints we significantly improve over purely local models.
Blind Signal Classification for Non-Orthogonal Multiple Access in Internet-of-Things Networks

For downlink multiple-user (MU) transmission based on non-orthogonal multiple access (NOMA), the advanced receiver strategy is required to cancel the inter-user interference, e.g., successive interference cancellation (SIC). The SIC process can be applicable only when information about the co-scheduled signal is known at the user terminal (UT) side. In particular, the UT should know whether the received signal is OMA or NOMA, whether SIC is required or not, and which modulation orders and power ratios have been used for the superposed UTs, before decoding the signal. An efficient network, e.g., Internet-of-Things (IoT), requires that UT blindly classify the received signal and apply a matching receiver strategy to reduce the high-layer signaling overhead and to increase resource efficiency. In this paper, we first analyze the performance impact of errors in NOMA signal classification and address ensuing receiver challenges in practical IoT-based MU usage cases. In order to reduce the blind signal classification error, we propose transmission schemes that rotate data symbols or pilots to a specific phase according to the transmitted signal format. In the case of pilot rotation, a new signal classification algorithm is also proposed. The performance improvements by the proposed methods are verified by intensive simulation results.
Efficient Fusion of Sparse and Complementary Convolutions for Object Recognition and Detection

We propose a new method to create compact convolutional neural networks (CNNs) by exploiting sparse convolutions. Different from previous works that learn sparsity in models, we directly employ hand-crafted kernels with regular sparse patterns, which result in the computational gain in practice without sophisticated and dedicated software or hardware. The core of our approach is an efficient network module that linearly combines sparse kernels to yield feature representations as strong as those from regular kernels. We integrate this module into various network architectures and demonstrate its effectiveness on three vision tasks, object classification, localization and detection. For object classification and localization, our approach achieves comparable or better performance than several baselines and related works while providing lower computational costs with fewer parameters (on average, a $2-4\times$ reduction of convolutional parameters and computation). For object detection, our approach leads to a VGG-16-based Faster RCNN detector that is 12.4$\times$ smaller and about 3$\times$ faster than the baseline.