Reviewerly/exHarmony huggingface dataset:
Direct Speech-to-Image Translation

Direct speech-to-image translation without text is an interesting and useful topic due to the potential applications in human-computer interaction, art creation, computer-aided design. etc. Not to mention that many languages have no writing form. However, as far as we know, it has not been well-studied how to translate the speech signals into images directly and how well they can be translated. In this paper, we attempt to translate the speech signals into the image signals without the transcription stage. Specifically, a speech encoder is designed to represent the input speech signals as an embedding feature, and it is trained with a pretrained image encoder using teacher-student learning to obtain better generalization ability on new classes. Subsequently, a stacked generative adversarial network is used to synthesize high-quality images conditioned on the embedding feature. Experimental results on both synthesized and real data show that our proposed method is effective to translate the raw speech signals into images without the middle text representation. Ablation study gives more insights about our method.
A Fast Single-Image Dehazing Algorithm Based on Dark Channel Prior and Rayleigh Scattering

The scattering of atmospheric particles significantly alters images captured under hazy weather condition.Images appear distorted, blurry and low in contrast attenuation, which extensively affects computer vision systems.There has been development of several prior based methods to address this problem.However, these methods come at a high computational cost.We present a fast, single image dehazing method based on dark channel prior and Rayleigh scattering.Firstly, we present a simple but effective methodology for estimating the atmospheric light through the computation of average, minimum and maximum of the pixels in each of the three RGB colour channels.Then, using the theory of Rayleigh scattering, we model a scattering coefficient to estimate the initial transmission map.Also, a fast-guided filter is adopted to refine the initial transmission map due to inaccurate halo edges.Finally, we restore the haze-free image through the atmospheric scattering model.Extensive qualitative and computational experiments on hazy outdoor images demonstrate that the proposed method produces excellent results whiles achieving a faster processing time.
Argus: Efficient Activity Detection System for Extended Video Analysis

We propose an Efficient Activity Detection System, Argus, for Extended Video Analysis in the surveillance scenario. For the spatial-temporal event detection in the surveillance video, we first generate video proposals by applying object detection and tracking algorithm which shared the detection features. After that, we extract several different features and apply sequential activity classification with them. Finally, we eliminate inaccurate events and fuse all the predictions from different features. The proposed system wins Trecvid Activities in Extended Video (ActEV <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ) challenge 2019. It achieves the first place with 60.5 mean weighted P <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">miss</sub> , outperforming the second place system by 14.5 and the baseline R-C3D by 29.0. In TRECVID 2019 Challenge <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , the proposed system wins the first place with pAUDC@0.2tfa 0.48407.
Modeling interactive components by coordinate kernel polynomial models

We proposed the use of coordinate kernel polynomials in kernel regression. This new approach, called coordinate kernel polynomial regression, can simultaneously identify active variables and effective interactive components. Reparametrization refinement is found critical to improve the modeling accuracy and prediction power. The post-training component selection allows one to identify effective interactive components. Generalization error bounds are used to explain the effectiveness of the algorithm from a learning theory perspective and simulation studies are used to show its empirical effectiveness.
DeepWeave: Accelerating Job Completion Time with Deep Reinforcement Learning-based Coflow Scheduling

To improve the processing efficiency of jobs in distributed computing, the concept of coflow is proposed. A coflow is a collection of flows that are semantically correlated in a multi-stage computation task. A job consists of multiple coflows and can be usually formulated as a Directed-Acyclic Graph (DAG). A proper scheduling of coflows can significantly reduce the completion time of jobs in distributed computing. However, this scheduling problem is proved to be NP-hard. Different from existing schemes that use hand-crafted heuristic algorithms to solve this problem, in this paper, we propose a Deep Reinforcement Learning (DRL) framework named DeepWeave to generate coflow scheduling policies. To improve the inter-coflow scheduling ability in the job DAG, DeepWeave employs a Graph Neural Network (GNN) to process the DAG information. DeepWeave learns from the history workload trace to train the neural networks of the DRL agent and encodes the scheduling policy in the neural networks, which make coflow scheduling decisions without expert knowledge or a pre-assumed model. The proposed scheme is evaluated with a simulator using real-life traces. Simulation results show that DeepWeave completes jobs at least 1.7X faster than the state-of-the-art solutions.
An efficient cryptographic technique using modified Diffie–Hellman in wireless sensor networks

In wireless sensor networks, the sensors transfer data through radio signals to a remote base station. Sensor nodes are used to sense environmental conditions such as temperature, strain, humidity, sound, vibration, and position. Data security is a major issue in wireless sensor networks since data travel over the naturally exposed wireless channel where malicious attackers may get access to critical information. The sensors in wireless sensor networks are resource-constrained devices whereas the existing data security approaches have complex security mechanisms with high computational and response times affecting the network lifetime. Furthermore, existing systems, such as secure efficient encryption algorithm, use the Diffie–Hellman approach for key generation and exchange; however, Diffie–Hellman is highly vulnerable to the man-in-the-middle attack. This article introduces a data security approach with less computational and response times based on a modified version of Diffie–Hellman. The Diffie–Hellman has been modified to secure it against attacks by generating a hash of each value that is transmitted over the network. The proposed approach has been analyzed for security against various attacks. Furthermore, it has also been analyzed in terms of encryption/decryption time, computation time, and key generation time for different sizes of data. The comparative analysis with the existing approaches shows that the proposed approach performs better in most of the cases.
GP-VAE: Deep Probabilistic Time Series Imputation

Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years. This raises the question whether deep learning methodologies can outperform classical data imputation methods in this domain. However, naive applications of deep learning fall short in giving reliable confidence estimates and lack interpretability. We propose a new deep sequential latent variable model for dimensionality reduction and data imputation. Our modeling assumption is simple and interpretable: the high dimensional time series has a lower-dimensional representation which evolves smoothly in time according to a Gaussian process. The non-linear dimensionality reduction in the presence of missing data is achieved using a VAE approach with a novel structured variational approximation. We demonstrate that our approach outperforms several classical and deep learning-based data imputation methods on high-dimensional data from the domains of computer vision and healthcare, while additionally improving the smoothness of the imputations and providing interpretable uncertainty estimates.
Transformative agency by double stimulation: advances in theory and methodology

Theories of agency are often borrowed from psychology and sociology; truly educational theories of agency are rare – theories that not only classify and measure dimensions of agency but enable us to understand how its formation can be pedagogically facilitated. Based on Sannino's recent work on transformative agency by double stimulation (TADS), a truly educational approach to agency is gaining traction. In double stimulation, a person or group faces a paralysing conflict of motives (first stimulus) which is resolved by identifying a meaningful artefact that is turned into a sign (second stimulus). Today's critical learning challenges typically do not have obvious or 'correct' solutions. They require a pedagogy that allows learners to face conflicts and construct artefacts to help break out of their paralysis. The paradigm of double stimulation, as explored in the articles in this issue, offers as tarting point for a pedagogy of agentive actions and expanding possibilities.
Evaluation of 1-Year in-Home Monitoring Technology by Home-Dwelling Older Adults, Family Caregivers, and Nurses

Introduction: Population ageing is increasing the healthcare needs and costs. Both frailty and chronic diseases affecting older people reduce their ability to live independently. However, most older people prefer to age in their own homes. New development of in-home monitoring can play a role in staying independent, active, and healthy for older people. This 12-month observational study aimed to evaluate a new in-home monitoring system among home-dwelling older adults (OA), their family caregivers (FC), and nurses for the support of home care. Methods: In-home monitoring system evaluated in this study continuously monitored OA’s daily activities (e.g. mobility, sleep habits, fridge visits, door events) by ambient sensor system (DomoCare®) and health-related events by wearable sensors (Activity tracker, ECG). In case of deviations in daily activities, alerts were transmitted to nurses via email. Using specific questionnaires, the opinion of 13 OA, 13 FC, and 20 nurses were collected at the end of 12-months follow-up focusing on user experience and the impact of in-home monitoring on home care services. Results: The majority of OA, FC, and nurses considered that in-home sensors can help staying at home, improving home care and quality of life, preventing domestic accidents, and reducing family stress. The opinion tended to be more frequently favorable toward ambient sensors (76%; 95% CI: 61% - 87%) than toward wearable sensors (Activity tracker: 65%; 95% CI: 50% - 79%); ECG: 60%; 95% CI: 45% - 75%). On average, OA (74%; 95% CI: 46% - 95%) and FC (70%; 95% CI: 39% - 91%) tended to be more enthusiastic than nurses (60%; 95% CI: 36% - 81%). Some barriers reported by nurses were a fear of weakening of the relationship with OA and lack of time. Discussion/Conclusion: Overall, the opinion of OA, FC and nurses were positive related to in-home sensors, with nurses being less enthusiastic about their use in clinical practice.
Blockchain Platforms and Access Control Classification for IoT Systems

The Internet of Things paradigm is growing rapidly. In fact, controlling this massive growth of IoT globally raises new security and privacy issues. The traditional access control mechanisms provide security to IoT systems such as DAC (discretionary access control) and mandatory access control (MAC). However, these mechanisms are based on central authority management, which raises some issues such as absence of scalability, single point of failure, and lack of privacy. Recently, the decentralized and immutable nature of blockchain technology integrated with access control can help to overcome privacy and security issues in the IoT. This paper presents a review of different access control mechanisms in IoT systems. We present a comparison table of reviewed access control mechanisms. The mechanisms’ scalability, distribution, security, user-centric, privacy and policy enforcing are compared. In addition, we provide access control classifications. Finally, we highlight challenges and future research directions in developing decentralized access control mechanisms for IoT systems.
An unsupervised method for extractive multi-document summarization based on centroid approach and sentence embeddings

Extractive multi-document summarization (MDS) is the process of automatically summarizing a collection of documents by ranking sentences according to their importance and informativeness. Text representation is a fundamental process that affects the effectiveness of many text summarization methods. Word embedding representations have shown to be effective for several Natural Language Processing (NLP) tasks including Automatic Text Summarization (ATS). However, most of these representations do not consider the order and the semantic relationships between words in a sentence. This does not fully allow grasping the sentence semantics and the syntactic relationships between sentences constituents. In this paper, to overcome this problem, we propose an unsupervised method for generic extractive multi-document summarization based on the sentence embedding representations and the centroid approach. The proposed method selects relevant sentences according to the final score obtained by combining three scores: sentence content relevance, sentence novelty, and sentence position scores. The sentence content relevance score is computed as the cosine similarity between the centroid embedding vector of the cluster of documents and the sentence embedding vectors. The sentence novelty metric is explicitly adopted to deal with redundancy. The sentence position metric assumes that the first sentences of a document are more relevant to the summary, and it assigns high scores to these sentences. Moreover, this paper provides a comparative analysis of nine sentence embedding models used to represent sentences as dense vectors in a low dimensional vector space in the context of extractive multi-document summarization. Experiments are performed on the standard DUC’2002–2004 benchmark datasets and the Multi-News dataset. The overall obtained results have shown that our method outperforms several state-of-the-art methods and achieves promising results compared to the best performing methods including supervised deep learning based methods.
Coarse-to-Fine Pre-training for Named Entity Recognition

More recently, Named Entity Recognition has achieved great advances aided by pre-training approaches such as BERT.However, current pre-training techniques focus on building language modeling objectives to learn a general representation, ignoring the named entityrelated knowledge.To this end, we propose a NER-specific pre-training framework to inject coarse-to-fine automatically mined entity knowledge into pre-trained models.Specifically, we first warm-up the model via an entity span identification task by training it with Wikipedia anchors, which can be deemed as general-typed entities.Then we leverage the gazetteer-based distant supervision strategy to train the model extract coarse-grained typed entities.Finally, we devise a self-supervised auxiliary task to mine the fine-grained named entity knowledge via clustering.Empirical studies on three public NER datasets demonstrate that our framework achieves significant improvements against several pre-trained baselines, establishing the new state-of-the-art performance on three benchmarks.Besides, we show that our framework gains promising results without using human-labeled training data, demonstrating its effectiveness in labelfew and low-resource scenarios.
Hierarchical Neural Architecture Search for Deep Stereo Matching

To reduce the human efforts in neural network design, Neural Architecture Search (NAS) has been applied with remarkable success to various high-level vision tasks such as classification and semantic segmentation. The underlying idea for the NAS algorithm is straightforward, namely, to enable the network the ability to choose among a set of operations (e.g., convolution with different filter sizes), one is able to find an optimal architecture that is better adapted to the problem at hand. However, so far the success of NAS has not been enjoyed by low-level geometric vision tasks such as stereo matching. This is partly due to the fact that state-of-the-art deep stereo matching networks, designed by humans, are already sheer in size. Directly applying the NAS to such massive structures is computationally prohibitive based on the currently available mainstream computing resources. In this paper, we propose the first end-to-end hierarchical NAS framework for deep stereo matching by incorporating task-specific human knowledge into the neural architecture search framework. Specifically, following the gold standard pipeline for deep stereo matching (i.e., feature extraction -- feature volume construction and dense matching), we optimize the architectures of the entire pipeline jointly. Extensive experiments show that our searched network outperforms all state-of-the-art deep stereo matching architectures and is ranked at the top 1 accuracy on KITTI stereo 2012, 2015 and Middlebury benchmarks, as well as the top 1 on SceneFlow dataset with a substantial improvement on the size of the network and the speed of inference. The code is available at this https URL.
Logic2Text: High-Fidelity Natural Language Generation from Logical Forms

Previous studies on Natural Language Generation (NLG) from structured data have primarily focused on surface-level descriptions of record sequences.However, for complex structured data, e.g., multi-row tables, it is often desirable for an NLG system to describe interesting facts from logical inferences across records.If only provided with the table, it is hard for existing models to produce controllable and high-fidelity logical generations.In this work, we formulate highfidelity NLG as generation from logical forms in order to obtain controllable and faithful generations.We present a new large-scale dataset, LOGIC2TEXT, with 10,753 descriptions involving common logic types paired with the underlying logical forms.The logical forms show diversified graph structure of free schema, which pose great challenges on the model's ability to understand the semantics.We experiment on (1) Fullysupervised training with the full datasets, and (2) Few-shot setting, provided with hundreds of paired examples; We compare several popular generation models and analyze their performances.We hope our dataset can encourage research towards building an advanced NLG system capable of natural, faithful, and human-like generation.The dataset and code is available at https://github.com/czyssrs/Logic2Text.
Convolutional neural networks for enhanced classification mechanisms of metamodels

Conventional wisdom on Model-Driven Engineering suggests that metamodels are crucial elements for modeling environments consisting of graphical editors, transformations, code generators, and analysis tools. Software repositories are commonly used in practice for locating existing artifacts provided that a classification procedure is available. However, the manual classification of metamodel in repositories produces results that are influenced by the subjectivity of human perception besides being tedious and prone to errors. Therefore, automated techniques for classifying metamodels stored in repositories are highly desirable and stringent. In this work, we propose memoCNN as a novel approach to classification of metamodels. In particular, we consider metamodels as data points and classify them using supervised learning techniques. A convolutional neural network has been built to learn from labeled data, and use the trained weights to group unlabeled metamodels. A comprehensive experimental evaluation proves that the proposal effectively categorizes input data and outperforms a state-of-the-art baseline.
Refined Belief Propagation Decoding of Sparse-Graph Quantum Codes

Quantum stabilizer codes constructed from sparse matrices have good performance and can be efficiently decoded by belief propagation (BP). A conventional BP decoding algorithm treats binary stabilizer codes as additive codes over GF(4). This algorithm has a relatively complex process of handling check-node messages, which incurs higher decoding complexity. Moreover, BP decoding of a stabilizer code usually suffers a performance loss due to the many short cycles in the underlying Tanner graph. In this paper, we propose a refined BP decoding algorithm for quantum codes with complexity roughly the same as binary BP. For a given error syndrome, this algorithm decodes to the same output as the conventional quaternary BP but the passed node-to-node messages are single-valued, unlike the quaternary BP, where multivalued node-to-node messages are required. Furthermore, the techniques of message strength normalization can naturally be applied to these single-valued messages to improve the performance. Another observation is that the message-update schedule affects the performance of BP decoding against short cycles. We show that running BP with message strength normalization according to a serial schedule (or other schedules) may significantly improve the decoding performance and error floor in computer simulation.
Differential Privacy Protection Against Membership Inference Attack on Machine Learning for Genomic Data

Machine learning is powerful to model massive genomic data while genome privacy is a growing concern. Studies have shown that not only the raw data but also the trained model can potentially infringe genome privacy. An example is the membership inference attack (MIA), by which the adversary can determine whether a specific record was included in the training dataset of the target model. Differential privacy (DP) has been used to defend against MIA with rigorous privacy guarantee by perturbing model weights. In this paper, we investigate the vulnerability of machine learning against MIA on genomic data, and evaluate the effectiveness of using DP as a defense mechanism. We consider two widely-used machine learning models, namely Lasso and convolutional neural network (CNN), as the target models. We study the trade-off between the defense power against MIA and the prediction accuracy of the target model under various privacy settings of DP. Our results show that the relationship between the privacy budget and target model accuracy can be modeled as a log-like curve, thus a smaller privacy budget provides stronger privacy guarantee with the cost of losing more model accuracy. We also investigate the effect of model sparsity on model vulnerability against MIA. Our results demonstrate that in addition to prevent overfitting, model sparsity can work together with DP to significantly mitigate the risk of MIA.