Reviewerly/exHarmony huggingface dataset:
AutoDAL: Distributed Active Learning with Automatic Hyperparameter Selection

Automated machine learning (AutoML) strives to establish an appropriate machine learning model for any dataset automatically with minimal human intervention. Although extensive research has been conducted on AutoML, most of it has focused on supervised learning. Research of automated semi-supervised learning and active learning algorithms is still limited. Implementation becomes more challenging when the algorithm is designed for a distributed computing environment. With this as motivation, we propose a novel automated learning system for distributed active learning (AutoDAL) to address these challenges. First, automated graph-based semi-supervised learning is conducted by aggregating the proposed cost functions from different compute nodes in a distributed manner. Subsequently, automated active learning is addressed by jointly optimizing hyperparameters in both the classification and query selection stages leveraging the graph loss minimization and entropy regularization. Moreover, we propose an efficient distributed active learning algorithm which is scalable for big data by first partitioning the unlabeled data and replicating the labeled data to different worker nodes in the classification stage, and then aggregating the data in the controller in the query selection stage. The proposed AutoDAL algorithm is applied to multiple benchmark datasets and a real-world electrocardiogram (ECG) dataset for classification. We demonstrate that the proposed AutoDAL algorithm is capable of achieving significantly better performance compared to several state-of-the-art AutoML approaches and active learning algorithms.
Transparent Classification with Multilayer Logical Perceptrons and Random Binarization

Models with transparent inner structure and high classification performance are required to reduce potential risk and provide trust for users in domains like health care, finance, security, etc. However, existing models are hard to simultaneously satisfy the above two properties. In this paper, we propose a new hierarchical rule-based model for classification tasks, named Concept Rule Sets (CRS), which has both a strong expressive ability and a transparent inner structure. To address the challenge of efficiently learning the non-differentiable CRS model, we propose a novel neural network architecture, Multilayer Logical Perceptron (MLLP), which is a continuous version of CRS. Using MLLP and the Random Binarization (RB) method we proposed, we can search the discrete solution of CRS in continuous space using gradient descent and ensure the discrete CRS acts almost the same as the corresponding continuous MLLP. Experiments on 12 public data sets show that CRS outperforms the state-of-the-art approaches and the complexity of the learned CRS is close to the simple decision tree.
Find Objects and Focus on Highlights: Mining Object Semantics for Video Highlight Detection via Graph Neural Networks

With the increasing prevalence of portable computing devices, browsing unedited videos is time-consuming and tedious. Video highlight detection has the potential to significantly ease this situation, which discoveries moments of user's major or special interest in a video. Existing methods suffer from two problems. Firstly, most existing approaches only focus on learning holistic visual representations of videos but ignore object semantics for inferring video highlights. Secondly, current state-of-the-art approaches often adopt the pairwise ranking-based strategy, which cannot enjoy the global information to infer highlights. Therefore, we propose a novel video highlight framework, named VH-GNN, to construct an object-aware graph and model the relationships between objects from a global view. To reduce computational cost, we decompose the whole graph into two types of graphs: a spatial graph to capture the complex interactions of object within each frame, and a temporal graph to obtain object-aware representation of each frame and capture the global information. In addition, we optimize the framework via a proposed multi-stage loss, where the first stage aims to determine the highlight-probability and the second stage leverage the relationships between frames and focus on hard examples from the former stage. Extensive experiments on two standard datasets strongly evidence that VH-GNN obtains significant performance compared with state-of-the-arts.
Robust Perceptual Image Hashing using SIFT and SVD

With the advancement in technology, digital data such as image, video, etc. can be easily manipulated.Image hashing is a method that can be used for authentication and identification of digital images.In this communication, a robust image hashing technique is proposed using scale invariant feature transform (SIFT), singular value decomposition (SVD) and Zernike moment.Zernike moment is used to restore the image against the rotation attack.Potential points are generated from the image by using SIFT.Block processing of equal size is performed on the input image.Key points within the same block are used to generate hashing values.The experimental outcome shows that the proposed technique can withstand different types of attacks.Receiver operating characteristic curve comparison specifies that our method outperforms other existing methods under consideration.
How to Ask Better Questions? A Large-Scale Multi-Domain Dataset for Rewriting Ill-Formed Questions

We present a large-scale dataset for the task of rewriting an ill-formed natural language question to a well-formed one. Our multi-domain question rewriting (MQR) dataset is constructed from human contributed Stack Exchange question edit histories. The dataset contains 427,719 question pairs which come from 303 domains. We provide human annotations for a subset of the dataset as a quality estimate. When moving from ill-formed to well-formed questions, the question quality improves by an average of 45 points across three aspects. We train sequence-to-sequence neural models on the constructed dataset and obtain an improvement of 13.2% in BLEU-4 over baseline methods built from other data resources. We release the MQR dataset to encourage research on the problem of question rewriting.1
Plug-in, Trainable Gate for Streamlining Arbitrary Neural Networks

Architecture optimization, which is a technique for finding an efficient neural network that meets certain requirements, generally reduces to a set of multiple-choice selection problems among alternative sub-structures or parameters. The discrete nature of the selection problem, however, makes this optimization difficult. To tackle this problem we introduce a novel concept of a trainable gate function. The trainable gate function, which confers a differentiable property to discrete-valued variables, allows us to directly optimize loss functions that include non-differentiable discrete values such as 0-1 selection. The proposed trainable gate can be applied to pruning. Pruning can be carried out simply by appending the proposed trainable gate functions to each intermediate output tensor followed by fine-tuning the overall model, using any gradient-based training methods. So the proposed method can jointly optimize the selection of the pruned channels while fine-tuning the weights of the pruned model at the same time. Our experimental results demonstrate that the proposed method efficiently optimizes arbitrary neural networks in various tasks such as image classification, style transfer, optical flow estimation, and neural machine translation.
Contract-Based Approach for Security Deposit in Blockchain Networks with Shards

As a decentralized ledger technology, blockchain is considered to be a potential solution for applications with highly concentrated management mechanism. However, most of the existing blockchain networks are employed with the hash-puzzle-solving consensus protocol, known as proof-of-work. The competition of solving the puzzle introduces high latency, which directly leads to a long transaction-processing time. One solution of this dilemma is to establish a blockchain network with shards. In this paper, we focus on the blockchain network with shards and adopt the security-deposit based consensus protocol, studying the problem of how to balance the security incentive and the economic incentive. Also, the inherent features of the blockchain, i.e., anonymity and decentralization, introduce the information asymmetric issue between the beacon chain and the participants. The contract theory is utilized to formulate the problem between them. As such, the optimal rewards related to the different types of validators can be obtained, as well as the reasonable deposits accordingly. Compared with the fixed deposits, the flexible deposits can provide enough economic incentive for the participants without losing the security incentives. Besides, the simulation results demonstrate that the contract theory approach is capable of maximizing the beacon chain's utility and satisfying the incentive compatibility and individual rationality of the participants.
Sequence Generation with Optimal-Transport-Enhanced Reinforcement Learning

Reinforcement learning (RL) has been widely used to aid training in language generation. This is achieved by enhancing standard maximum likelihood objectives with user-specified reward functions that encourage global semantic consistency. We propose a principled approach to address the difficulties associated with RL-based solutions, namely, high-variance gradients, uninformative rewards and brittle training. By leveraging the optimal transport distance, we introduce a regularizer that significantly alleviates the above issues. Our formulation emphasizes the preservation of semantic features, enabling end-to-end training instead of ad-hoc fine-tuning, and when combined with RL, it controls the exploration space for more efficient model updates. To validate the effectiveness of the proposed solution, we perform a comprehensive evaluation covering a wide variety of NLP tasks: machine translation, abstractive text summarization and image caption, with consistent improvements over competing solutions.
Generating Text Sequence Images for Recognition

Recently, methods based on deep learning have dominated the field of text recognition. With a large number of training data, most of them can achieve the state-of-the-art performances. However, it is hard to harvest and label sufficient text sequence images from the real scenes. To mitigate this issue, several methods to synthesize text sequence images were proposed, yet they usually need complicated preceding or follow-up steps. In this work, we present a method which is able to generate infinite training data without any auxiliary pre/post-process. We tackle the generation task as an image-to-image translation one and utilize conditional adversarial networks to produce realistic text sequence images in the light of the semantic ones. Some evaluation metrics are involved to assess our method and the results demonstrate that the caliber of the data is satisfactory. The code and dataset will be publicly available soon.
Image Hazing Algorithm Based on Generative Adversarial Networks

Haze is an important factor in photography with a special aesthetic, emotional, or composi- tional meaning, an image hazing method is proposed based on generative adversarial network. The proposed network consists of two parts: the generator has a symmetric encoder and decoder structure with skip connection, which is used to generate hazy images; and the discriminator is a global fully convolutional network, which is used to identify the reality of the generated hazy images. The haze-free image is used as the input of the generative network to obtain a corresponding hazy image. Then the discriminative network judges the similarity between the original haze-free image and the corresponding hazy image to optimize the network parameters, which means that all the parameters of the entire network can be automatically learned through training. The loss function of the network is the combination of the GAN(Generative Adversarial Networks) loss and the REG(Regression) loss with a coefficient $\lambda $ . We design a REG loss which includes feature loss $loss_{FL}$ and image loss $loss_{IL}$ . By calculating the loss, a model mapping the relationship between the two corresponding pixels from the original image and the synthetic hazy image respectively is well trained. In the last part of the paper, we have shown some impressing synthetic hazy images obtained by our model as well as analyzing the effects on images with different scenes. Moreover, we have contrasted the experimental results of our algorithm with that of other classical ones. Experiments demonstrate that the proposed algorithm has a better performance than other state-of-the-art methods on both virtual-scene and real-world images qualitatively and quantitatively.
La llegada de las nuevas tecnologías a la educación y sus implicaciones

In recent decades we have observed a process of integrating new technologies into education. The arrival of ICT in the classroom has brought with it certain advantages, but also some inconveniences that force us to question ourselves about the relationship between the incorporation of new technologies and the improvement of educational quality.Therefore, the integration of new technologies into education has resulted in changes in the curriculum, in schools and their facilities, in the roles of students and teachers, etc. However, the current reality of the classrooms shows that most of the new technologies have involved a technological innovation (use of projectors, virtual platforms, resources and teaching materials, etc.), but has not been accompanied by a pedagogical innovation.However, the most relevant consequence of the process of integrating ICTs into classrooms and which still constitutes a pending subject for our education is the possibility of launching educational innovations, leading to a constructivist, active, collaborative and personalized teaching-learning process. A teaching-learning process designed, tutored, mediated, guided and evaluated by teachers, but whose main protagonist is the student.
Public key encryption with conjunctive keyword search on lattice

Outsourcing data in cloud storage is gaining popularity recently. For protecting data privacy, the data should be encrypted, and it is thus critical to provide search services for encrypted outsourced data. However, it becomes a problem about how to retrieve encrypted data while protecting data privacy at the same time. Therefore, public key encryption with keyword search (PEKS) is proposed. Nevertheless, the majority of the existing PEKS schemes are based on bilinear mappings and do not provide quantum security. Although some lattice-based PEKS schemes have been proposed, they cannot support conjunctive keyword search. In this paper, we first propose a public key encryption with conjunctive keyword search (PECK) using lattices. Our PECK scheme provides efficient search over encrypted data containing multiple keywords for a single data user. To support multiple data users, a multi-user PECK scheme is also presented. The two schemes are proved secure against indistinguishable chosen keyword attacks (IND-CKA) based on learning with errors (LWE) problem in the random oracle model. Finally, we provide performance evaluation of our proposed scheme by theoretical analysis and experimental simulation.
Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learning

We explore fixed-horizon temporal difference (TD) methods, reinforcement learning algorithms for a new kind of value function that predicts the sum of rewards over a fixed number of future time steps. To learn the value function for horizon h, these algorithms bootstrap from the value function for horizon h−1, or some shorter horizon. Because no value function bootstraps from itself, fixed-horizon methods are immune to the stability problems that plague other off-policy TD methods using function approximation (also known as “the deadly triad”). Although fixed-horizon methods require the storage of additional value functions, this gives the agent additional predictive power, while the added complexity can be substantially reduced via parallel updates, shared weights, and n-step bootstrapping. We show how to use fixed-horizon value functions to solve reinforcement learning problems competitively with methods such as Q-learning that learn conventional value functions. We also prove convergence of fixed-horizon temporal difference methods with linear and general function approximation. Taken together, our results establish fixed-horizon TD methods as a viable new way of avoiding the stability problems of the deadly triad.
Adversarial symmetric GANs: Bridging adversarial samples and adversarial networks

Generative adversarial networks have achieved remarkable performance on various tasks but suffer from training instability. Despite many training strategies proposed to improve training stability, this issue remains as a challenge. In this paper, we investigate the training instability from the perspective of adversarial samples and reveal that adversarial training on fake samples is implemented in vanilla GANs, but adversarial training on real samples has long been overlooked. Consequently, the discriminator is extremely vulnerable to adversarial perturbation and the gradient given by the discriminator contains non-informative adversarial noises, which hinders the generator from catching the pattern of real samples. Here, we develop adversarial symmetric GANs (AS-GANs) that incorporate adversarial training of the discriminator on real samples into vanilla GANs, making adversarial training symmetrical. The discriminator is therefore more robust and provides more informative gradient with less adversarial noise, thereby stabilizing training and accelerating convergence. The effectiveness of the AS-GANs is verified on image generation on CIFAR-10, CIFAR-100, CelebA, and LSUN with varied network architectures. Not only the training is more stabilized, but the FID scores of generated samples are consistently improved by a large margin compared to the baseline. Theoretical analysis is also conducted to explain why AS-GAN can improve training. The bridging of adversarial samples and adversarial networks provides a new approach to further develop adversarial networks.
An Abstraction Based Approach for Reconstruction of TimeLine in Digital Forensics

Acquiring a clear perspective of events and artefacts that occur over time is a challenging objective to accomplish in digital forensics. Reconstruction of the timeline of events and artefacts, which enables digital investigators to understand the timeline of digital crime and interpret the conclusion in the form of digital evidence, is one of the most paramount and challenging tasks in digital forensics. This challenging task requires the analysis of immense amounts of events because of the explosive growth of the internet, interconnected devices, and innovative technology nowadays. Various approaches have been developed during the last decade, but most of them are not able to handle huge volumes of data, explore evidence, and enhance the understandability of timelines in a competent way to assist the investigator. For this purpose, we introduce a methodology backed by an abstraction concept and forensic tools that can support investigators during the reconstruction, understanding of the timeline of events and artefacts, and interpretation of evidence by tracing the activities performed by users of the typical computer system. The Java programming language is used to implement the proposed methodology, which is object-oriented and follows the symmetry definition in software. Generally, symmetry in software can be viewed as an invariant change that aims to preserve a specific property of the system, namely its structure, behaviour, regularity, similarity, familiarity and uniformity. Similarly, the abstraction-based methodology also permits us to follow the properties of symmetry. For instance, a uniform structure is stipulated for all the sources at the particular level of abstraction, such as the number of fields to be considered to provide the abstract level of timeline. The primary purpose of this approach is to assist with the analysis of the timeline in an optimum way. This paper illustrates the approach and then focuses on conceptual aspects of the methodology. The performed experiment shows that the proposed approach enhanced the analysis of the timeline.
Layerwise Sparse Coding for Pruned Deep Neural Networks with Extreme Compression Ratio

Deep neural network compression is important and increasingly developed especially in resource-constrained environments, such as autonomous drones and wearable devices. Basically, we can easily and largely reduce the number of weights of a trained deep model by adopting a widely used model compression technique, e.g., pruning. In this way, two kinds of data are usually preserved for this compressed model, i.e., non-zero weights and meta-data, where meta-data is employed to help encode and decode these non-zero weights. Although we can obtain an ideally small number of non-zero weights through pruning, existing sparse matrix coding methods still need a much larger amount of meta-data (may several times larger than non-zero weights), which will be a severe bottleneck of the deploying of very deep models. To tackle this issue, we propose a layerwise sparse coding (LSC) method to maximize the compression ratio by extremely reducing the amount of meta-data. We first divide a sparse matrix into multiple small blocks and remove zero blocks, and then propose a novel signed relative index (SRI) algorithm to encode the remaining non-zero blocks (with much less meta-data). In addition, the proposed LSC performs parallel matrix multiplication without full decoding, while traditional methods cannot. Through extensive experiments, we demonstrate that LSC achieves substantial gains in pruned DNN compression (e.g., 51.03x compression ratio on ADMM-Lenet) and inference computation (i.e., time reduction and extremely less memory bandwidth), over state-of-the-art baselines.
Bringing Blockchain Technology to Construction Engineering Management

The objectives of this research are to study the feasibility and application of blockchain technology in the management of construction engineering, which using qualitative research methods.From in-depth interviews with experts who work in the construction management industry and blockchain technology.The study found that, the bringing Blockchain technology in the management of construction engineering is possible, consisting of 5 areas: cost, time, quality, transparency, and information security.The apply Blockchain technology to reduce costs or limit the cost of centralized data auditing, including security and transparency.That it can use in the management of construction engineering since beginning the project through the process on the way to the project delivery.
Words in Kitchen: An Instance of Leveraging Virtual Reality Technology to Learn Vocabulary

Vocabulary is an essential part in second language learning. The traditional way of learning vocabulary is dull and boring in class with thin context. Nevertheless, a rich context for memorizing words is necessary according to the situated learning theory. Thanks to virtual reality (VR) technology, such a rich context can be achieved by creating a virtual simulation scenario. "Words In Kitchen" is a VR system designed as an educational tool for second language learners to learn vocabulary. With the help of the proposed system, a user can interact with virtual objects in the virtual scene to learn the spelling and pronunciation of specific vocabularies. We aim to make full use of immersion and interaction of VR and propose a prototype of vocabulary learning in VR, so that the system can promote learners' common vocabulary learning as a supplement to class and textbooks. A pilot study was conducted to identify its usability and enjoyment and the result indicated that the system could help remember words and stimulate interests.