Reviewerly/exHarmony huggingface dataset:
Joint Commonsense and Relation Reasoning for Image and Video Captioning

Exploiting relationships between objects for image and video captioning has received increasing attention. Most existing methods depend heavily on pre-trained detectors of objects and their relationships, and thus may not work well when facing detection challenges such as heavy occlusion, tiny-size objects, and long-tail classes. In this paper, we propose a joint commonsense and relation reasoning method that exploits prior knowledge for image and video captioning without relying on any detectors. The prior knowledge provides semantic correlations and constraints between objects, serving as guidance to build semantic graphs that summarize object relationships, some of which cannot be directly perceived from images or videos. Particularly, our method is implemented by an iterative learning algorithm that alternates between 1) commonsense reasoning for embedding visual regions into the semantic space to build a semantic graph and 2) relation reasoning for encoding semantic graphs to generate sentences. Experiments on several benchmark datasets validate the effectiveness of our prior knowledge-based approach.
Towards Oracle Knowledge Distillation with Neural Architecture Search

We present a novel framework of knowledge distillation that is capable of learning powerful and efficient student models from ensemble teacher networks. Our approach addresses the inherent model capacity issue between teacher and student and aims to maximize benefit from teacher models during distillation by reducing their capacity gap. Specifically, we employ a neural architecture search technique to augment useful structures and operations, where the searched network is appropriate for knowledge distillation towards student models and free from sacrificing its performance by fixing the network capacity. We also introduce an oracle knowledge distillation loss to facilitate model search and distillation using an ensemble-based teacher model, where a student network is learned to imitate oracle performance of the teacher. We perform extensive experiments on the image classification datasets—CIFAR-100 and TinyImageNet—using various networks. We also show that searching for a new student model is effective in both accuracy and memory size and that the searched models often outperform their teacher models thanks to neural architecture search with oracle knowledge distillation.
Reinforcement Learning with Non-Markovian Rewards

The standard RL world model is that of a Markov Decision Process (MDP). A basic premise of MDPs is that the rewards depend on the last state and action only. Yet, many real-world rewards are non-Markovian. For example, a reward for bringing coffee only if requested earlier and not yet served, is non-Markovian if the state only records current requests and deliveries. Past work considered the problem of modeling and solving MDPs with non-Markovian rewards (NMR), but we know of no principled approaches for RL with NMR. Here, we address the problem of policy learning from experience with such rewards. We describe and evaluate empirically four combinations of the classical RL algorithm Q-learning and R-max with automata learning algorithms to obtain new RL algorithms for domains with NMR. We also prove that some of these variants converge to an optimal policy in the limit.
Perspective Transformation Data Augmentation for Object Detection

One major reason for the success of convolutional neural networks (CNNs) is the availability of large-scale labeled data.Effective training of CNNs relies on large annotated data.Unfortunately, large amounts of data with corresponding annotations are too expensive to obtain in some real-world applications.One reasonable alternative is to use data augmentation techniques to automatically generate annotated samples.In this paper, a novel data augmentation framework based on perspective transformation is proposed.This method automatically generates new annotated data without extra manual labeling, thus effectively extends the inadequate dataset.Perspective transformation can produce new images captured from any cameras viewpoints.Therefore, our method can mimic images taken at the angle that the camera cannot reach.Extensive experimental results on several datasets have demonstrated that our perspective transformation data augmentation strategy is an effective tool when using deep CNNs on small or imbalance datasets.
Human-Centric Clothing Segmentation via Deformable Semantic Locality-Preserving Network

In the fields of computer vision and graphics, clothing segmentation is a challenging and practical task which is typically implemented in a fine-grained semantic segmentation framework. Unlike the generic semantic segmentation task, clothing segmentation has some domain-specific properties such as diverse appearance variations, non-rigid geometry deformations, and small sample learning. To deal with these points, we propose a semantic locality-preserving segmentation model, which adaptively attaches an original clothing image with a semantically similar (e.g., appearance or pose) auxiliary exemplar by search. Through considering the interactions of the clothing image and its exemplar, more intrinsic knowledge about the locality manifold structures of clothing images is discovered to make the learning process of small sample problem more stable and tractable. Besides, we present a CNN model based on the deformable convolutions to extract the non-rigid geometry-aware features for clothing images. Furthermore, we apply our semantic locality-preserving segmentation model in both image and video cases, resulting in favorable clothing segmentation performance. Experimental results demonstrate the effectiveness of the proposed model against the state-of-the-art approaches.
Smart City Ecosystem Using Blockchain Technology

Given the continuous development of urban areas, integrated policies are needed to improve the lives of urban residents by implementing integrated, interoperable and secure electronic services.Public administrations recognize the potential of blockchain in solving the problems of local communities.Blockchain, initially perceived through its association with cryptocurrency bitcoin, offers an innovative perspective on how smart cities can be organized and a more transparent economic model for resource management.This study analyses how blockchain technology-based services can contribute to the development of smart cities and proposes a Smart City ecosystem model based on SSI (Self-Sovereign Identity) authentication model and smart contracts between entities, citizens and administrations.It also overviews the domains in which this technology can be used.The results can be a starting point for the development of local initiatives for using the blockchain as a platform for communications and transactions in the public sector.
FastLAS: Scalable Inductive Logic Programming Incorporating Domain-Specific Optimisation Criteria

Inductive Logic Programming (ILP) systems aim to find a set of logical rules, called a hypothesis, that explain a set of examples. In cases where many such hypotheses exist, ILP systems often bias towards shorter solutions, leading to highly general rules being learned. In some application domains like security and access control policies, this bias may not be desirable, as when data is sparse more specific rules that guarantee tighter security should be preferred. This paper presents a new general notion of a scoring function over hypotheses that allows a user to express domain-specific optimisation criteria. This is incorporated into a new ILP system, called FastLAS, that takes as input a learning task and a customised scoring function, and computes an optimal solution with respect to the given scoring function. We evaluate the accuracy of FastLAS over real-world datasets for access control policies and show that varying the scoring function allows a user to target domain-specific performance metrics. We also compare FastLAS to state-of-the-art ILP systems, using the standard ILP bias for shorter solutions, and demonstrate that FastLAS is significantly faster and more scalable.
Sequential Modeling of Hierarchical User Intention and Preference for Next-item Recommendation

The next-item recommendation has attracted great research interests with both static and dynamic users' preferences considered. Existing approaches typically utilize user-item binary relations, and assume a flat preference distribution over items for each user. However, this assumption neglects the hierarchical discrimination between user intentions and user preferences, causing the methods have limited capacity to depict intention-specific preference. In fact, a consumer's purchasing behavior involves a natural sequential process, i.e., he/she first has an intention to buy one type of items, followed by choosing a specific item according to his/her preference under this intention. To this end, we propose a novel key-array memory network (KA-MemNN), which takes both user intentions and preferences into account for next-item recommendation. Specifically, the user behavioral intention tendency is determined through key addressing. Further, each array outputs an intention-specific preference representation of a user. Then, the degree of user's behavioral intention tendency and intention-specific preference representation are combined to form a hierarchical representation of a user. This representation is further utilized to replace the static profile of users in traditional matrix factorization for the purposes of reasoning. The experimental results on real-world data demonstrate the advantages of our approach over state-of-the-art methods.
AWK AND GNU OCTAVE PROGRAMMING LANGUAGESINTEGRATED WITH GENERIC MAPPING TOOLS FOR GEOMORPHOLOGICAL ANALYSIS

The paper presents a sequential use of the AWK and GNU Octave programming languages integrated with Generic Mapping Tools (GMT) for geospatial data analysis. The geographic scope of the research is focused on the Kuril-Kamchatka Trench, north Pacific Ocean. Practical research aim is to analyse and compare bathymetry in the southern and northern part of the trench using digitized cross-section profiles. The initial mapping and geospatial analysis was performed in GMT scripting toolset. The GMT was used for cartographic mapping based on the raster ETOPO1 grid and automatic digitizing of the profiles crossing the trench perpendicularly. Besides visualized map, the processed geodata were received in a numerical form as a complex multi-field table for each segment. These tables were generated by the GMT in its native format and could not be directly processed by the MATLAB/Octave. Therefore, the tables were exported to AWK, a data-driven programming language and a powerful tool for data extraction. The table was then restructured, sorted and reshaped by the AWK script. Because the total amount of profiles overstepped 100 (62 and 52 for the northern and southern trench segments), only selected profiles were visualized. For this purpose, at the next step, the modified tables were converted to GNU Octave language for visualizing and plotting selected profiles. Finally, the geomorphology was analysed and two segments compared. The results show that the southern part has deeper bathymetric values, vary in geomorphic structure and has steeper gradient slopes comparing to the north, which is caused by the seismicity, volcanism, geologic and tectonic settings. Three full scripts of GMT, AWK and GNU Octave programming languages are presented for replicability in the Appendix. Keywords: AWK, Octave, GMT, programming language, data analysis, geospatial analysis, trench &nbsp; DOI 10.35180/gse-2019-0020
Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent

Despite the great achievements of the modern deep neural networks (DNNs), the vulnerability/robustness of state-of-the-art DNNs raises security concerns in many application domains requiring high reliability. Various adversarial attacks are proposed to sabotage the learning performance of DNN models. Among those, the black-box adversarial attack methods have received special attentions owing to their practicality and simplicity. Black-box attacks usually prefer less queries in order to maintain stealthy and low costs. However, most of the current black-box attack methods adopt the first-order gradient descent method, which may come with certain deficiencies such as relatively slow convergence and high sensitivity to hyper-parameter settings. In this paper, we propose a zeroth-order natural gradient descent (ZO-NGD) method to design the adversarial attacks, which incorporates the zeroth-order gradient estimation technique catering to the black-box attack scenario and the second-order natural gradient descent to achieve higher query efficiency. The empirical evaluations on image classification datasets demonstrate that ZO-NGD can obtain significantly lower model query complexities compared with state-of-the-art attack methods.
Pseudorandom Number Generator (PRNG) Design Using Hyper-Chaotic Modified Robust Logistic Map (HC-MRLM)

Robust chaotic systems, due to their inherent properties of mixing, ergodicity, and larger chaotic parameter space, constitute a perfect candidate for cryptography. This paper reports a novel method to generate random numbers using modified robust logistic map (MRLM). The non-smooth probability distribution function of robust logistic map (RLM) trajectories gives an un-even binary distribution in randomness test. To overcome this disadvantage in RLM, control of chaos (CoC) is proposed for smooth probability distribution function of RLM. For testing the proposed design, cryptographic random numbers generated by MRLM were vetted with National Institute of Standards and Technology statistical test suite (NIST 800-22). The results showed that proposed MRLM generates cryptographically secure random numbers (CSPRNG).
A Byte-level CNN Method to Detect DNS Tunnels

DNS tunnel is a technology used to bypass firewalls for data exfiltration. It takes advantage of the characteristic of firewalls that always allow DNS traffic pass. Most of the studies have realized the detection of DNS tunnels from the perspective of traffic and payload. In recent years, due to the popularity of machine learning and data mining methods, researchers began trying to use them to detect DNS tunnels. However, these detection methods require a lot of professional knowledge to extract and construct feature set, and the detection results are often unsatisfactory. In order to solve the problem of manual feature extraction limitation and improve the detection accuracy of DNS tunnels, we propose a deep learning method, called Byte-level CNN, to detect the DNS tunnels in this paper. Furthermore, we propose to represent DNS packets with bytes for the first time, as the converted computable data to the CNN model. Compared with traditional machine learning methods, our byte-level CNN method can learn full information in the whole DNS packets, especially the sequential and structural information, besides common statistical information, and therefore achieves good performance results. The simulation showed that the detection accuracy and recall rate are significantly improved, and the complex manual feature extraction process is avoided as well.
Multi-hop Knowledge Base Question Answering with an Iterative Sequence Matching Model

Knowledge Base Question Answering (KBQA) has attracted much attention and recently there has been more interest in multi-hop KBQA. In this paper, we propose a novel iterative sequence matching model to address several limitations of previous methods for multi-hop KBQA. Our method iteratively grows the candidate relation paths that may lead to answer entities. The method prunes away less relevant branches and incrementally assigns matching scores to the paths. Empirical results demonstrate that our method can significantly outperform existing methods on three different benchmark datasets.
Differential Privacy and Social Science: An Urgent Puzzle

Accessing and combining large amounts of data is important for quantitative social scientists, but increasing amounts of data also increase privacy risks.To mitigate these risks, important players in official statistics, academia, and business see a solution in the concept of differential privacy.In this opinion piece, we ask how differential privacy can benefit from social-scientific insights, and, conversely, how differential privacy is likely to transform social science.First, we put differential privacy in the larger context of social science.We argue that the discussion on implementing differential privacy has been clouded by incompatible subjective beliefs about risk, each perspective having merit for different data types.Moreover, we point out existing social-scientific insights that suggest limitations to the premises of differential privacy as a data protection approach.Second, we examine the likely consequences for social science if differential privacy is widely implemented.Clearly, workflows must change, and common social science data collection will become more costly.However, in addition to data protection, differential privacy may bring other positive side effects.These could solve some issues social scientists currently struggle with, such as p-hacking, data peeking, or overfitting; after all, differential privacy is basically a robust method to analyze data.We conclude that, in the discussion around privacy risks and data protection, a large number of disciplines must band together to solve this urgent puzzle of our time, including social science, computer science, ethics, law, and statistics, as well as public and private policy.
Recommendations for Services in a FAIR Data Ecosystem

The development and growing adoption of the FAIR data principles and associated standards as a part of research policies and practices place novel demands on research data services. This article highlights common challenges and priorities and proposes a set of recommendations on how data infrastructures can evolve and collaborate to provide services that support the implementation of the FAIR data principles, in particular in the context of building the European Open Science Cloud (EOSC). The recommendations cover a broad area of topics, including certification, infrastructure components, stewardship, costs, rewards, collaboration, training, support, and data management. These recommendations were prioritized according to their perceived urgency by different stakeholder groups and associated with actions as well as suggested action owners. This article is the output of three workshops organized by the projects FAIRsFAIR, RDA Europe, OpenAIRE, EOSC-hub, and FREYA designed to explore, discuss, and formulate recommendations among stakeholders in the scientific community. While the results are a work-in-progress, the challenges and priorities outlined provide a detailed and unique overview of current issues seen as crucial by the community that can sharpen and improve the roadmap toward a FAIR data ecosystem.
DIoTA: Decentralized-Ledger-Based Framework for Data Authenticity Protection in IoT Systems

It is predicted that more than 20 billion IoT devices will be deployed worldwide by 2020. These devices form the critical infrastructure to support a variety of important applications such as smart city, smart grid, and industrial internet. To guarantee that these applications work properly, it is imperative to authenticate these devices and data generated from them. Although digital signatures can be applied for these purposes, the scale of the overall system and the limited computation capability of IoT devices pose two big challenges. In order to overcome these obstacles, we propose DIoTA, a novel decentralized ledger-based authentication framework for IoT devices. DIoTA uses a two-layer decentralized ledger architecture together with a lightweight data authentication mechanism to facilitate IoT devices and data management. We also analyze the performance and security of DIoTA, and explicitly give the major parameters an administrator can choose to achieve a desirable balance between different metrics.
A Robust Image Encryption Algorithm Based on a 32-bit Chaotic System

In these years, image encryption systems based on chaotic maps have received considerable attention from researchers.For the security of encryption algorithms, it is important to ensure that chaotic systems can produce high quality pseudo-random sequences.However, when a chaotic system that theoretically has good random performance is deployed in a limited precision platform, its random performance will deteriorate.We prove that even if the precision reaches 256-bit, the piecewise linear chaotic map still fails to achieve the desired random performance.Moreover, in the case of low precision, the cascade structure commonly used in mixed chaotic maps will aggravate the problem of random degeneration.To generate good pseudo-random numbers in low-precision platforms, we propose a positive integer random sequence generator, called 32-bit PL_PWLCM&LM, which comprises two 32-bit PL_PWLCMs and one 32-bit logistic map in parallel.In our proposed image encryption system, we adopt the permutation algorithm with four pixels as a permutation unit and the diffusion algorithm based on rows and columns.The security and performance analyses prove that the proposed scheme is a secure, robust, and fast image encryption algorithm.
Face Hallucination With Finishing Touches

Obtaining a high-quality frontal face image from a low-resolution (LR) non-frontal face image is primarily important for many facial analysis applications. However, mainstreams either focus on super-resolving near-frontal LR faces or frontalizing non-frontal high-resolution (HR) faces. It is desirable to perform both tasks seamlessly for daily-life unconstrained face images. In this paper, we present a novel Vivid Face Hallucination Generative Adversarial Network (VividGAN) for simultaneously super-resolving and frontalizing tiny non-frontal face images. VividGAN consists of coarse-level and fine-level Face Hallucination Networks (FHnet) and two discriminators, i.e., Coarse-D and Fine-D. The coarse-level FHnet generates a frontal coarse HR face and then the fine-level FHnet makes use of the facial component appearance prior, i.e., fine-grained facial components, to attain a frontal HR face image with authentic details. In the fine-level FHnet, we also design a facial component-aware module that adopts the facial geometry guidance as clues to accurately align and merge the frontal coarse HR face and prior information. Meanwhile, two-level discriminators are designed to capture both the global outline of a face image as well as detailed facial characteristics. The Coarse-D enforces the coarsely hallucinated faces to be upright and complete while the Fine-D focuses on the fine hallucinated ones for sharper details. Extensive experiments demonstrate that our VividGAN achieves photo-realistic frontal HR faces, reaching superior performance in downstream tasks, i.e., face recognition and expression classification, compared with other state-of-the-art methods.