Reviewerly/exHarmony huggingface dataset:
Improving What Cross-Modal Retrieval Models Learn through Object-Oriented Inter- and Intra-Modal Attention Networks

Although significant progress has been made for cross-modal retrieval models in recent years, few have explored what those models truly learn and what makes one model superior to another. Start by training two state-of-the-art text-to-image retrieval models with adversarial text inputs, we investigate and quantify the importance of syntactic structure and lexical information in learning the joint visual-semantic embedding space for cross-modal retrieval. The results show that the retrieval power mainly comes from localizing and connecting the visual objects and their cross-modal counter-parts, the textual phrases. Inspired by this observation, we propose a novel model which employs object-oriented encoders along with inter- and intra-modal attention networks to improve inter-modal dependencies for cross-modal retrieval. In addition, we develop a new multimodal structure-preserving objective which additionally emphasizes intra-modal hard negative examples to promote intra-modal discrepancies. Extensive experiments show that the proposed approach outperforms the existing best method by a large margin (16.4% and 6.7% relatively with [email protected] in the text-to-image retrieval task on the Flickr30K dataset and the MS-COCO dataset respectively).
Subverting Decryption in AEAD

This work introduces a new class of Algorithm Substitution Attack (ASA) on Symmetric Encryption Schemes. ASAs were introduced by Bellare, Paterson and Rogaway in light of revelations concerning mass surveillance. An ASA replaces an encryption scheme with a subverted version that aims to reveal information to an adversary engaged in mass surveillance, while remaining undetected by users. Previous work posited that a particular class of AEAD scheme (satisfying certain correctness and uniqueness properties) is resilient against subversion. Many if not all real-world constructions – such as GCM, CCM and OCB – are members of this class. Our results stand in opposition to those prior results. We present a potent ASA that generically applies to any AEAD scheme, is undetectable in all previous frameworks and which achieves successful exfiltration of user keys. We give even more efficient non-generic attacks against a selection of AEAD implementations that are most used in practice. In contrast to prior work, our new class of attack targets the decryption algorithm rather than encryption. We argue that this attack represents an attractive opportunity for a mass surveillance adversary. Our work serves to refine the ASA model and contributes to a series of papers that raises awareness and understanding about what is possible with ASAs.
Secure Multi-Constellation GNSS Receivers with Clustering-Based Solution Separation Algorithm

Because of the limited satellite visibility, reduced signal reception reliability and constraining spatial geometry, e.g., in urban areas, the development of multi-constellation global navigation satellite systems (GNSS) has gained traction rapidly. GNSS-based applications are expected to handle observations from different navigation systems, e.g., GPS, GLONASS, Bei-Dou and Galileo, in order to improve positioning accuracy and reliability. Furthermore, multi-constellation receivers present an opportunity to better counter spoofing and replaying attacks, leveraging approaches take advantage of the redundant measurements. In particular, cluster-based solution separation algorithm (CSSA) proposes to detect and identify faulty/malicious signals in a single GPS constellation by checking the consistency of receiver positions calculated with different number of satellites. Intuitively, the algorithm targets directly the consequence of spoofing/replaying attacks: the victim receiver position error estimation. It works independently of how the attacks are launched, either through modifying pseudorange measurements or manipulating the navigation messages, without changing the receiver hardware. Multi-constellation GNSS receivers utilize all observations from different navigation systems, there are more than 30 available satellites at each epoch after Galileo and BeiDou systems become fully operational; in other words using abundant redundancy. Therefore, we introduce such a CSSA to a multi-constellation receiver. The work shows that a multi-constellation GNSS receiver equipped with our algorithm works effectively against a strong spoofing/replaying attacker that can manipulate a large number of signals, or even an entire constellation. The results show that CSSA with multi-constellation significantly improves the performance of detecting and identifying the malicious signals; particularly, when the adversary cannot control all the constellations, a multi-constellation receiver can identify the faults even the adversary induces very small errors to pseudorange measurements, comparing with a single constellation receiver. Moreover, when the attacker is powerful to manipulate most of signals of all the constellations, a multi-constellation receiver with CSSA can still detect and identify the faulty signals with high probability when the attacker tries to mislead the victim more than a couple of hundred meters from its true location.
Using Attribute-Based Encryption on IoT Devices with instant Key Revocation

The Internet of Things (IoT) relies on sensor devices to measure real-world phenomena in order to provide IoT services. The sensor readings are shared with multiple entities, such as IoT services, other IoT devices or other third parties. The collected data may be sensitive and include personal information. To protect the privacy of the users, the data needs to be protected through an encryption algorithm. For sharing cryptographic cipher-texts with a group of users Attribute-Based Encryption (ABE) is well suited, as it does not require to create group keys. However, the creation of ABE cipher-texts is slow when executed on resource constraint devices, such as IoT sensors. In this paper, we present a modification of an ABE scheme, which not only allows to encrypt data efficiently using ABE but also reduces the size of the cipher-text, that must be transmitted by the sensor. We also show how our modification can be used to realise an instantaneous key revocation mechanism.
An Interactive Approach to Integrating External Textual Knowledge for Multimodal Lifelog Retrieval

The semantic gap between textual queries and visual concepts is one of the key challenges in lifelog retrieval. This work presents an interactive system aimed at improving the retrieval accuracy by query term suggestion. Besides, this system also assists users to refine the retrieval results by image similarity clustering. For recommending a list of candidate words, we extract visual concepts from images by using computer vision models, and then incorporate both official and additional concepts into our system using pre-trained word embedding, in which textual knowledge is inherent. We also purpose an intelligent mechanism for rapidly removing multiple irrelevant search results. For reaching out this purpose, we build kd-trees [1] offline for reducing the computational overhead and cluster similar images by nearest neighbor search in the embedding space. Whenever users exclude some irrelevant images, their nearest neighbors in the image embedding space are also removed. In this way, users can efficiently screen out the relevant results and purge the irrelevant ones, scanning over more retrieval results in a shorter period of time.
A Method of Ultrasonic Finger Gesture Recognition Based on the Micro-Doppler Effect

With the popularity of small-screen smart mobile devices, gestures as a new type of human–computer interaction are highly demanded. Furthermore, finger gestures are more familiar to people in controlling devices. In this paper, a new method for recognizing finger gestures is proposed. Ultrasound was actively emitted to measure the micro-Doppler effect caused by finger motions and was obtained at high resolution. By micro-Doppler processing, micro-Doppler feature maps of finger gestures were generated. Since the feature map has a similar structure to the single channel color image, a recognition model based on a convolutional neural network was constructed for classification. The optimized recognition model achieved an average accuracy of 96.51% in the experiment.
<i>SoundUAV</i>

Drones are gaining a lot of traction in a wide spectrum of applications. This popularity makes them attractive attack surfaces, which necessitates the need for ensuring their security. Specifically, in the case of drone delivery, an attacker drone may impersonate the legitimate one in order to steal packages, which makes drone authentication important. Recent efforts have pushed to incorporate digital certificates as an authenticator for drones. However, such software-based techniques are often compromised and can be launched on a large scale, making them a bigger threat. To this end, we propose SoundUAV as an additional factor of authentication that leverages differences in the acoustic noise characteristics of drones to fingerprint them. These differences are caused due to manufacturing defects in their motors, making them hard to replicate. Moreover, SoundUAV does not require any hardware modifications to existing drones as they leverage the freely available sound information, and it is also robust to large-scale attacks as the attacks involve hardware alterations. To test the feasibility of SoundUAV, we evaluate it on 54 motors, and 11 drones of the same make and model, and report fingerprinting accuracy of 99.48%.
Digital watermarking scheme for colour remote sensing image based on quaternion wavelet transform and tensor decomposition

Digital watermarking is important for protecting the intellectual property of remote sensing images. Unlike watermarking in ordinary colour images, in colour remote sensing images, watermarking has an important requirement: robustness. In this paper, a robust nonblind watermarking scheme for colour remote sensing images, which considers both frequency and statistical pattern features, is constructed based on the quaternion wavelet transform (QWT) and tensor decomposition. Using the QWT, not only the abundant phase information can be used to preserve detailed host image features to improve the imperceptibility of the watermark, but also the frequency coefficients of the host image can provide a stable position to embed the watermark. To further strengthen the robustness, the global statistical feature structure acquired through the tensor Tucker decomposition is employed to distribute the watermark's energy among different colour bands. Because both the QWT frequency coefficients and the tensor decomposition global statistical feature structure are highly stable against external distortion, their integration yields the proposed scheme, which is robust to many image manipulations. A simulation experiment shows that our method can balance the trade‐off between imperceptibility and robustness and that it is more robust than the traditional QWT and discrete wavelet transform (DWT) methods under many different types of image manipulations.
On the improvement of reinforcement active learning with the involvement of cross entropy to address one-shot learning problem

As a promising research direction in recent decades, active learning allows an oracle to assign labels to typical examples for performance improvement in learning systems. Existing works mainly focus on designing criteria for screening examples of high value to be labeled in a handcrafted manner. Instead of manually developing strategies of querying the user to access labels for the desired examples, we utilized the reinforcement learning algorithm parameterized with the neural network to automatically explore query strategies in active learning when addressing stream-based one-shot classification problems. With the involvement of cross-entropy in the loss function of Q-learning, an efficient policy to decide when and where to predict or query an instance is learned through the developed framework. Compared with a former influential work, the advantages of our method are demonstrated experimentally with two image classification tasks, and it exhibited better performance, quick convergence, relatively good stability and fewer requests for labels.
Finding Densest Lasting Subgraphs in Dynamic Graphs: A Stochastic Approach

One important problem that is insufficiently studied is finding densest lasting-subgraphs in large dynamic graphs, which considers the time duration of the subgraph pattern. We propose a framework called Expectation-Maximization with Utility functions (EMU), a novel stochastic approach that nontrivially extends the conventional EM approach. EMU has the flexibility of optimizing any user-defined utility functions. We validate our EMU approach by showing that it converges to the optimum-by proving that it is a specification of the general Minorization-Maximization (MM) framework with convergence guarantees. We then devise EMU algorithms for the densest lasting subgraph problem. Using real-world graph data, we experimentally verify the effectiveness and efficiency of our techniques, and compare with two prior approaches on dense subgraph detection.
Content-based Image Retrieval for Breast Ultrasound Images using Convolutional Autoencoders: A Feasibility Study

Ultrasound imaging is one of the most widely used medical imaging modalities for detecting breast cancer. However, the accuracy of diagnosing the tumors in breast ultrasound (BUS) images might vary based on the experience level of the radiologist. Content-based image retrieval (CBIR) systems can be employed to improve the diagnosis accuracy by providing the radiologist with BUS images of previous, clinically relevant cases. In this study, a CBIR system is developed based on deep learning technology to support the diagnosis of BUS images. In particular, each query BUS image submitted to the system is analyzed using a custom-made convolutional autoencoder (CAE) to extract a latent features vector that represents the image. An important advantage of the proposed CAE is its ability to extract the latent features vector without the need to localize or outline the tumor. The latent features vector of the query image is analyzed using a similarity measure to identify and retrieve the most relevant BUS images from a reference BUS image database. Finally, the retrieved BUS images are displayed to the radiologist. The performance of the proposed CBIR system has been evaluated using a set of BUS images that includes benign and malignant breast tumors. The results reported in this study suggest the feasibility of employing the proposed CBIR system to improve the diagnosis of BUS images.
Reconstruction of non-degenerate homogeneous depth three circuits

A homogeneous depth three circuit C computes a polynomial f = T1 + T2 + ... + Ts, where each Ti is a product of d linear forms in n variables over some underlying field F. Given black-box access to f, can we efficiently reconstruct (i.e. proper learn) a homogeneous depth three circuit computing f? Learning various subclasses of circuits is natural and interesting from both theoretical and practical standpoints and in particular, properly learning homogeneous depth three circuits efficiently is stated as an open problem in a work by Klivans and Shpilka (COLT 2003) and is well-studied. Unfortunately, there is substantial amount of evidence to show that this is a hard problem in the worst case. We give a (randomized) poly(n,d,s)-time algorithm to reconstruct non-degenerate homogeneous depth three circuits for n = Ω(d2) (with some additional mild requirements on s and the characteristic of F). We call a circuit C as non-degenerate if the dimension of the partial derivative space of f equals the sum of the dimensions of the partial derivative spaces of the terms T1, T2, …, Ts. In this sense, the terms are "independent" of each other in a non-degenerate circuit. A random homogeneous depth three circuit (where the coefficients of the linear forms are chosen according to the uniform distribution or any other reasonable distribution) is almost surely non-degenerate. In comparison, previous learning algorithms for this circuit class were either improper (with an exponential dependence on d), or they only worked for s < n (with a doubly exponential dependence of the running time on s). The main contribution of this work is to formulate the following paradigm for efficiently handling addition gates and to successfully implement it for the class of homogeneous depth three circuits. The problem of finding the children of an addition gate with large fan-in s is first reduced to the problem of decomposing a suitable vector space U into a (direct) sum of simpler subspaces U1, U2, …, Us. One then constructs a suitable space of operators S consisting of linear maps acting on U such that analyzing the simultaneous global structure of S enables us to efficiently decompose U. In our case, we exploit the structure of the set of low rank matrices in S and of the invariant subspaces of U induced by S. We feel that this paradigm is novel and powerful: it should lead to efficient reconstruction of many other subclasses of circuits for which the efficient reconstruction problem had hitherto looked unapproachable because of the presence of large fan-in addition gates.
Emergent Leadership in Agile Teams--an Initial Exploration

The adoption of agile development methods is now widespread. A key aspect of the special type of teamwork promoted in agile methods is the proposition of self-organized teams without a formally assigned leader within the team. Contradicting this proposition, agile development is still in need of the outcomes of leadership such as coordination and facilitation of activities, which in turn drive performance. General group research has found leadership to emerge in an informal manner within leaderless teams. Integrating knowledge on governance in IT teams and general group research, we propose a first step to close the knowledge void on emergent leadership within agile teams and its effects on team outcomes. In this research-in-progress, we develop a conceptual model and deduct hypotheses on the emergence and effects of leadership in agile teams. We expect leadership to emerge informally within agile teams. Based on the propositions of agile development, we do not expect a single leader to emerge but leadership to be attributed to several team members. Given the different roles in agile development, this attribution may moreover differ based on different areas of reference. Drawing on extant knowledge of emergent leadership, we expect positive effects on performance if leadership is shared.
Towards Robust and Discriminative Sequential Data Learning

The last decade has witnessed a surge of interest in applying deep learning models for discovering sequential patterns from a large volume of data. Recent works show that deep learning models can be further improved by enforcing models to learn a smooth output distribution around each data point. This can be achieved by augmenting training data with slight perturbations that are designed to alter model outputs. Such adversarial training approaches have shown much success in improving the generalization performance of deep learning models on static data, e.g., transaction data or image data captured on a single snapshot. However, when applied to sequential data, the standard adversarial training approaches cannot fully capture the discriminative structure of a sequence. This is because real-world sequential data are often collected over a long period of time and may include much irrelevant information to the classification task. To this end, we develop a novel adversarial training approach for sequential data classification by investigating when and how to perturb a sequence for an effective data augmentation. Finally, we demonstrate the superiority of the proposed method over baselines in a diversity of real-world sequential datasets.
A Hybrid Autoencoder Network for Unsupervised Image Clustering

Image clustering involves the process of mapping an archive image into a cluster such that the set of clusters has the same information. It is an important field of machine learning and computer vision. While traditional clustering methods, such as k-means or the agglomerative clustering method, have been widely used for the task of clustering, it is difficult for them to handle image data due to having no predefined distance metrics and high dimensionality. Recently, deep unsupervised feature learning methods, such as the autoencoder (AE), have been employed for image clustering with great success. However, each model has its specialty and advantages for image clustering. Hence, we combine three AE-based models—the convolutional autoencoder (CAE), adversarial autoencoder (AAE), and stacked autoencoder (SAE)—to form a hybrid autoencoder (BAE) model for image clustering. The MNIST and CIFAR-10 datasets are used to test the result of the proposed models and compare the results with others. The results of the clustering criteria indicate that the proposed models outperform others in the numerical experiment.
TUBE

Given a project plan and the goal, can we predict the plan's success rate? The key challenge is to learn the feature vectors of billions of the plan's components for effective prediction. However, existing methods did not model the behavior outcomes but component proximities. In this work, we define a measurement of behavior outcomes, which forms a test tube-shaped region to represent "success", in a vector space. We propose a novel representation learning method to learn the embeddings of behavior components (including contexts, plans, and goals) by preserving the behavior outcome information. Experiments on real datasets show that our proposed method significantly improves the performance of goal prediction as well as context recommendation over the state-of-the-art.
Efficient Realization of Toffoli and NCV Circuits for IBM QX Architectures

The quantum computers available from IBM’s QX project, implement circuits with Clifford+T gates. In order to implement Toffoli or NCV circuits in such architectures, they need to be mapped to Clifford+T gates. Another challenge is that some CNOT gates cannot be directly implemented in the IBM quantum computers and must be changed to comply with the specific architecture constrains. In this paper we propose a methodology to map Toffoli and NCV circuits such that they are compliant with a given IBM architecture. The proposed approach to accomplish this, is to find a set of low cost mappings for NCV and Toffoli circuits targeting IBM’s architecture constraints. With this approach, the number of CNOT that need to be changed will be reduced, resulting in a smaller circuit regarding the number of gates. To evaluate the proposed approach, Toffoli circuits were mapped to Clifford+T and then realized on IBM’s QX4 architecture. The benchmarks were compared with Toffoli circuits mapped without the methodology proposed in this paper and implemented on IBM QX4 using two different approaches. The results show that the proposed approach resulted in circuits with up to 67% fewer gates compared with Qiskit and with up to 50% fewer gates compared to a Clifford+T mapping algorithm.
Lexical simplification approach to support the accessibility guidelines

Individuals with intellectual disabilities face accessibility barriers due to the fact that text contents found on web pages are often not accessible. In order to prevent this situation from occurring, accessibility guidelines as Easy to Read, WCAG 2.1 and Plain Language guidelines must be followed. This paper presents a study of accessibility guidelines and a system which performs lexical simplification in order to systematize compliance with accessibility guidelines, as well as generating an accessible user interface with the simplified text.