Miracle-dz/UniFAR-FTUs huggingface dataset:
Semantics-Consistent Representation Learning for Remote Sensing Image–Voice Retrieval

With the development of earth observation technology, massive amounts of remote sensing (RS) images are acquired. To find useful information from these images, cross-modal RS image–voice retrieval provides a new insight. This article aims to study the task of RS image–voice retrieval so as to search effective information from massive amounts of RS data. Existing methods for RS image–voice retrieval rely primarily on the pairwise relationship to narrow the heterogeneous semantic gap between images and voices. However, apart from the pairwise relationship included in the data sets, the intramodality and nonpaired intermodality relationships should also be considered simultaneously since the semantic consistency among nonpaired representations plays an important role in the RS image–voice retrieval task. Inspired by this, a semantics-consistent representation learning (SCRL) method is proposed for RS image–voice retrieval. The main novelty is that the proposed method takes the pairwise, intramodality, and nonpaired intermodality relationships into account simultaneously, thereby improving the semantic consistency of the learned representations for the RS image–voice retrieval. The proposed SCRL method consists of two main steps: 1) semantics encoding and 2) SCRL. First, an image encoding network is adopted to extract high-level image features with a transfer learning strategy, and a voice encoding network with dilated convolution is devised to obtain high-level voice features. Second, a consistent representation space is conducted by modeling the three kinds of relationships to narrow the heterogeneous semantic gap and learn semantics-consistent representations across two modalities. Extensive experimental results on three challenging RS image–voice data sets, including Sydney, UCM, and RSICD image–voice data sets, show the effectiveness of the proposed method.
Shift Equivariance for Pixel-based Self-supervised SAR-optical Feature Fusion

—The effective combination of the complementary information provided by the huge amount of unlabeled multi-sensor data (e.g., Synthetic Aperture Radar (SAR), optical images) is a critical topic in remote sensing. Recently, contrastive learning methods have reached remarkable success in obtaining meaningful feature representations from multi-view data. However, these methods only focus on the image-level features, which may not satisfy the requirement for dense prediction tasks such as the land-cover mapping. In this work, we propose a new self-supervised approach to SAR-optical data fusion that can learn disentangled pixel-wise feature representations directly by taking advantage of both multi-view contrastive loss and the bootstrap your own latent (BYOL) methods. Two key contributions of the proposed approach are a multi-view contrastive loss to encode the multimodal images and a shift operation to reconstruct learned representations for each pixel by building the local consistency between different augmented views. In the experimental period, we ﬁrst veriﬁed the effectiveness of multi-view contrastive loss and BYOL in self-supervised learning on SAR-optical fusion using an image-level classiﬁcation task. Then we validated the proposed approach on a land-cover mapping task by training it with unlabeled SAR-optical image pairs. There we used labeled data pairs to evaluate the discriminative capability of learned features in downstream tasks. Results show that the proposed approach extracts features that result in higher accuracy and that reduces the dimension of representations with respect to the image-level contrastive learning method.
Atoms of Confusion in Java

Although writing code seems trivial at times, problems arise when humans misinterpret what the code actually does. One of the potential causes are "atoms of confusion", the smallest possible patterns of misinterpretable source code. Previous research has investigated the impact of atoms of confusion in C code. Results show that developers make significantly more mistakes in code where atoms are present. In this paper, we replicate the work of Gopstein et al. to the Java language. After deriving a set of atoms of confusion for Java, we perform a two-phase experiment with 132 computer science students (i.e., novice developers). Our results show that participants are 2.7 up to 56 times more likely to make mistakes in code snippets affected by 7 out of the 14 studied atoms of confusion, and when faced with both versions of the code snippets, participants perceived the version affected by the atom of confusion to be more confusing and/or less readable in 10 out of the 14 studied atoms of confusion.
How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines

Visual localization is the problem of estimating the camera pose of a given image with respect to a known scene. Visual localization algorithms are a fundamental building block in advanced computer vision applications, including Mixed and Virtual Reality systems. Many algorithms used in practice represent the scene through a Structure-from-Motion (SfM) point cloud and use 2D-3D matches between a query image and the 3D points for camera pose estimation. As recently shown, image details can be accurately recovered from SfM point clouds by translating renderings of the sparse point clouds to images. To address the resulting potential privacy risks for user-generated content, it was recently proposed to lift point clouds to line clouds by replacing 3D points by randomly oriented 3D lines passing through these points. The resulting representation is unintelligible to humans and effectively prevents point cloud-to-image translation. This paper shows that a significant amount of information about the 3D scene geometry is preserved in these line clouds, allowing us to (approximately) recover the 3D point positions and thus to (approximately) recover image content. Our approach is based on the observation that the closest points between lines can yield a good approximation to the original 3D points. Code is available at https://github.com/kunalchelani/Line2Point.
Higher-order Network Analysis Takes Off, Fueled by Classical Ideas and New Data

Higher-order network analysis uses the ideas of hypergraphs, simplicial complexes, multilinear and tensor algebra, and more, to study complex systems. These are by now well established mathematical abstractions. What's new is that the ideas can be tested and refined on the type of large-scale data arising in today's digital world. This research area therefore is making an impact across many applications. Here, we provide a brief history, guide, and survey.
Learning Class-Agnostic Pseudo Mask Generation for Box-Supervised Semantic Segmentation

Recently, several weakly supervised learning methods have been devoted to utilize bounding box supervision for training deep semantic segmentation models. Most existing methods usually leverage the generic proposal generators (\eg, dense CRF and MCG) to produce enhanced segmentation masks for further training segmentation models. These proposal generators, however, are generic and not specifically designed for box-supervised semantic segmentation, thereby leaving some leeway for improving segmentation performance. In this paper, we aim at seeking for a more accurate learning-based class-agnostic pseudo mask generator tailored to box-supervised semantic segmentation. To this end, we resort to a pixel-level annotated auxiliary dataset where the class labels are non-overlapped with those of the box-annotated dataset. For learning pseudo mask generator from the auxiliary dataset, we present a bi-level optimization formulation. In particular, the lower subproblem is used to learn box-supervised semantic segmentation, while the upper subproblem is used to learn an optimal class-agnostic pseudo mask generator. The learned pseudo segmentation mask generator can then be deployed to the box-annotated dataset for improving weakly supervised semantic segmentation. Experiments on PASCAL VOC 2012 dataset show that the learned pseudo mask generator is effective in boosting segmentation performance, and our method can further close the performance gap between box-supervised and fully-supervised models. Our code will be made publicly available at this https URL .
Mechanical Analyses using Discrete Harmonic Expansions of Grain-Based Field Data

A methodology for computing expansion basis functions using discrete harmonic modes is presented. The discrete harmonic modes are determined grain-by-grain for virtual polycrystals for which finite element meshes are available. The expansion weights associated with representing field variables over grain domains are determined by exploiting the orthogonality of the harmonic modes. The methodology is demonstrated with the representation of the axial stress distributions during tensile loading of a polycrystalline sample. An open source code, MechMonics, is available to researchers wishing to use the methodology to analyze data.
Effect of sponge liner on the internal heat transfer coefficients in a simple solar still

In the present study, an attempt has been made to enhance the productivity of a solar still by increasing temperature difference between water and glass, using sponge liners at the inner wall surfaces. Two conventional basin still units were fabricated with the same design parameters. The experimental studies were conducted on the simple solar still with various thicknesses of sponge liners like 3, 5, 7, 10 and 12 mm. To evaluate the convection heat transfer correlation C and n, a thermal model is developed in the present work by using the experimental observations. The regression analysis is used to develop thermal model. Also a mathematical model has been developed to predict the water, glass temperature and mass of distilled output. The values of convection and evaporation heat transfer coefficients obtained in thermal model have been used in the mathematical model. From the experimental and analytical studies, it is concluded that, (i) sponge liner stills works towards increasing the temperature diffe.
ByteSGAN: A Semi-supervised Generative Adversarial Network for Encrypted Traffic Classification of SDN Edge Gateway in Green Communication Network

With the rapid development of Green Communication Network, the types and quantity of network traffic data are accordingly increasing. Network traffic classification become a non-trivial research task in the area of network management and security, which not only help to improve the fine-grained network resource allocation, but also enable policy-driven network management. Meanwhile, the combination of SDN and Edge Computing can leverage both SDN at its global visiability of network-wide and Edge Computing at its low latency and good privacy-preserving. However, capturing large labeled datasets is a cumbersome and time-consuming manual labor. Semi-Supervised learning is an appropriate technique to overcome this problem. With that in mind, we proposed a Generative Adversarial Network (GAN)-based Semi-Supervised Learning Encrypted Traffic Classification method called \emph{ByteSGAN} embedded in SDN Edge Gateway to achieve the goal of traffic classification in a fine-grained manner to further improve network resource utilization. ByteSGAN can only use a small number of labeled traffic samples and a large number of unlabeled samples to achieve a good performance of traffic classification by modifying the structure and loss function of the regular GAN discriminator network in a semi-supervised learning way. Based on public dataset 'ISCX2012 VPN-nonVPN', two experimental results show that the ByteSGAN can efficiently improve the performance of traffic classifier and outperform the other supervised learning method like CNN.
Diagnosis of information scrambling from Hamiltonian evolution under decoherence

We apply a quantum teleportation protocol based on the Hayden-Preskill thought experiment to quantify how scrambling a given quantum evolution is. It has an advantage over the direct measurement of out-oftime ordered correlators when used to diagnose the information scrambling in the presence of decoherence effects stemming from a noisy quantum device. We demonstrate the protocol by applying it to two physical systems: Ising spin chain and SU(2) lattice Yang-Mills theory. To this end, we numerically simulate the time evolution of the two theories in the Hamiltonian formalism. The lattice Yang-Mills theory is implemented with a suitable truncation of Hilbert space on the basis of the Kogut-Susskind formalism. On a two-leg ladder geometry and with the lowest nontrivial spin representations, it can be mapped to a spin chain, which we call Yang-Mills-Ising model and is also directly applicable to future digital quantum simulations. We find that the Yang-Mills-Ising model shows the signal of information scrambling at late times.
Learning Dependencies in Distributed Cloud Applications to Identify and Localize Anomalies

Operation and maintenance of large distributed cloud applications can quickly become unmanageably complex, putting human operators under immense stress when problems occur. Utilizing machine learning for identification and localization of anomalies in such systems supports human experts and enables fast mitigation. However, due to the various interdependencies of system components, anomalies do not only affect their origin but propagate through the distributed system. Taking this into account, we present Arvalus and its variant D-Arvalus, a neural graph transformation method that models system components as nodes and their dependencies and placement as edges to improve the identification and localization of anomalies. Given a series of metric KPIs, our method predicts the most likely system state - either normal or an anomaly class - and performs localization when an anomaly is detected. During our experiments, we simulate a distributed cloud application deployment and synthetically inject anomalies. The evaluation shows the generally good prediction performance of Arvalus and reveals the advantage of D-Arvalus which incorporates information about system component dependencies.
ANDRuspex: Leveraging Graph Representation Learning to Predict Harmful App Installations on Mobile Devices

Android's security model severely limits the capabilities of anti-malware software. Unlike commodity anti-malware solutions on desktop systems, their Android counterparts run as sandboxed applications without root privileges and are limited by Android's permission system. As such, PHAs on Android are usually willingly installed by victims, as they come disguised as useful applications with hidden malicious functionality, and are encountered on mobile app stores as suggestions based on the apps that a user previously installed. Users with similar interests and app installation history are likely to be exposed and to decide to install the same PHA. This observation gives us the opportunity to develop predictive approaches that can warn the user about which PHAs they will encounter and potentially be tempted to install in the near future. These approaches could then be used to complement commodity anti-malware solutions, which are focused on post-fact detection, closing the window of opportunity that existing solutions suffer from. In this paper we develop ANDRuspex, a system based on graph representation learning, allowing us to learn latent relationships between user devices and PHAs and leverage them for prediction. We test ANDRuspex on a real world dataset of PHA installations collected by a security company, and show that our approach achieves very high prediction results (up to 0.994 TPR at 0.0001 FPR), while at the same time outperforming alternative baseline methods. We also demonstrate that ANDRuspex is robust and its runtime performance is acceptable for a real world deployment.
Competition in Signaling

I study a multi-sender signaling game between an uninformed decision maker and two senders with common private information and opposed interests. Senders can misreport information at a cost that is tied to the size of the misrepresentation. The main results concern the amount of information that is transmitted in equilibrium and the language used by senders to convey such information. Fully revealing and pure strategy equilibria exist but are not plausible. I identify suﬃcient conditions under which equilibria always exist, are plausible, and essentially unique, and deliver a complete characterization of such equilibria. As an application, I study the informative value of diﬀerent judicial procedures.
A model-based framework for learning transparent swarm behaviors

This paper proposes a model-based framework to automatically and efficiently design understandable and verifiable behaviors for swarms of robots. The framework is based on the automatic extraction of two distinct models: 1) a neural network model trained to estimate the relationship between the robots' sensor readings and the global performance of the swarm, and 2) a probabilistic state transition model that explicitly models the local state transitions (i.e., transitions in observations from the perspective of a single robot in the swarm) given a policy. The models can be trained from a data set of simulated runs featuring random policies. The first model is used to automatically extract a set of local states that are expected to maximize the global performance. These local states are referred to as desired local states. The second model is used to optimize a stochastic policy so as to increase the probability that the robots in the swarm observe one of the desired local states. Following these steps, the framework proposed in this paper can efficiently lead to effective controllers. This is tested on four case studies, featuring aggregation and foraging tasks. Importantly, thanks to the models, the framework allows us to understand and inspect a swarm's behavior. To this end, we propose verification checks to identify some potential issues that may prevent the swarm from achieving the desired global objective. In addition, we explore how the framework can be used in combination with a"standard"evolutionary robotics strategy (i.e., where performance is measured via simulation), or with online learning.
When is it permissible for artificial intelligence to lie? A trust-based approach

Conversational Artificial Intelligence (AI) used in industry settings can be trained to closely mimic human behaviors, including lying and deception. However, lying is often a necessary part of negotiation. To address this, we develop a normative framework for when it is ethical or unethical for a conversational AI to lie to humans, based on whether there is what we call"invitation of trust"in a particular scenario. Importantly, cultural norms play an important role in determining whether there is invitation of trust across negotiation settings, and thus an AI trained in one culture may not be generalizable to others. Moreover, individuals may have different expectations regarding the invitation of trust and propensity to lie for human vs. AI negotiators, and these expectations may vary across cultures as well. Finally, we outline how a conversational chatbot can be trained to negotiate ethically by applying autoregressive models to large dialog and negotiations datasets.
AfriVEC: Word Embedding Models for African Languages. Case Study of Fon and Nobiin

From Word2Vec to GloVe, word embedding models have played key roles in the current state-of-the-art results achieved in Natural Language Processing. Designed to give significant and unique vectorized representations of words and entities, those models have proven to efficiently extract similarities and establish relationships reflecting semantic and contextual meaning among words and entities. African Languages, representing more than 31% of the worldwide spoken languages, have recently been subject to lots of research. However, to the best of our knowledge, there are currently very few to none word embedding models for those languages words and entities, and none for the languages under study in this paper. After describing Glove, Word2Vec, and Poincar\'e embeddings functionalities, we build Word2Vec and Poincar\'e word embedding models for Fon and Nobiin, which show promising results. We test the applicability of transfer learning between these models as a landmark for African Languages to jointly involve in mitigating the scarcity of their resources, and attempt to provide linguistic and social interpretations of our results. Our main contribution is to arouse more interest in creating word embedding models proper to African Languages, ready for use, and that can significantly improve the performances of Natural Language Processing downstream tasks on them. The official repository and implementation is at https://github.com/bonaventuredossou/afrivec
Deep Learning based 3D Segmentation: A Survey

3D segmentation is a fundamental and challenging problem in computer vision with applications in autonomous driving, robotics, augmented reality and medical image analysis. It has received significant attention from the computer vision, graphics and machine learning communities. Conventional methods for 3D segmentation, based on hand-crafted features and machine learning classifiers, lack generalization ability. Driven by their success in 2D computer vision, deep learning techniques have recently become the tool of choice for 3D segmentation tasks. This has led to an influx of a large number of methods in the literature that have been evaluated on different benchmark datasets. Whereas survey papers on RGB-D and point cloud segmentation exist, there is a lack of an in-depth and recent survey that covers all 3D data modalities and application domains. This paper fills the gap and provides a comprehensive survey of the recent progress made in deep learning based 3D segmentation. It covers over 180 works, analyzes their strengths and limitations and discusses their competitive results on benchmark datasets. The survey provides a summary of the most commonly used pipelines and finally highlights promising research directions for the future.
Graph Metrics for Internet Robustness - A Survey

Research on the robustness of the Internet has gained critical importance in the last decades because more and more individuals, societies and firms rely on this global network infrastructure for communication, knowledge transfer, business processes and e-commerce. In particular, modeling the structure of Internet has inspired several novel graph metrics for assessing important topological robustness features of large complex networks such as the Internet. This survey provides a comparative overview of these metrics, presents their strengths and limitations for analyzing the robustness of the Internet topology, and outlines a conceptual tool set in order to facilitate future analysis and adoption in research and practice.
NeX: Real-time View Synthesis with Neural Basis Expansion

We present NeX, a new approach to novel view synthesis based on enhancements of multiplane image (MPI) that can reproduce next-level view-dependent effects—in real time. Unlike traditional MPI that uses a set of simple RGBα planes, our technique models view-dependent effects by instead parameterizing each pixel as a linear combination of basis functions learned from a neural network. Moreover, we propose a hybrid implicit-explicit modeling strategy that improves upon fine detail and produces state-of-the-art results. Our method is evaluated on benchmark forwardfacing datasets as well as our newly-introduced dataset designed to test the limit of view-dependent modeling with significantly more challenging effects such as the rainbow reflections on a CD. Our method achieves the best overall scores across all major metrics on these datasets with more than 1000× faster rendering time than the state of the art. For real-time demos, visit https://nex-mpi.github. io/