Miracle-dz/UniFAR-FTUs huggingface dataset:
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units

In this paper we present the first model for directly synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or source of supervision. Instead, we connect the image captioning module and the speech synthesis module with a set of discrete, sub-word speech units that are discovered with a self-supervised visual grounding task. We conduct experiments on the Flickr8k spoken caption dataset in addition to a novel corpus of spoken audio captions collected for the popular MSCOCO dataset, demonstrating that our generated captions also capture diverse visual semantics of the images they describe. We investigate several different intermediate speech representations, and empirically find that the representation must satisfy several important properties to serve as drop-in replacements for text.
An analysis of scatter characteristics in x-ray CT spectral correction

X-ray scatter remains a major physics challenge in volumetric computed tomography (CT), whose physical and statistical behaviors have been commonly leveraged in order to eliminate its impact on CT image quality. In this work, we conduct an in-depth derivation of how the scatter distribution and scatter to primary ratio (SPR) will change during the spectral correction, leading to an interesting finding on the property of scatter. Such a characterization of scatter’s behavior provides an analytic approach of compensating for the SPR as well as approximating the change of scatter distribution after spectral correction, even though both of them might be significantly distorted as the linearization mapping function in spectral correction could vary a lot from one detector pixel to another. We conduct an evaluation of SPR compensations (SPRCs) on a Catphan phantom and an anthropomorphic chest phantom to validate the characteristics of scatter. In addition, this scatter property is also directly adopted into CT imaging using a spectral modulator with flying focal spot technology (SMFFS) as an example to demonstrate its potential in practical applications. For cone-beam CT (CBCT) scans at both 80 and 120 kVp, CT images with accurate CT numbers can be achieved after spectral correction followed by the appropriate SPRC based on our presented scatter property. In the case of the SMFFS based CBCT scan of the Catphan phantom at 120 kVp, after a scatter correction using an analytic algorithm derived from the scatter property, CT image quality was significantly improved, with the averaged root mean square error reduced from 297.9 to 6.5 Hounsfield units.
Introducing Orthogonal Constraint in Structural Probes

With the recent success of pre-trained models in NLP, a significant focus was put on interpreting their representations. One of the most prominent approaches is structural probing (Hewitt and Manning, 2019), where a linear projection of word embeddings is performed in order to approximate the topology of dependency structures. In this work, we introduce a new type of structural probing, where the linear projection is decomposed into 1. iso-morphic space rotation; 2. linear scaling that identifies and scales the most relevant dimensions. In addition to syntactic dependency, we evaluate our method on two novel tasks (lexical hypernymy and position in a sentence). We jointly train the probes for multiple tasks and experimentally show that lexical and syntactic information is separated in the representations. Moreover, the orthogonal constraint makes the Structural Probes less vulnerable to memorization.
Jones matrix holography with metasurfaces

A new mathematical framework uncovers overlooked possibilities for polarization-dependent metasurfaces. We propose a new class of computer-generated holograms whose far-fields have designer-specified polarization response. We dub these Jones matrix holograms. We provide a simple procedure for their implementation using form-birefringent metasurfaces. Jones matrix holography generalizes a wide body of past work with a consistent mathematical framework, particularly in the field of metasurfaces, and suggests previously unrealized devices, examples of which are demonstrated here. In particular, we demonstrate holograms whose far-fields implement parallel polarization analysis and custom waveplate-like behavior.
Example-Driven User Intent Discovery: Empowering Users to Cross the SQL Barrier Through Query by Example

Traditional data systems require specialized technical skills where users need to understand the data organization and write precise queries to access data. Therefore, novice users who lack technical expertise face hurdles in perusing and analyzing data. Existing tools assist in formulating queries through keyword search, query recommendation, and query auto-completion, but still require some technical expertise. An alternative method for accessing data is Query by Example (QBE), where users express their data exploration intent simply by providing examples of their intended data. We study a state-of-the-art QBE system called SQuID, and contrast it with traditional SQL querying. Our comparative user studies demonstrate that users with varying expertise are significantly more effective and efficient with SQuID than SQL. We find that SQuID eliminates the barriers in studying the database schema, formalizing task semantics, and writing syntactically correct SQL queries, and thus, substantially alleviates the need for technical expertise in data exploration.
Edge Computing Enabled by Unmanned Autonomous Vehicles

Pervasive applications are revolutionizing the perception that users have towards the environment. Indeed, pervasive applications perform resource intensive computations over large amounts of stream sensor data collected from multiple sources. This allows applications to provide richer and deep insights into the natural characteristics that govern everything that surrounds us. A key limitation of these applications is that they have high energy footprints, which in turn hampers the quality of experience of users. While cloud and edge computing solutions can be applied to alleviate the problem, these solutions are hard to adopt in existing architecture and far from become ubiquitous. Fortunately, cloudlets are becoming portable enough, such that they can be transported and integrated into any environment easily and dynamically. In this article, we investigate how cloudlets can be transported by unmanned autonomous vehicles (UAV)s to provide computation support on the edge. Based on our study, we develop GEESE, a novel UAVbased system that enables the dynamic deployment of an edge computing infrastructure through the cooperation of multiple UAVs carrying cloudlets. By using GEESE, we conduct rigorous experiments to analyze the effort to deliver cloudlets using aerial, ground, and underwater UAVs. Our results indicate that UAVs can work in a cooperative manner to enable edge computing in the wild.
Nonreversible MCMC from conditional invertible transforms: a complete recipe with convergence guarantees

Markov Chain Monte Carlo (MCMC) is a class of algorithms to sample complex and high-dimensional probability distributions. The Metropolis-Hastings (MH) algorithm, the workhorse of MCMC, provides a simple recipe to construct reversible Markov kernels. Reversibility is a tractable property which implies a less tractable but essential property here, invariance. Reversibility is however not necessarily desirable when considering performance. This has prompted recent interest in designing kernels breaking this property. At the same time, an active stream of research has focused on the design of novel versions of the MH kernel, some nonreversible, relying on the use of complex invertible deterministic transforms. While standard implementations of the MH kernel are well understood, aforementioned developments have not received the same systematic treatment to ensure their validity. This paper fills the gap by developing general tools to ensure that a class of nonreversible Markov kernels, possibly relying on complex transforms, has the desired invariance property and lead to convergent algorithms. This leads to a set of simple and practically verifiable conditions.
Joint Verification and Reranking for Open Fact Checking Over Tables

Structured information is an important knowledge source for automatic verification of factual claims. Nevertheless, the majority of existing research into this task has focused on textual data, and the few recent inquiries into structured data have been for the closed-domain setting where appropriate evidence for each claim is assumed to have already been retrieved. In this paper, we investigate verification over structured data in the open-domain setting, introducing a joint reranking-and-verification model which fuses evidence documents in the verification component. Our open-domain model achieves performance comparable to the closed-domain state-of-the-art on the TabFact dataset, and demonstrates performance gains from the inclusion of multiple tables as well as a significant improvement over a heuristic retrieval baseline.
Fermion Sampling: A Robust Quantum Computational Advantage Scheme Using Fermionic Linear Optics and Magic Input States

Micha l Oszmaniec, Ninnat Dangniam, Mauro E.S. Morales, and Zoltán Zimborás 4, 5 Center for Theoretical Physics, Polish Academy of Sciences, Al. Lotników 32/46, 02-668 Warszawa, Poland∗ Centre for Quantum Computation and Communication Technology, Centre for Quantum Software and Information, University of Technology Sydney, NSW 2007, Australia Wigner Research Centre for Physics, H-1121, Budapest, Hungary† MTA-BME Lendület Quantum Information Theory Research Group Budapest, Hungary Mathematical Institute, Budapest University of Technology and Economics, H-1111, Budapest, Hungary
Scaling Replicated State Machines with Compartmentalization

State machine replication protocols, like MultiPaxos and Raft, are a critical component of many distributed systems and databases. However, these protocols offer relatively low throughput due to several bottlenecked components. Numerous existing protocols fix different bottlenecks in isolation but fall short of a complete solution. When you fix one bottleneck, another arises. In this paper, we introduce compartmentalization, the first comprehensive technique to eliminate state machine replication bottlenecks. Compartmentalization involves decoupling individual bottlenecks into distinct components and scaling these components independently. Compartmentalization has two key strengths. First, compartmentalization leads to strong performance. In this paper, we demonstrate how to compartmentalize MultiPaxos to increase its throughput by 6× on a write-only workload and 16× on a mixed read-write workload. Unlike other approaches, we achieve this performance without the need for specialized hardware. Second, compartmentalization is a technique, not a protocol. Industry practitioners can apply compartmentalization to their protocols incrementally without having to adopt a completely new protocol.
UCCA's Foundational Layer: Annotation Guidelines v2.1

This is the annotation manual for Universal Conceptual Cognitive Annotation (UCCA; Abend and Rappoport, 2013), specifically the Foundational Layer. UCCA is a graph-based semantic annotation scheme based on typological linguistic principles. It has been applied to several languages; for ease of exposition these guidelines give examples mainly in English. New annotators may wish to start with the tutorial on the UCCA framework (Abend et al., 2020). Further resources are available at the project homepage: https://universalconceptualcognitiveannotation.github. io
Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients

As reinforcement learning techniques are increasingly applied to real-world decision problems, attention has turned to how these algorithms use potentially sensitive information. We consider the task of training a policy that maximizes reward while minimizing disclosure of certain sensitive state variables through the actions. We give examples of how this setting covers real-world problems in privacy for sequential decision-making. We solve this problem in the policy gradients framework by introducing a regularizer based on the mutual information (MI) between the sensitive state and the actions at a given timestep. We develop a model-based stochastic gradient estimator for optimization of privacy-constrained policies. We also discuss an alternative MI regularizer that serves as an upper bound to our main MI regularizer and can be optimized in a model-free setting. We contrast previous work in differentially-private RL to our mutual-information formulation of information disclosure. Experimental results show that our training method results in policies which hide the sensitive state.
Explanations of Machine Learning predictions: a mandatory step for its application to Operational Processes

In the global economy, credit companies play a central role in economic development, through their activity as money lenders. This important task comes with some drawbacks, mainly the risk of the debtors not being able to repay the provided credit. Therefore, Credit Risk Modelling (CRM), namely the evaluation of the probability that a debtor will not repay the due amount, plays a paramount role. Statistical approaches have been successfully exploited since long, becoming the most used methods for CRM. Recently, also machine and deep learning techniques have been applied to the CRM task, showing an important increase in prediction quality and performances. However, such techniques usually do not provide reliable explanations for the scores they come up with. As a consequence, many machine and deep learning techniques fail to comply with western countries' regulations such as, for example, GDPR. In this paper we suggest to use LIME (Local Interpretable Model-agnostic Explanations) technique to tackle the explainability problem in this field, we show its employment on a real credit-risk dataset and eventually discuss its soundness and the necessary improvements to guarantee its adoption and compliance with the task.
Multiple Structural Priors Guided Self Attention Network for Language Understanding

Self attention networks (SANs) have been widely utilized in recent NLP studies. Unlike CNNs or RNNs, standard SANs are usually position-independent, and thus are incapable of capturing the structural priors between sequences of words. Existing studies commonly apply one single mask strategy on SANs for incorporating structural priors while failing at modeling more abundant structural information of texts. In this paper, we aim at introducing multiple types of structural priors into SAN models, proposing the Multiple Structural Priors Guided Self Attention Network (MS-SAN) that transforms different structural priors into different attention heads by using a novel multi-mask based multi-head attention mechanism. In particular, we integrate two categories of structural priors, including the sequential order and the relative position of words. For the purpose of capturing the latent hierarchical structure of the texts, we extract these information not only from the word contexts but also from the dependency syntax trees. Experimental results on two tasks show that MS-SAN achieves significant improvements against other strong baselines.
EC-GAN: Low-Sample Classification using Semi-Supervised Algorithms and GANs

Semi-supervised learning has been gaining attention as it allows for performing image analysis tasks such as classification with limited labeled data. Some popular algorithms using Generative Adversarial Networks (GANs) for semi-supervised classification share a single architecture for classification and discrimination. However, this may require a model to converge to a separate data distribution for each task, which may reduce overall performance. While progress in semi-supervised learning has been made, less addressed are small-scale, fully-supervised tasks where even unlabeled data is unavailable and unattainable. We therefore, propose a novel GAN model namely External Classifier GAN (EC-GAN), that utilizes GANs and semi-supervised algorithms to improve classification in fully-supervised regimes. Our method leverages a GAN to generate artificial data used to supplement supervised classification. More specifically, we attach an external classifier, hence the name EC-GAN, to the GAN’s generator, as opposed to sharing an architecture with the discriminator. Our experiments demonstrate that EC-GAN's performance is comparable to the shared architecture method, far superior to the standard data augmentation and regularization-based approach, and effective on a small, realistic dataset.
Learning and teaching Einstein's Theory of Special Relativity: state of the art

This work analyzes the difficulties in learning and teaching Einstein's theory of special relativity. An extensive bibliographic review has been performed, considering articles published in the most relevant journals on science education, which were selected taking into account the following impact factors: JCR, SJR, IN-RECS and ICDS. The typical thinking of students and teachers is discussed pointing out that, occasionally, it does not befit the proper scientific perspective. Different educational proposals are examined and particular didactic implications are inferred. The conclusions of this inquiry constitute the basis of a proposal that relies on a Minkowskian geometrical formulation for teaching special relativity in upper secondary education.
SASSI — Super-Pixelated Adaptive Spatio-Spectral Imaging

We introduce a novel video-rate hyperspectral imager with high spatial, temporal and spectral resolutions. Our key hypothesis is that spectral profiles of pixels within each super-pixel tend to be similar. Hence, a scene-adaptive spatial sampling of a hyperspectral scene, guided by its super-pixel segmented image, is capable of obtaining high-quality reconstructions. To achieve this, we acquire an RGB image of the scene, compute its super-pixels, from which we generate a spatial mask of locations where we measure high-resolution spectrum. The hyperspectral image is subsequently estimated by fusing the RGB image and the spectral measurements using a learnable guided filtering approach. Due to low computational complexity of the superpixel estimation step, our setup can capture hyperspectral images of the scenes with little overhead over traditional snapshot hyperspectral cameras, but with significantly higher spatial and spectral resolutions. We validate the proposed technique with extensive simulations as well as a lab prototype that measures hyperspectral video at a spatial resolution of <inline-formula><tex-math notation="LaTeX">$600 \times 900$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>600</mml:mn><mml:mo>×</mml:mo><mml:mn>900</mml:mn></mml:mrow></mml:math><inline-graphic xlink:href="saragadam-ieq1-3075228.gif"/></alternatives></inline-formula> pixels, at a spectral resolution of 10 nm over visible wavebands, and achieving a frame rate at 18fps.
Crossover‐SGD: A gossip‐based communication in distributed deep learning for alleviating large mini‐batch problem and enhancing scalability

Distributed deep learning is an effective way to reduce the training time for large datasets as well as complex models. However, the limited scalability caused by network‐overheads makes it difficult to synchronize the parameters of all workers and gossip‐based methods that demonstrate stable scalability regardless of the number of workers have been proposed. However, to use gossip‐based methods in general cases, the validation accuracy for a large mini‐batch needs to be verified. For this, we first empirically study the characteristics of gossip methods in a large mini‐batch problem and observe that gossip methods preserve higher validation accuracy than AllReduce‐SGD (stochastic gradient descent) when the number of batch sizes is increased, and the number of workers is fixed. However, the delayed parameter propagation of the gossip‐based models decreases validation accuracy in large node scales. To cope with this problem, we propose Crossover‐SGD that alleviates the delay propagation of weight parameters via segment‐wise communication and random network topology with fair peer selection. We also adapt hierarchical communication to limit the number of workers in gossip‐based communication methods. To validate the effectiveness of our method, we conduct empirical experiments and observe that our Crossover‐SGD shows higher node scalability than stochastic gradient push.
Machine learning modeling of materials with a group-subgroup structure

A cornerstone of materials science is Landau’s theory of continuous phase transitions. Crystal structures connected by Landau-type transitions are mathematically related through groupsubgroup relationships. In this study, we introduce “group-subgroup learning” and show including small unit cell phases of materials in the training set to decrease out-of-sample errors for modeling larger phases. The proposed approach is generic and is independent of the ML formalism, descriptors, or datasets; and extendable to other symmetry abstractions such as spin-, valency-, or charge order. Since available materials datasets are heterogeneous with too few examples for realizing the group-subgroup structure, we present the “FriezeRMQ1D” dataset of 8393 Q1D organometallic materials uniformly distributed across seven frieze groups and provide a proof-of-the-concept. For these materials, we report < 3% error with 25% training with the Faber–Christensen–Huang–Lilienfeld descriptor and compare its performance with a fingerprint representation that encodes materials composition as well as crystallographic Wyckoff positions.
On clean, weakly clean and feebly clean commutative group rings

A ring R is said to be clean if each element of R can be written as the sum of a unit and an idempotent. R is said to be weakly clean if each element of R is either a sum or a difference of a unit and an idempotent, and R is said to be feebly clean if every element r can be written as r = u+ e1− e2, where u is a unit and e1, e2 are orthogonal idempotents. Clearly clean rings are weakly clean rings and both of them are feebly clean. In a recent article (J. Algebra Appl. 17 (2018), 1850111(5 pages)), McGoven characterized when the group ring Z(p)[Cq] is weakly clean and feebly clean, where p, q are distinct primes. In this paper, we consider a more general setting. Let K be an algebraic number field, OK its ring of integers, p ⊂ O a nonzero prime ideal, and Op the localization of O at p. We investigate when the group ring Op[G] is weakly clean and feebly clean, where G is a finite abelian group, and establish an explicit characterization for such a group ring to be weakly clean and feebly clean for the case when K = Q(ζn) is a cyclotomic field or K = Q( √ d) is a quadratic field.
SemGloVe: Semantic Co-Occurrences for GloVe From BERT

GloVe learns word embeddings by leveraging statistical information from word co-occurrence matrices. However, word pairs in the matrices are extracted from a predefined local context window, which might lead to limited word pairs and potentially semantic irrelevant word pairs. In this paper, we propose SemGloVe, which distills semantic co-occurrences from BERT into static GloVe word embeddings. Particularly, we propose two models to extract co-occurrence statistics based on either the masked language model or the multi-head attention weights of BERT. Our methods can extract word pairs limited by the local window assumption, and can define the co-occurrence weights by directly considering the semantic distance between word pairs. Experiments on several word similarity datasets and external tasks show that SemGloVe can outperform GloVe.