Miracle-dz/UniFAR-FTUs huggingface dataset:
PATSQL: Efficient Synthesis of SQL Queries from Example Tables with Quick Inference of Projected Columns

SQL is one of the most popular tools for data analysis, and it is now used by an increasing number of users without having expertise in databases. Several studies have proposed programming-by-example approaches to help such non-experts to write correct SQL queries. While existing methods support a variety of SQL features such as aggregation and nested query, they suffer a significant increase in computational cost as the scale of example tables increases. In this paper, we propose an efficient algorithm utilizing properties known in relational algebra to synthesize SQL queries from input and output tables. Our key insight is that a projection operator in a program sketch can be lifted above other operators by applying transformation rules in relational algebra, while preserving the semantics of the program. This enables a quick inference of appropriate columns in the projection operator, which is an essential component in synthesis but causes combinatorial explosions in prior work. We also introduce a novel form of constraints and its top-down propagation mechanism for efficient sketch completion. We implemented this algorithm in our tool PATSQL and evaluated it on 226 queries from prior benchmarks and Kaggle's tutorials. As a result, PATSQL solved 68% of the benchmarks and found 89% of the solutions within a second. Our tool is available at https://naist-se.github. io/patsql/.
Counterfactual Variable Control for Robust and Interpretable Question Answering

Deep neural network based question answering (QA) models are neither robust nor explainable in many cases. For example, a multiple-choice QA model, tested without any input of question, is surprisingly "capable" to predict the most of correct options. In this paper, we inspect such spurious "capability" of QA models using causal inference. We find the crux is the shortcut correlation, e.g., unrobust word alignment between passage and options learned by the models. We propose a novel approach called Counterfactual Variable Control (CVC) that explicitly mitigates any shortcut correlation and preserves the comprehensive reasoning for robust QA. Specifically, we leverage multi-branch architecture that allows us to disentangle robust and shortcut correlations in the training process of QA. We then conduct two novel CVC inference methods (on trained models) to capture the effect of comprehensive reasoning as the final prediction. For evaluation, we conduct extensive experiments using two BERT backbones on both multi-choice and span-extraction QA benchmarks. The results show that our CVC achieves high robustness against a variety of adversarial attacks in QA while maintaining good interpretation ability.
ByzShield: An Efficient and Robust System for Distributed Training

Training of large scale models on distributed clusters is a critical component of the machine learning pipeline. However, this training can easily be made to fail if some workers behave in an adversarial (Byzantine) fashion whereby they return arbitrary results to the parameter server (PS). A plethora of existing papers consider a variety of attack models and propose robust aggregation and/or computational redundancy to alleviate the effects of these attacks. In this work we consider an omniscient attack model where the adversary has full knowledge about the gradient computation assignments of the workers and can choose to attack (up to) any q out of n worker nodes to induce maximal damage. Our redundancy-based method ByzShield leverages the properties of bipartite expander graphs for the assignment of tasks to workers; this helps to effectively mitigate the effect of the Byzantine behavior. Specifically, we demonstrate an upper bound on the worst case fraction of corrupted gradients based on the eigenvalues of our constructions which are based on mutually orthogonal Latin squares and Ramanujan graphs. Our numerical experiments indicate over a 36% reduction on average in the fraction of corrupted gradients compared to the state of the art. Likewise, our experiments on training followed by image classification on the CIFAR-10 dataset show that ByzShield has on average a 20% advantage in accuracy under the most sophisticated attacks. ByzShield also tolerates a much larger fraction of adversarial nodes compared to prior work.
Parameterized reinforcement learning for optical system optimization

Engineering a physical system to feature designated characteristics states an inverse design problem, which is often determined by several discrete and continuous parameters. If such a system must feature a particular behavior, the mentioned combination of both, discrete and continuous, parameters results in a challenging optimization problem that requires an extensive search for an optimal system design. However, if the corresponding inverse design problem can be reformulated as a parameterized Markov decision process, reinforcement learning (RL) provides a heuristic framework to solve it. In this work, we use multi-layer thin films as an example of the aforementioned optimization problems and consider three design parameters: Each of the thin film layer’s dielectric material (discrete) and thickness (continuous), as well as the total number of layers (discrete). While recent methods merely determine the optimal thicknesses and—less commonly—the layers’ materials, our approach optimizes the total number of stacked layers as well. In summary, we further develop a Q-learning variant to solve inverse design optimization and thereby outperform human experts and current approaches like needle-point optimization or naive RL. For this purpose, we propose an exponentially transformed reward signal that eases policy search and enables constrained optimization. Moreover, the learned Q-values contain information about the optical properties of multi-layer thin films, which allows us a physical interpretation or what-if analysis and thus enables explainability.
MUSE observations of comet 67P/Churyumov-Gerasimenko

Aims. Observations of comet 67P/Churyumov-Gerasimenko were performed with MUSE at large heliocentric distances post-perihelion between 3 and 7 March 2016. These observations are part of a simultaneous ground-based campaign aimed at providing broad-scale information about comet 67P to complement the ESA/Rosetta mission. Methods. We obtained a total of 38 datacubes over five nights. We took advantage of the integral field unit nature of the instrument to carry out a simultaneous study of the spectrum of 67P’s dust and its spatial distribution in the coma. We also looked for evidence of gas emission in the coma. Results. We produced a high-quality spectrum of the dust coma over the optical range that could be used as a reference for future comet observations with this instrument. The slope of the dust reflectivity is of 10%∕100 nm over the 480–900 nm interval, with a shallower slope towards redder wavelengths. We used the Afρ to quantify the dust production and measure values of 65 ± 4 cm, 75 ± 4 cm, and 82 ± 4 cm in the V, R, and I bands, respectively. We detected several jets in the coma as well as the dust trail. Finally, using a novel method combining spectral and spatial information, we detected the forbidden oxygen emission line at 630 nm. Using this line, we derived a water production rate of 1.5 ± 0.6 × 1026 molec. s−1, assuming all oxygen atoms come from the photo-dissociation of water.
Optimal speed profile on a given road for motion sickness reduction

Motion Sickness (MS) is an issue of most transportation systems. Several countermeasures for such problem in cars are proposed in the literature, but most of them are qualitative, behavioural or involving complex chassis systems. With the growing interest in self-employed vehicles, the issue of MS may be so important that it undermines their benefits in terms of increased productivity; not addressing such issue may limit the users' acceptance reducing the safety and environmental impact of autonomous vehicles. The present study presents a novel approach to combine minimal travel time with minimal Motion Sickness Incidence (MSI) by optimising the speed profile for a given path. Through simulation, a simple vehicle model is used to compare several strategies evaluating which of them are effective and which not. The optimisation task is formulated as a Non-linear Model Predictive Control (NMPC) and a series of optimisation routines are computed along the path; the strategies are implemented within the cost function of the NMPC problem evaluating their performance and if using a numerical MS model is mandatory to get a significant reduction of the MSI. The results show that not all the cost functions are effective, but it is possible to reduce MS without modelling its dynamics; however, the strategy taking into account the current MSI using a MS model outperforms the other cost functions in term of efficacy and efficiency. Such a quantitative approach can be used during motion planning in autonomous vehicles, to suggest an optimal speed in human-driven vehicles and to improve the comfort for bus-line services or even trains.
Graph-Regularized Non-Negative Tensor-Ring Decomposition for Multiway Representation Learning

Tensor-ring (TR) decomposition is a powerful tool for exploiting the low-rank property of multiway data and has been demonstrated great potential in a variety of important applications. In this article, non-negative TR (NTR) decomposition and graph-regularized NTR (GNTR) decomposition are proposed. The former equips TR decomposition with the ability to learn the parts-based representation by imposing non-negativity on the core tensors, and the latter additionally introduces a graph regularization to the NTR model to capture manifold geometry information from tensor data. Both of the proposed models extend TR decomposition and can be served as powerful representation learning tools for non-negative multiway data. The optimization algorithms based on an accelerated proximal gradient are derived for NTR and GNTR. We also empirically justified that the proposed methods can provide more interpretable and physically meaningful representations. For example, they are able to extract parts-based components with meaningful color and line patterns from objects. Extensive experimental results demonstrated that the proposed methods have better performance than state-of-the-art tensor-based methods in clustering and classification tasks.
SLIP: Learning to Predict in Unknown Dynamical Systems with Long-Term Memory

We present an efficient and practical (polynomial time) algorithm for online prediction in unknown and partially observed linear dynamical systems (LDS) under stochastic noise. When the system parameters are known, the optimal linear predictor is the Kalman filter. However, the performance of existing predictive models is poor in important classes of LDS that are only marginally stable and exhibit long-term forecast memory. We tackle this problem through bounding the generalized Kolmogorov width of the Kalman filter model by spectral methods and conducting tight convex relaxation. We provide a finite-sample analysis, showing that our algorithm competes with Kalman filter in hindsight with only logarithmic regret. Our regret analysis relies on Mendelson's small-ball method, providing sharp error bounds without concentration, boundedness, or exponential forgetting assumptions. We also give experimental results demonstrating that our algorithm outperforms state-of-the-art methods. Our theoretical and experimental results shed light on the conditions required for efficient probably approximately correct (PAC) learning of the Kalman filter from partially observed data.
TurboTransformers: an efficient GPU serving system for transformer models

The transformer is the most critical algorithm innovation of the Nature Language Processing (NLP) field in recent years. Unlike the Recurrent Neural Network (RNN) models, transformers are able to process on dimensions of sequence lengths in parallel, therefore leads to better accuracy on long sequences. However, efficient deployments of them for online services in data centers equipped with GPUs are not easy. First, more computation introduced by transformer structures makes it more challenging to meet the latency and throughput constraints of serving. Second, NLP tasks take in sentences of variable length. The variability of input dimensions brings a severe problem to efficient memory management and serving optimization. To solve the above challenges, this paper designed a transformer serving system called TurboTransformers, which consists of a computing runtime and a serving framework. Three innovative features make it stand out from other similar works. An efficient parallel algorithm is proposed for GPU-based batch reduction operations, like Softmax and LayerNorm, which are major hot spots besides BLAS routines. A memory allocation algorithm, which better balances the memory footprint and allocation/free efficiency, is designed for variable-length input situations. A serving framework equipped with a new batch scheduler using dynamic programming achieves the optimal throughput on variable-length requests. The system can achieve the state-of-the-art transformer model serving performance on GPU platforms and can be seamlessly integrated into your PyTorch code with a few lines of code.
Structured Self-Attention Weights Encodes Semantics in Sentiment Analysis

Neural attention, especially the self-attention made popular by the Transformer, has become the workhorse of state-of-the-art natural language processing (NLP) models. Very recent work suggests that the self-attention in the Transformer encodes syntactic information; Here, we show that self-attention scores encode semantics by considering sentiment analysis tasks. In contrast to gradient-based feature attribution methods, we propose a simple and effective Layer-wise Attention Tracing (LAT) method to analyze structured attention weights. We apply our method to Transformer models trained on two tasks that have surface dissimilarities, but share common semantics—sentiment analysis of movie reviews and time-series valence prediction in life story narratives. Across both tasks, words with high aggregated attention weights were rich in emotional semantics, as quantitatively validated by an emotion lexicon labeled by human annotators. Our results show that structured attention weights encode rich semantics in sentiment analysis, and match human interpretations of semantics.
Discriminative Sounding Objects Localization via Self-supervised Audiovisual Matching

Discriminatively localizing sounding objects in cocktail-party, i.e., mixed sound scenes, is commonplace for humans, but still challenging for machines. In this paper, we propose a two-stage learning framework to perform self-supervised class-aware sounding object localization. First, we propose to learn robust object representations by aggregating the candidate sound localization results in the single source scenes. Then, class-aware object localization maps are generated in the cocktail-party scenarios by referring the pre-learned object knowledge, and the sounding objects are accordingly selected by matching audio and visual object category distributions, where the audiovisual consistency is viewed as the self-supervised signal. Experimental results in both realistic and synthesized cocktail-party videos demonstrate that our model is superior in filtering out silent objects and pointing out the location of sounding objects of different classes. Code is available at this https URL.
How Important is the Train-Validation Split in Meta-Learning?

Meta-learning aims to perform fast adaptation on a new task through learning a "prior" from multiple existing tasks. A common practice in meta-learning is to perform a train-validation split where the prior adapts to the task on one split of the data, and the resulting predictor is evaluated on another split. Despite its prevalence, the importance of the train-validation split is not well understood either in theory or in practice, particularly in comparison to the more direct non-splitting method, which uses all the per-task data for both training and evaluation. We provide a detailed theoretical study on whether and when the train-validation split is helpful on the linear centroid meta-learning problem, in the asymptotic setting where the number of tasks goes to infinity. We show that the splitting method converges to the optimal prior as expected, whereas the non-splitting method does not in general without structural assumptions on the data. In contrast, if the data are generated from linear models (the realizable regime), we show that both the splitting and non-splitting methods converge to the optimal prior. Further, perhaps surprisingly, our main result shows that the non-splitting method achieves a strictly better asymptotic excess risk under this data distribution, even when the regularization parameter and split ratio are optimally tuned for both methods. Our results highlight that data splitting may not always be preferable, especially when the data is realizable by the model. We validate our theories by experimentally showing that the non-splitting method can indeed outperform the splitting method, on both simulations and real meta-learning tasks.
HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Several recent work on speech synthesis have employed generative adversarial networks (GANs) to produce raw waveforms. Although such methods improve the sampling efficiency and memory usage, their sample quality has not yet reached that of autoregressive and flow-based generative models. In this work, we propose HiFi-GAN, which achieves both efficient and high-fidelity speech synthesis. As speech audio consists of sinusoidal signals with various periods, we demonstrate that modeling periodic patterns of an audio is crucial for enhancing sample quality. A subjective human evaluation (mean opinion score, MOS) of a single speaker dataset indicates that our proposed method demonstrates similarity to human quality while generating 22.05 kHz high-fidelity audio 167.9 times faster than real-time on a single V100 GPU. We further show the generality of HiFi-GAN to the mel-spectrogram inversion of unseen speakers and end-to-end speech synthesis. Finally, a small footprint version of HiFi-GAN generates samples 13.4 times faster than real-time on CPU with comparable quality to an autoregressive counterpart.
Constraining dark energy using the cross correlations of weak lensing with post-reionization probes of neutral hydrogen

We investigate the prospects of detecting the cross correlation of CMBR weak-lensing convergence field with the large scale tracers of the underlying dark matter distribution in the post-reionization epoch. The cross-correlation is then used to make error projections for dark energy equation of state (EoS)for models with a time evolving dark energy. We study the cross-correlation angular power spectrum of the weak-lensing field with the Lyman-α forest and the redshifted HI 21 cm signal from the post reionization epoch. The angular power spectra is expressed as a line of sight average over the tomographic slices. We find that on using multiple 400 hrs observation with an extended uGMRT like instrument or with a BOSS like survey with quasar (QSO) density of 16 deg−2 the cross-correlation with weak-lensing convergence field covering half the sky can be detected at a very high SNR (> 20). The cross-correlation of weak-lensing with Lyman-α forest allows the 1−σ errors on the dark energy EoS parameters for different parametrizations to be constrained at a level of precision comparable to combined Planck+SNIa+BAO+HST projections. The 21-cm weak-lensing cross-correlation is found to provide strong constraints on the present value of the dark energy EoS parameter at 4% for the 7CPL model. The constraints on wa is comparable (~12%) for models other than the 7CPL model. We also find that the CPL parametrization may not be the best constrained parametrization for dark energy evolution. The cross-correlation of CMBR weak-lensing with the post-reionization probes of neutral hydrogen thus holds the potential to give us valuable understanding about the nature of evolving dark energy.
Non-Archimedean TSI Polish groups and their potential Borel complexity spectrum

In this paper, the Hjorth analysis of Polish group actions is applied to the special case of a non-Archimedean TSI Polish group acting continuously on a Polish spaces. We give results on the potential Borel complexity spectrum of such groups, and define orbit equivalence relations that are universal for each Borel complexity class. We also study classifiability by non-Archimedean TSI Polish groups, and give a dynamical obstruction to such classifications, which we apply to show that an equivalence relation of Clemens and Coskey is not classifiable by non-Archimedean TSI Polish groups. Finally, we characterize the equivalence relations that are both Borel-reducible to $=^+$ and classifiable by non-Archimedean TSI Polish groups, extending a result of Ding and Gao.
Studies on DNA Polymorphism of Main Restorer Lines of Hybrid Rice in China by RAPD Markers

Thirty-five main restorer lines of hybrid rice in China were analyzed by RAPD with 258 random primers. Forty-two primers detected polymorphism among lines tested, thirteen primers were selected from them. These thirteen primers could detect 93 reproducible polymorphism fragments, by which the above restorer lines were distinguished individually. They were, therefore, recommended as candidates for the identification among restorer lines of hybrid rice. The results showed from cluster analysis: Thirty-five rice restorer lines were divided into four groups near GS=0.57. The genetic similarly coefficient of most restorer lines was over 0.80. It was indicated that rice restore lines have abundant resource in China, but the genetic diversity was small and the genetic background was vulnerable among them, which restricted the application of rice heterosis in a large extent.
Biochemical screening of pure honey and its antibacterial activity on some bacterial isolates compared with a common antibiotics

The biochemical components and antibacterial efficacy of pure honey produced by honeybee(Apis mellifera) was investigated. Biochemical analysis revealed the presence of reducing sugar, saponins, glycosides, alkaloids and flavonoids and absence of phenols and tannins. Antibacterial activity of pure honey on some medically important bacteria including Staphylococcusaureus, Salmonella typhi, Escherichia coli, Klebsiella pneumonia and Pseudomonas aeruginosa was determined using the agar well diffusion method. The result shows that pure honey exhibit strong antibacterial activity producing zones of inhibition against the tested bacteria.Also, honey sample used in this study showed antibacterial activity than the commercially available antibiotics. Our findings shows that honey,apart from their roles as food and supplements, could be suitable for the treatment of various infection caused by bacteria hence should be used as a preventive and curative measure to common diseases related to the test organisms.