Miracle-dz/UniFAR-FTUs huggingface dataset:
Controlling Factors and Formation Models of Organic Matter Accumulation for the Upper Permian Dalong Formation Black Shale in the Lower Yangtze Region, South China: Constraints from Geochemical Evidence

The black shale in the upper Permian Dalong Formation is considered as an excellent source rock in the Lower Yangtze region. However, mechanisms of organic matter (OM) accumulation in such a setting are scarcely understood. Here, the characteristics of total organic carbon (TOC) and elemental geochemistry of 33 rock samples from GD1 well are systematically investigated to characterize the paleoenvironmental conditions and OM accumulation mechanisms. Results show that the lower and middle parts of Dalong Formation (section A) display high TOC contents ranging from 1.19 to 6.45% (average 3.19%), whereas the upper part (section B) exhibits medium TOC contents varying from 1.18 to 4.90% (average 2.09%). These data also elucidate that the target shales were deposited in a complex paleoenvironment with moderate to strong water-mass restriction that was characterized by warm and semiarid–semihumid paleoclimate, high biotic productivity, fluctuating plaeoredox conditions, and a relatively high sedimentary rate. Compared to the organic-rich shales from section A mainly developed under an anoxic condition, shales from section B formed in an oxic-to-dysoxic water environment exhibited a comparatively higher sedimentary rate. Moreover, among all these factors that might affect OM accumulation, the paleoredox conditions appear to be the dominant controlling factors for section A, whereas the biotic productivity, paleoredox conditions, and sedimentary rate are the main controlling factors for section B. Finally, two formation models for OM accumulation in Dalong Formation shale in the Lower Yangtze region are proposed. The “preservation model” for OM accumulation in section A emphasizes that the reducing deep-water environment, which was mainly caused by the regional sea level rise, is favorable for OM preservation. The “integrated model” for OM accumulation in section B stresses that greater biotic productivity intensifies respiratory oxygen consumption in a water column and a higher sedimentary rate can greatly shorten OM exposure time for respiration by oxygen, both of which cause OM accumulation under an oxidizing water environment. These findings also add to our knowledge that despite the oxygenated water environment during shale deposition, TOC contents are not necessarily lower.
Self-Organizing Teams in Online Work Settings

IOANNA LYKOURENTZOU, Utrecht University, The Netherlands FEDERICA LUCIA VINELLA, Utrecht University, The Netherlands FAEZ AHMED,Massachusetts Institute of Technology, USA COSTAS PAPASTATHIS, University of Peloponnese, Greece KONSTANTINOS PAPANGELIS, Rochester Institute of Technology, USA VASSILIS-JAVED KHAN, Eindhoven University of Technology, The Netherlands JUDITH MASTHOFF, Utrecht University, The Netherlands
A Momentum-Assisted Single-Timescale Stochastic Approximation Algorithm for Bilevel Optimization

This paper proposes a new algorithm – the Momentum-assisted Single-timescale Stochastic Approximation ( MSTSA ) – for tackling unconstrained bilevel optimization problems. We focus on bilevel problems where the lower level subproblem is strongly-convex. Unlike prior works which rely on two timescale or double loop techniques that track the optimal solution to the lower level subproblem, we design a stochastic momentum assisted gradient estimator for the upper level subproblem’s updates. The latter allows us to gradually control the error in stochastic gradient updates due to inaccurate solution to the lower level subproblem. We show that if the upper objective function is smooth but possibly non-convex (resp. strongly-convex), MSTSA requires O ( (cid:15) − 2 ) (resp. O ( (cid:15) − 1 )) iterations (each using constant samples) to ﬁnd an (cid:15) -stationary (resp. (cid:15) -optimal) solution. This achieves the best-known guarantees for stochastic bilevel problems. We validate our theoretical results by showing the eﬃciency of the MSTSA algorithm on hyperparameter optimization and data hyper-cleaning problems.
Translational Equivariance in Kernelizable Attention

While Transformer architectures have show remarkable success, they are bound to the computation of all pairwise interactions of input element and thus suffer from limited scalability. Recent work has been successful by avoiding the computation of the complete attention matrix, yet leads to problems down the line. The absence of an explicit attention matrix makes the inclusion of inductive biases relying on relative interactions between elements more challenging. An extremely powerful inductive bias is translational equivariance, which has been conjectured to be responsible for much of the success of Convolutional Neural Networks on image recognition tasks. In this work we show how translational equivariance can be implemented in efficient Transformers based on kernelizable attention - Performers. Our experiments highlight that the devised approach significantly improves robustness of Performers to shifts of input images compared to their naive application. This represents an important step on the path of replacing Convolutional Neural Networks with more expressive Transformer architectures and will help to improve sample efficiency and robustness in this realm.
Generalized Social Marginal Welfare Weights Imply Inconsistent Comparisons of Tax Policies

This paper concerns Saez and Stantcheva's (2016) generalized social marginal welfare weights (GSMWW), which are used to aggregate losses and gains due to the tax system, while incorporating non-utilitarian ethical considerations. That approach evaluates local changes in tax policy without appealing to a global social objective. However, I argue that local comparisons between different tax systems implicitly entail global comparisons. Moreover, whenever welfare weights are not of a utilitarian kind, these implied global comparisons are inconsistent. Part of the motivation for the GSMWW approach is that it provides a way to incorporate broader ethical judgements into the evaluation of the tax system while preserving the Pareto principle. I suggest that the problems with the approach ought to spark a reconsideration of Pareto if one wants to represent broader values in formal policy analysis.
On the commuting probability for subgroups of a finite group

Let $K$ be a subgroup of a finite group $G$. The probability that an element of $G$ commutes with an element of $K$ is denoted by $Pr(K,G)$. Assume that $Pr(K,G)\geq \epsilon$ for some fixed $\epsilon >0$. We show that there is a normal subgroup $T\leq G$ and a subgroup $B\leq K$ such that the indices $[G:T]$ and $[K:B]$ and the order of the commutator subgroup $[T,B]$ are $\epsilon$-bounded. This extends the well-known theorem, due to P. M. Neumann, that covers the case where $K=G$. We deduce a number of corollaries of this result. A typical application is that if $K$ is the generalized Fitting subgroup $F^{*}(G)$ then $G$ has a class-2-nilpotent normal subgroup $R$ such that both the index $[G:R]$ and the order of the commutator subgroup $[R,R]$ are $\epsilon$-bounded. In the same spirit we consider the cases where $K$ is a term of the lower central series of $G$, or a Sylow subgroup, etc.
Why are ELEvoHI CME Arrival Predictions Different if Based on STEREO‐A or STEREO‐B Heliospheric Imager Observations?

Accurate forecasting of the arrival time and arrival speed of coronal mass ejections (CMEs) is an unsolved problem in space weather research. In this study, a comparison of the predicted arrival times and speeds for each CME based, independently, on the inputs from the two STEREO vantage points is carried out. We perform hindcasts using ELlipse Evolution model based on Heliospheric Imager observations (ELEvoHI) ensemble modeling. An estimate of the ambient solar wind conditions is obtained by the Wang‐Sheeley‐Arge/Heliospheric Upwind eXtrapolation (WSA/HUX) model combination that serves as input to ELEvoHI. We carefully select 12 CMEs between February 2010 and July 2012 that show clear signatures in both STEREO‐A and STEREO‐B HI time‐elongation maps, that propagate close to the ecliptic plane, and that have corresponding in situ signatures at Earth. We find a mean arrival time difference of 6.5 h between predictions from the two different viewpoints, which can reach up to 9.5 h for individual CMEs, while the mean arrival speed difference is 63 km s−1. An ambient solar wind with a large speed variance leads to larger differences in the STEREO‐A and STEREO‐B CME arrival time predictions (cc = 0.92). Additionally, we compare the predicted arrivals, from both spacecraft, to the actual in situ arrivals at Earth and find a mean absolute error of 7.5 ± 9.5 h for the arrival time and 87 ± 111 km s−1 for the arrival speed. There is no tendency for one spacecraft to provide more accurate arrival predictions than the other.
Exploiting Shared Representations for Personalized Federated Learning

Deep neural networks have shown the ability to extract universal feature representations from data such as images and text that have been useful for a variety of learning tasks. However, the fruits of representation learning have yet to be fully-realized in federated settings. Although data in federated settings is often non-i.i. d. across clients, the success of centralized deep learning suggests that data often shares a global feature representation, while the statistical heterogeneity across clients or tasks is concentrated in the labels. Based on this intuition, we propose a novel federated learning framework and algorithm for learning a shared data representation across clients and unique local heads for each client. Our algorithm harnesses the distributed computational power across clients to perform many local-updates with respect to the low-dimensional local parameters for every update of the representation. We prove that this method obtains linear convergence to the ground-truth representation with near-optimal sample complexity in a linear setting, demonstrating that it can efficiently reduce the problem dimension for each client. This result is of interest beyond federated learning to a broad class of problems in which we aim to learn a shared low-dimensional representation among data distributions, for example in meta-learning and multi-task learning. Further, extensive experimental results show the empirical improvement of our method over alternative personalized federated learning approaches in federated environments with heterogeneous data.
Robust and Efficient Planning using Adaptive Entropy Tree Search

In this paper, we present the Adaptive Entropy Tree Search (ANTS) algorithm. ANTS builds on recent successes of maximum entropy planning (Xiao et al., 2019; Dam et al., 2020) while mitigating its arguably major drawback - sensitivity to the temperature setting. We endow ANTS with a mechanism, which adapts the temperature to match a given range of action selection entropy in the nodes of the planning tree. With this mechanism, the ANTS planner enjoys remarkable hyper-parameter robustness, achieves high scores on the Atari benchmark, and is a capable component of a planning-learning loop akin to AlphaZero. We believe that all these features make ANTS a compelling choice for a general planner for complex tasks.
Doping: A technique for efficient compression of LSTM models using sparse structured additive matrices

Structured matrices, such as those derived from Kronecker products (KP), are effective at compressing neural networks, but can lead to unacceptable accuracy loss when applied to large models. In this paper, we propose the notion of doping -- addition of an extremely sparse matrix to a structured matrix. Doping facilitates additional degrees of freedom for a small number of parameters, allowing them to independently diverge from the fixed structure. To train LSTMs with doped structured matrices, we introduce the additional parameter matrix while slowly annealing its sparsity level. However, we find that performance degrades as we slowly sparsify the doping matrix, due to co-matrix adaptation (CMA) between the structured and the sparse matrices. We address this over dependence on the sparse matrix using a co-matrix dropout regularization (CMR) scheme. We provide empirical evidence to show that doping, CMA and CMR are concepts generally applicable to multiple structured matrices (Kronecker Product, LMF, Hybrid Matrix Decomposition). Additionally, results with doped kronecker product matrices demonstrate state-of-the-art accuracy at large compression factors (10 - 25x) across 4 natural language processing applications with minor loss in accuracy. Doped KP compression technique outperforms previous state-of-the art compression results by achieving 1.3 - 2.4x higher compression factor at a similar accuracy, while also beating strong alternatives like pruning and low-rank methods by a large margin (8% or more). Additionally, we show that doped KP can be deployed on commodity hardware using the current software stack and achieve 2.5 - 5.5x inference run-time speed-up over baseline.
Connecting Interpretability and Robustness in Decision Trees through Separation

Recent research has recognized interpretability and robustness as essential properties of trustworthy classification. Curiously, a connection between robustness and interpretability was empirically observed, but the theoretical reasoning behind it remained elusive. In this paper, we rigorously investigate this connection. Specifically, we focus on interpretation using decision trees and robustness to $l_{\infty}$-perturbation. Previous works defined the notion of $r$-separation as a sufficient condition for robustness. We prove upper and lower bounds on the tree size in case the data is $r$-separated. We then show that a tighter bound on the size is possible when the data is linearly separated. We provide the first algorithm with provable guarantees both on robustness, interpretability, and accuracy in the context of decision trees. Experiments confirm that our algorithm yields classifiers that are both interpretable and robust and have high accuracy. The code for the experiments is available at https://github.com/yangarbiter/interpretable-robust-trees .
TransGAN: Two Transformers Can Make One Strong GAN

The recent explosive interest on transformers has suggested their potential to become powerful “universal” models for computer vision tasks, such as classiﬁcation, detection, and segmentation. However, how further transformers can go - are they ready to take some more notoriously difﬁcult vision tasks, e.g., generative adversarial networks (GANs)? Driven by that curiosity, we conduct the ﬁrst pilot study in building a GAN completely free of convolutions , using only pure transformer-based architectures. Our vanilla GAN architecture, dubbed TransGAN , consists of a memory-friendly transformer-based generator that progressively increases feature resolution while decreasing embedding dimension, and a patch-level discriminator that is also transformer-based. We then demonstrate TransGAN to notably beneﬁt from data augmentations (more than standard GANs), a multi-task co-training strategy for the generator, and a locally initialized self-attention that emphasizes the neighborhood smoothness of natural images. Equipped with those ﬁndings, TransGAN can effectively scale up with bigger models and high-resolution image datasets. Speciﬁcally, our best architecture achieves highly competitive performance compared to current state-of-the-art GANs based on convolutional backbones. Speciﬁcally, Trans-GAN sets new state-of-the-art IS score of 10.10 and FID score of 25.32 on STL-10. It also reaches competitive 8.64 IS score and 11.89 FID score on Cifar-10, and 12.23 FID score on CelebA 64 × 64 , respectively. We also conclude with a discussion of the current limitations and future potential of TransGAN. The code is available at https:// github.com/VITA-Group/TransGAN .
A Theory of Choice Bracketing under Risk

Decision makers in the real world usually face multiple risky choice problems. For instance, an investor might need to take care of her investment accounts simultaneously in different financial markets, including stocks, bonds, and cryptocurrencies. One implicit assumption of the long-standing focus on single choice problems in economics is that agents can rationally aggregate and assess risks and consequences in multiple sources. However, aggregating risks from multiple sources can be complex and demanding, and decision makers usually adopt heuristics, such as narrow bracketing and correlation neglect, to simplify the decision process. Narrow bracketing describes the situation where a decision maker faced with multiple choice problems tends to choose an option in each decision without full regard to other decisions. Correlation neglect means that the decision maker might ignore the interdependence of risks from different sources. Both heuristics are well-documented in experiments and behavioral economics, but have received inadequate attention in the choice-theoretic literature. One possible reason is that they are typically interpreted as exotic behavioral or "irrational" biases and supposed to deviate drastically from the standard framework. This paper axiomatically characterizes the two heuristics by introducing an intuitive relaxation of the independence axiom in the standard von Neumann-Morgenstern expected utility framework. The representation theorem allows for either narrow bracketing, or correlation neglect, or both of them, or none of them. This behavioral foundation suggests that the two heuristics are no more "irrational'' than other commonly accepted non-EU theories like certainty effect and reference dependence, and hence deserve more attention in the future. The flexibility of our framework allows for applications in various choice domains. First, my model can accommodate the experimental evidence on narrow bracketing, where subjects tend to violate first-order stochastic dominance when making choices in simultaneous and independent monetary gambles. Second, in environments with background risk, I show how the model can explain non-trivial risk aversion over small gambles. Finally, in intertemporal choices, I show that my framework unifies three commonly used and seemingly distinct models of time preferences in the literature. This is ex-ante surprising since the analysis is based solely on simplifying heuristics to deal with multi-source risk and contains no normative properties required in each of the three models. Then, I propose a novel class of time preferences that can simultaneously satisfy indifference to time resolution of uncertainty, dynamic consistency and separation of time and risk preferences. This provides an alternative solution to the impossibility result in [3]. One specification of my new model shares the same predictions and explanatory power as [1] in macroeconomics and finance applications, without being subject to the recent critiques in [2].
Multi-Scale Neural Networks for to Fluid Flow in 3D Porous Media

The permeability of complex porous materials can be obtained via direct flow simulation, which provides the most accurate results, but is very computationally expensive. In particular, the simulation convergence time scales poorly as simulation domains become tighter or more heterogeneous. Semi-analytical models that rely on averaged structural properties (i.e. porosity and tortuosity) have been proposed, but these features only summarize the domain, resulting in limited applicability. On the other hand, data-driven machine learning approaches have shown great promise for building more general models by virtue of accounting for the spatial arrangement of the domains solid boundaries. However, prior approaches building on the Convolutional Neural Network (ConvNet) literature concerning 2D image recognition problems do not scale well to the large 3D domains required to obtain a Representative Elementary Volume (REV). As such, most prior work focused on homogeneous samples, where a small REV entails that that the global nature of fluid flow could be mostly neglected, and accordingly, the memory bottleneck of addressing 3D domains with ConvNets was side-stepped. Therefore, important geometries such as fractures and vuggy domains could not be well-modeled. In this work, we address this limitation with a general multiscale deep learning model that is able to learn from porous media simulation data. By using a coupled set of neural networks that view the domain on different scales, we enable the evaluation of large images in approximately one second on a single Graphics Processing Unit. This model architecture opens up the possibility of modeling domain sizes that would not be feasible using traditional direct simulation tools on a desktop computer.
The Phase Transition of Discrepancy in Random Hypergraphs

Motivated by the Beck-Fiala conjecture, we study the discrepancy problem in two related models of random hypergraphs on $n$ vertices and $m$ edges. In the first (edge-independent) model, a random hypergraph $H_1$ is constructed by fixing a parameter $p$ and allowing each of the $n$ vertices to join each of the $m$ edges independently with probability $p$. In the parameter range in which $pn \rightarrow \infty$ and $pm \rightarrow \infty$, we show that with high probability (w.h.p.) $H_1$ has discrepancy at least $\Omega(2^{-n/m} \sqrt{pn})$ when $m = O(n)$, and at least $\Omega(\sqrt{pn \log\gamma })$ when $m \gg n$, where $\gamma = \min\{ m/n, pn\}$. In the second (edge-dependent) model, $d$ is fixed and each vertex of $H_2$ independently joins exactly $d$ edges uniformly at random. We obtain analogous results for this model by generalizing the techniques used for the edge-independent model with $p=d/m$. Namely, for $d \rightarrow \infty$ and $dn/m \rightarrow \infty$, we prove that w.h. p. $H_{2}$ has discrepancy at least $\Omega(2^{-n/m} \sqrt{dn/m})$ when $m = O(n)$, and at least $\Omega(\sqrt{(dn/m) \log\gamma})$ when $m \gg n$, where $\gamma =\min\{m/n, dn/m\}$. Furthermore, we obtain nearly matching asymptotic upper bounds on the discrepancy in both models (when $p=d/m$), in the dense regime of $m \gg n$. Specifically, we apply the partial colouring lemma of Lovett and Meka to show that w.h. p. $H_{1}$ and $H_{2}$ each have discrepancy $O( \sqrt{dn/m} \log(m/n))$, provided $d \rightarrow \infty$, $d n/m \rightarrow \infty$ and $m \gg n$. This result is algorithmic, and together with the work of Bansal and Meka characterizes how the discrepancy of each random hypergraph model transitions from $\Theta(\sqrt{d})$ to $o(\sqrt{d})$ as $m$ varies from $m=\Theta(n)$ to $m \gg n$.
How Framelets Enhance Graph Neural Networks

This paper presents a new approach for assembling graph neural networks based on framelet transforms. The latter provides a multi-scale representation for graph-structured data. We decompose an input graph into low-pass and high-pass frequencies coefficients for network training, which then defines a framelet-based graph convolution. The framelet decomposition naturally induces a graph pooling strategy by aggregating the graph feature into low-pass and high-pass spectra, which considers both the feature values and geometry of the graph data and conserves the total information. The graph neural networks with the proposed framelet convolution and pooling achieve state-of-the-art performance in many node and graph prediction tasks. Moreover, we propose shrinkage as a new activation for the framelet convolution, which thresholds high-frequency information at different scales. Compared to ReLU, shrinkage activation improves model performance on denoising and signal compression: noises in both node and structure can be significantly reduced by accurately cutting off the high-pass coefficients from framelet decomposition, and the signal can be compressed to less than half its original size with well-preserved prediction performance.