Miracle-dz/UniFAR-FTUs huggingface dataset:
Growth and characterization of Si1−xGex and Ge epilayers on (100) Si

Two approaches to the growth of high‐quality epitaxial Ge epilayers on (100) Si have been investigated. The first consisted of compositional‐grading Si1−xGex layers and the use of strained‐layer superlattices as dislocation filters. In general, this method produced unsatisfactory results, due to the difficulty in achieving good epitaxial growth in the Ge concentration interval 30%−70%. The second approach consisted of simply depositing pure Ge directly on (100) Si. Excellent epitaxial films with dislocation densities of less than 107 cm−2 and smooth morphology were obtained after optimization of the growth parameters. The initial growth temperature and post‐growth annealing were found to be critical in obtaining good epitaxial material.
VisCo Grids: Surface Reconstruction with Viscosity and Coarea Grids

Surface reconstruction has been seeing a lot of progress lately by utilizing Implicit Neural Representations (INRs). Despite their success, INRs often introduce hard to control inductive bias (i.e., the solution surface can exhibit unexplainable behaviours), have costly inference, and are slow to train. The goal of this work is to show that replacing neural networks with simple grid functions, along with two novel geometric priors achieve comparable results to INRs, with instant inference, and improved training times. To that end we introduce VisCo Grids: a grid-based surface reconstruction method incorporating Viscosity and Coarea priors. Intuitively, the Viscosity prior replaces the smoothness inductive bias of INRs, while the Coarea favors a minimal area solution. Experimenting with VisCo Grids on a standard reconstruction baseline provided comparable results to the best performing INRs on this dataset.
Analysis of DNA sequences through local distribution of nucleotides in strategic neighborhoods

We propose a new alignment-free algorithm by constructing a compact vector representation on $\mathbb{R}^{24}$ of a DNA sequence of arbitrary length. Each component of this vector is obtained from a representative sequence, the elements of which are the values realized by a function $\Gamma$. $\Gamma$ acts on neighborhoods of arbitrary radius that are located at strategic positions within the DNA sequence and carries complete information about the local distribution of frequencies of the nucleotides as a consequence of the uniqueness of prime factorization of integer. The algorithm exhibits linear time complexity and turns out to consume significantly small memory. The two natural parameters characterizing the radius and location of the neighbourhoods are fixed by comparing the phylogenetic tree with the benchmark for full genome sequences of fish mtDNA datasets. Using these fitting parameters, the method is applied to analyze a number of genome sequences from benchmark and other standard datasets. The algorithm proves to be computationally efficient compared to Co-phylog and CD-MAWS when applied over a certain range of a simulated dataset.
Identifiability of causal graphs under nonadditive conditionally parametric causal models

Causal discovery from observational data is a rather challenging, often impossible, task. However, an estimation of the causal structure is possible under certain assumptions on the data-generation process. Numerous commonly used methods rely on the additivity of noise in the structural equation models. Additivity implies that the variance or the tail of the effect, given the causes, is invariant; thus, the cause only affects the mean. However, the tail or other characteristics of the random variable can provide different information regarding the causal structure. Such cases have received very little attention in the literature thus far. Previous studies have revealed that the causal graph is identifiable under different models, such as linear non-Gaussian, post-nonlinear, or quadratic variance functional models. In this study, we introduce a new class of models called the conditional parametric causal models (CPCM), where the cause affects different characteristics of the effect. We use sufficient statistics to reveal the identifiability of the CPCM models in the exponential family of conditional distributions. Moreover, we propose an algorithm for estimating the causal structure from a random sample from the CPCM. The empirical properties of the methodology are studied for various datasets, including an application on the expenditure behavior of residents of the Philippines.
Train/Test-Time Adaptation with Retrieval

We introduce Train/Test-Time Adaptation with Retrieval (T3 AR), a method to adapt models both at train and test time by means of a retrieval module and a searchable pool of external samples. Before inference, T3AR adapts a given model to the downstream task using refined pseudo-labels and a self-supervised contrastive objective function whose noise distribution leverages retrieved real samples to improve feature adaptation on the target data manifold. The retrieval of real images is key to T3AR since it does not rely solely on synthetic data augmentations to compensate for the lack of adaptation data, as typically done by other adaptation algorithms. Furthermore, thanks to the retrieval module, our method gives the user or service provider the possibility to improve model adaptation on the downstream task by incorporating further relevant data or to fully remove samples that may no longer be available due to changes in user preference after deployment. First, we show that T3AR can be used at training time to improve downstream fine-grained classification over standard fine-tuning baselines, and the fewer the adaptation data the higher the relative improvement (up to 13%). Second, we apply T3ARfor test-time adaptation and show that exploiting a pool of external images at test-time leads to more robust representations over existing methods on DomainNet-126 and VISDA-C, especially when few adaptation data are available (up to 8%).
Multi-pooling 3D Convolutional Neural Network for fMRI Classification of Visual Brain States

Neural decoding of visual object classification via functional magnetic resonance imaging (fMRI) data is challenging and is vital to understand underlying brain mechanisms. This paper proposed a multi-pooling 3D convolutional neural network (MP3DCNN) to improve fMRI classification accuracy. MP3DCNN is mainly composed of a three-layer 3DCNN, where the first and second layers of 3D convolutions each have a branch of pooling connection. The results showed that this model can improve the classification accuracy for categorical (face vs. object), face sub-categorical (male face vs. female face), and object sub-categorical (natural object vs. artificial object) classifications from 1.684% to 14.918% over the previous study [1] in decoding brain mechanisms.
Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA)

Despite the recent advances in deep neural networks, standard convolutional kernels limit the applications of these networks to the Euclidean domain only. Considering the geodesic nature of the measurement of the earth's surface, remote sensing is one such area that can benefit from non-Euclidean and spherical domains. For this purpose, we propose a novel Graph Neural Network architecture for spatial and spatio-temporal classification using satellite imagery. We propose a hybrid attention method to learn the relative importance of irregular neighbors in remote sensing data. Instead of classifying each pixel, we propose a method based on Simple Linear Iterative Clustering (SLIC) image segmentation and Graph Attention GAT. The superpixels obtained from SLIC become the nodes of our Graph Convolution Network (GCN). We then construct a region adjacency graph (RAG) where each superpixel is connected to every other adjacent superpixel in the image, enabling information to propagate globally. Finally, we propose a Spatially driven Attention Graph Neural Network (SAG-NN) to classify each RAG. We also propose an extension to our SAG-NN for spatio-temporal data. Unlike regular grids of pixels in images, superpixels are irregular in nature and cannot be used to create spatio-temporal graphs. We introduce temporal bias by combining unconnected RAGs from each image into one supergraph. This is achieved by introducing block adjacency matrices resulting in novel Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA). We evaluate our proposed methods on two remote sensing datasets namely Asia14 and C2D2. In comparison with both non-graph and graph-based approaches our SAG-NN and STAG-NN-BA achieved superior accuracy on all the datasets while incurring less computation cost. The code and dataset will be made public via our GitHub repository.
Information Maximizing Curriculum: A Curriculum-Based Approach for Training Mixtures of Experts

Mixtures of Experts (MoE) are known for their ability to learn complex conditional distributions with multiple modes. However, despite their potential, these models are challenging to train and often tend to produce poor performance, explaining their limited popularity. Our hypothesis is that this under-performance is a result of the commonly utilized maximum likelihood (ML) optimization, which leads to mode averaging and a higher likelihood of getting stuck in local maxima. We propose a novel curriculum-based approach to learning mixture models in which each component of the MoE is able to select its own subset of the training data for learning. This approach allows for independent optimization of each component, resulting in a more modular architecture that enables the addition and deletion of components on the ﬂy, leading to an optimization less susceptible to local optima. The curricula can ignore data-points from modes not represented by the MoE, reducing the mode-averaging problem. To achieve a good data coverage, we couple the optimization of the curricula with a joint entropy objective and optimize a lower bound of this objective. We evaluate our curriculum-based approach on a variety of multimodal behavior learning tasks and demonstrate its superiority over competing methods for learning MoE models and conditional generative models.
Improving the Transferability of Adversarial Examples via Direction Tuning

In the transfer-based adversarial attacks, adversarial examples are only generated by the surrogate models and achieve effective perturbation in the victim models. Although considerable efforts have been developed on improving the transferability of adversarial examples generated by transfer-based adversarial attacks, our investigation found that, the big deviation between the actual and steepest update directions of the current transfer-based adversarial attacks is caused by the large update step length, resulting in the generated adversarial examples can not converge well. However, directly reducing the update step length will lead to serious update oscillation so that the generated adversarial examples also can not achieve great transferability to the victim models. To address these issues, a novel transfer-based attack, namely direction tuning attack, is proposed to not only decrease the update deviation in the large step length, but also mitigate the update oscillation in the small sampling step length, thereby making the generated adversarial examples converge well to achieve great transferability on victim models. In addition, a network pruning method is proposed to smooth the decision boundary, thereby further decreasing the update oscillation and enhancing the transferability of the generated adversarial examples. The experiment results on ImageNet demonstrate that the average attack success rate (ASR) of the adversarial examples generated by our method can be improved from 87.9\% to 94.5\% on five victim models without defenses, and from 69.1\% to 76.2\% on eight advanced defense methods, in comparison with that of latest gradient-based attacks.
Koala: An Index for Quantifying Overlaps with Pre-training Corpora

In very recent years more attention has been placed on probing the role of pre-training data in Large Language Models (LLMs) downstream behaviour. Despite the importance, there is no public tool that supports such analysis of pre-training corpora at large scale. To help research in this space, we launch Koala, a searchable index over large pre-training corpora using compressed suffix arrays with highly efficient compression rate and search support. In its first release we index the public proportion of OPT 175B pre-training data. Koala provides a framework to do forensic analysis on the current and future benchmarks as well as to assess the degree of memorization in the output from the LLMs. Koala is available for public use at https://koala-index.erc. monash.edu/.
Dual-Quaternion Julia Fractals

Fractals offer the ability to generate fascinating geometric shapes with all sorts of unique characteristics (for instance, fractal geometry provides a basis for modelling infinite detail found in nature). While fractals are non-euclidean mathematical objects which possess an assortment of properties (e.g., attractivity and symmetry), they are also able to be scaled down, rotated, skewed and replicated in embedded contexts. Hence, many different types of fractals have come into limelight since their origin discovery. One particularly popular method for generating fractal geometry is using Julia sets. Julia sets provide a straightforward and innovative method for generating fractal geometry using an iterative computational modelling algorithm. In this paper, we present a method that combines Julia sets with dual-quaternion algebra. Dual-quaternions are an alluring principal with a whole range interesting mathematical possibilities. Extending fractal Julia sets to encompass dual-quaternions algebra provides us with a novel visualize solution. We explain the method of fractals using the dual-quaternions in combination with Julia sets. Our prototype implementation demonstrate an efficient methods for rendering fractal geometry using dual-quaternion Julia sets based upon an uncomplicated ray tracing algorithm. We show a number of different experimental isosurface examples to demonstrate the viability of our approach.
SASS: Data and Methods for Subject Aware Sentence Simplification

Sentence simplification tends to focus on the generic simplification of sentences by making them more readable and easier to understand. This paper provides a dataset aimed at training models that perform subject aware sentence simplifications rather than simplifying sentences as a whole. We also test models on that dataset which are inspired by model architecture used in abstractive summarization. We hand generated portions of the data and augment the dataset by further manipulating those hand written simplifications. Our results show that data-augmentation, data-masking, and model architecture choices used in summarization provide a solid baseline for comparison on subject aware simplification.
The size of semigroup orbits modulo primes

Let $V$ be a projective variety defined over a number field $K$, let $S$ be a polarized set of endomorphisms of $V$ all defined over $K$, and let $P\in V(K)$. For each prime $\mathfrak{p}$ of $K$, let $m_{\mathfrak{p}}(S,P)$ denote the number of points in the orbit of $P\bmod\mathfrak{p}$ for the semigroup of maps generated by $S$. Under suitable hypotheses on $S$ and $P$, we prove an analytic estimate for $m_{\mathfrak{p}}(S,P)$ and use it to show that the set of primes for which $m_{\mathfrak{p}}(S,P)$ grows subexponentially as a function of $\operatorname{\mathsf{N}}_{K/\mathbb{Q}}\mathfrak{p}$ is a set of density zero. For $V=\mathbb{P}^1$ we show that this holds for a generic set of maps $S$ provided that at least two of the maps in $S$ have degree at least four.
Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learning

Most successes in robotic manipulation have been restricted to single-arm gripper robots, whose low dexterity limits the range of solvable tasks to pick-and-place, inser-tion, and object rearrangement. More complex tasks such as assembly require dual and multi-arm platforms, but entail a suite of unique challenges such as bi-arm coordination and collision avoidance, robust grasping, and long-horizon planning. In this work we investigate the feasibility of training deep reinforcement learning (RL) policies in simulation and transferring them to the real world (Sim2Real) as a generic methodology for obtaining performant controllers for real-world bi-manual robotic manipulation tasks. As a testbed for bi-manual manipulation, we develop the U-Shape Magnetic BlockAssembly Task, wherein two robots with parallel grippers must connect 3 magnetic blocks to form a U-shape. Without manually-designed controller nor human demonstrations, we demonstrate that with careful Sim2Real considerations, our policies trained with RL in simulation enable two xArm6 robots to solve the U-shape assembly task with a success rate of above90% in simulation, and 50% on real hardware without any additional real-world fine-tuning. Through careful ablations,we highlight how each component of the system is critical for such simple and successful policy learning and transfer,including task specification, learning algorithm, direct joint-space control, behavior constraints, perception and actuation noises, action delays and action interpolation. Our results present a significant step forward for bi-arm capability on real hardware, and we hope our system can inspire future research on deep RL and Sim2Real transfer of bi-manualpolicies, drastically scaling up the capability of real-world robot manipulators.
Bismuth telluride compounds with high thermoelectric figures of merit

The thermoelectric properties of the p-type (Bi0.25Sb0.75)2Te3 doped with 8 wt. % excess Te and the n-type Bi2(Te0.94Se0.06)3 doped substantially with 0.07 wt. % I, 0.02 wt. % Te, and 0.03 wt. % CuBr which were grown by the Bridgman method at a rate of 6 cm/h were measured before and after annealing, where annealing was done in the temperature range from 473 up to 673 K for 2–5 h in a vacuum and a hydrogen stream. No annealing effect on the power factor was observed for the p-type specimen, but the as-grown p-type specimen exhibited a large power factor of 5.53×10−3 W/mK2 at 308 K and a low thermal conductivity of 1.21 W/mK. When the n-type specimen was annealed at 473 K for 2 h in a hydrogen stream, however, the power factor at 308 K increased significantly from 3.26×10−3 to 4.73×10−3 K−1 but its thermal conductivity then increased by about 3% from 1.26 to 1.30 W/mK. As a result, the maximum thermoelectric figure of merits Z at 308 K for the as-grown p- and annealed n-type specimens reached surprisingly .
A Strong Duality Result for Constrained POMDPs with Multiple Cooperative Agents

The work studies the problem of decentralized constrained POMDPs in a team-setting where multiple nonstrategic agents have asymmetric information. Using an extension of Sion's Minimax theorem for functions with positive infinity and results on weak-convergence of measures, strong duality is established for the setting of infinite-horizon expected total discounted costs when the observations lie in a countable space, the actions are chosen from a finite space, the constraint costs are bounded, and the objective cost is bounded from below.
Weak Recovery Threshold for the Hypergraph Stochastic Block Model

We study the weak recovery problem on the $r$-uniform hypergraph stochastic block model ($r$-HSBM) with two balanced communities. In HSBM a random graph is constructed by placing hyperedges with higher density if all vertices of a hyperedge share the same binary label, and weak recovery asks to recover a non-trivial fraction of the labels. We introduce a multi-terminal version of strong data processing inequalities (SDPIs), which we call the multi-terminal SDPI, and use it to prove a variety of impossibility results for weak recovery. In particular, we prove that weak recovery is impossible below the Kesten-Stigum (KS) threshold if $r=3,4$, or a strength parameter $\lambda$ is at least $\frac 15$. Prior work Pal and Zhu (2021) established that weak recovery in HSBM is always possible above the KS threshold. Consequently, there is no information-computation gap for these cases, which (partially) resolves a conjecture of Angelini et al. (2015). To our knowledge this is the first impossibility result for HSBM weak recovery. As usual, we reduce the study of non-recovery of HSBM to the study of non-reconstruction in a related broadcasting on hypertrees (BOHT) model. While we show that BOHT's reconstruction threshold coincides with KS for $r=3,4$, surprisingly, we demonstrate that for $r\ge 7$ reconstruction is possible also below KS. This shows an interesting phase transition in the parameter $r$, and suggests that for $r\ge 7$, there might be an information-computation gap for the HSBM. For $r=5,6$ and large degree we propose an approach for showing non-reconstruction below KS, suggesting that $r=7$ is the correct threshold for onset of the new phase.
Prompt Tuning based Adapter for Vision-Language Model Adaption

Large pre-trained vision-language (VL) models have shown significant promise in adapting to various downstream tasks. However, fine-tuning the entire network is challenging due to the massive number of model parameters. To address this issue, efficient adaptation methods such as prompt tuning have been proposed. We explore the idea of prompt tuning with multi-task pre-trained initialization and find it can significantly improve model performance. Based on our findings, we introduce a new model, termed Prompt-Adapter, that combines pre-trained prompt tunning with an efficient adaptation network. Our approach beat the state-of-the-art methods in few-shot image classification on the public 11 datasets, especially in settings with limited data instances such as 1 shot, 2 shots, 4 shots, and 8 shots images. Our proposed method demonstrates the promise of combining prompt tuning and parameter-efficient networks for efficient vision-language model adaptation. The code is publicly available at: https://github.com/Jingchensun/prompt_adapter.
Revisiting the Chandra Observation on the Region of PSR J1J1809–193: Indication of the Existence of an X-Ray Halo and Implication for the Origin of HESS J1809–193

HESS J1809–193 is an extended TeV γ-ray source and the origin of its γ-ray emission remains ambiguous. The pulsar wind nebula (PWN) of PSR J1J1809–193 lying inside the extended γ-ray emission is a possible candidate. Powered by the central pulsar, ultrarelativistic electrons in the PWN can produce radio to X-ray emission through synchrotron and γ-ray emission by inverse Compton (IC) scattering. To check whether this PWN is the counterpart of HESS J1809–193, we analyzed the Chandra X-ray radial intensity profile and spectral index profile of this PWN. We then adopted a one-zone isotropic diffusion model to fit the keV and TeV data. We found diffuse nonthermal X-ray emission extending beyond the PWN, which is likely an X-ray halo radiated by escaping electron/positron pairs from the PWN. A relatively strong magnetic field of ∼20 μG is required to explain the spatial evolution of the X-ray spectrum (i.e., the significant softening of the spectrum with increasing distance from the pulsar), which, however, would suppress the IC radiation of pairs. Our result implies that a hadronic component may be needed to explain HESS J1809–193.