Miracle-dz/UniFAR-FTUs huggingface dataset:
Data Distributional Properties Drive Emergent In-Context Learning in Transformers

Large transformer-based models are able to perform in-context few-shot learning, without being explicitly trained for it. This observation raises the question: what aspects of the training regime lead to this emergent behavior? Here, we show that this behavior is driven by the distributions of the training data itself. In-context learning emerges when the training data exhibits particular distributional properties such as burstiness (items appear in clusters rather than being uniformly distributed over time) and having large numbers of rarely occurring classes. In-context learning also emerges more strongly when item meanings or interpretations are dynamic rather than fixed. These properties are exemplified by natural language, but are also inherent to naturalistic data in a wide range of other domains. They also depart significantly from the uniform, i.i. d. training distributions typically used for standard supervised learning. In our initial experiments, we found that in-context learning traded off against more conventional weight-based learning, and models were unable to achieve both simultaneously. However, our later experiments uncovered that the two modes of learning could co-exist in a single model when it was trained on data following a skewed Zipfian distribution -- another common property of naturalistic data, including language. In further experiments, we found that naturalistic data distributions were only able to elicit in-context learning in transformers, and not in recurrent models. In sum, our findings indicate how the transformer architecture works together with particular properties of the training data to drive the intriguing emergent in-context learning behaviour of large language models, and how future work might encourage both in-context and in-weights learning in domains beyond language.
Informed Steiner Trees: Sampling and Pruning for Multi-Goal Path Finding in High Dimensions

We interleave sampling based motion planning methods with pruning ideas from minimum spanning tree algorithms to develop a new approach for solving a Multi-Goal Path Finding (MGPF) problem in high dimensional spaces. The approach alternates between sampling points from selected regions in the search space and de-emphasizing regions that may not lead to good solutions for MGPF. Our approach provides an asymptotic, 2-approximation guarantee for MGPF. We also present extensive numerical results to illustrate the advantages of our proposed approach over uniform sampling in terms of the quality of the solutions found and computation speed.
Spectral Galerkin method for solving elastic wave scattering problems with multiple open arcs

We study the elastic time-harmonic wave scattering problems on unbounded domains with boundaries composed of finite collections of disjoints finite open arcs (or cracks) in two dimensions. Specifically, we present a fast spectral Galerkin method for solving the associated weakly- and hyper-singular boundary integral equations (BIEs) arising from Dirichlet and Neumann boundary conditions, respectively. Discretization bases of the resulting BIEs employ weighted Chebyshev polynomials that capture the solutions' edge behavior. We show that these bases guarantee exponential convergence in the polynomial degree when assuming analyticity of sources and arcs geometries. Numerical examples demonstrate the accuracy and robustness of the proposed method with respect to number of arcs and wavenumber.
Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solution

Recently, Loizou et al. (2021), proposed and analyzed stochastic gradient descent (SGD) with stochastic Polyak stepsize (SPS). The proposed SPS comes with strong convergence guarantees and competitive performance; however, it has two main drawbacks when it is used in non-over-parameterized regimes: (i) It requires a priori knowledge of the optimal mini-batch losses, which are not available when the interpolation condition is not satisfied (e.g., regularized objectives), and (ii) it guarantees convergence only to a neighborhood of the solution. In this work, we study the dynamics and the convergence properties of SGD equipped with new variants of the stochastic Polyak stepsize and provide solutions to both drawbacks of the original SPS. We first show that a simple modification of the original SPS that uses lower bounds instead of the optimal function values can directly solve issue (i). On the other hand, solving issue (ii) turns out to be more challenging and leads us to valuable insights into the method's behavior. We show that if interpolation is not satisfied, the correlation between SPS and stochastic gradients introduces a bias, which effectively distorts the expectation of the gradient signal near minimizers, leading to non-convergence - even if the stepsize is scaled down during training. To fix this issue, we propose DecSPS, a novel modification of SPS, which guarantees convergence to the exact minimizer - without a priori knowledge of the problem parameters. For strongly-convex optimization problems, DecSPS is the first stochastic adaptive optimization method that converges to the exact solution without restrictive assumptions like bounded iterates/gradients.
An acceleration technique for methods for finding the nearest point in a polytope and computing the distance between two polytopes

We present a simple and efficient acceleration technique for an arbitrary method for computing the Euclidean projection of a point onto a convex polytope, defined as the convex hull of a finite number of points, in the case when the number of points in the polytope is much greater than the dimension of the space. The technique consists in applying any given method to a"small"subpolytope of the original polytope and gradually shifting it, till the projection of the given point onto the subpolytope coincides with its projection onto the original polytope. The results of numerical experiments demonstrate the high efficiency of the proposed acceleration technique. In particular, they show that the reduction of computation time increases with an increase of the number of points in the polytope and is proportional to this number for some methods. In the second part of the paper, we also discuss a straightforward extension of the proposed acceleration technique to the case of arbitrary methods for computing the distance between two convex polytopes, defined as the convex hulls of finite sets of points.
Three Essays on Information Privacy of Mobile Users in the Context of Mobile Apps

The increasing demand for mobile apps is out the current capability of mobile app developers. In addition, the growing trend in smartphone ownership and the time people spend on mobile apps has raised several opportunities and risks for users and developers. The average time everyday a user spend on smartphones to use mobile apps is more than two hours. The worldwide mobile app revenue increase is estimated to grow 33%, $19 billion. Three quarter of the time used on mobile apps is solely for using game and social networking apps. To provide more customized services and function to users, mobile apps need to access to personal information. However, 80% of mobile apps put people's information privacy at risk. There is a major gap in the literature about the privacy concerns of mobile device users in the context of mobile apps. This dissertation addresses one fundamental research question: how does individuals' privacy change in the context of mobile apps? More precisely, the focus of this dissertation is on information privacy role in individuals' and mobile app developers' protective behaviors. We investigate the information sensitivity level influence on mobile app developers' emphasis on privacy across mobile app categories. The results show information sensitivity level has a significant impact on developers' emphasis on secondary usage of information. Moreover, we analyze the privacy trade-off dynamism in using a new social networking app and how it could result in emotional attachment. Results show initial use and initial disclosure influence the privacy trade-off from pre-use to initial-use period. Finally, the effect of privacy concern and engagement on emotional attachment is demonstrated. This dissertation addresses one fundamental research question: how does individuals' privacy change in the context of mobile apps? More precisely, the focus of this dissertation is on information privacy role in individuals' and mobile app developers' protective behaviors. We investigate the information sensitivity level influence on mobile app developers' emphasis on privacy across mobile app categories. The results show information sensitivity level has a significant impact on developers' emphasis on secondary usage of information. Moreover, we analyze the privacy trade-off dynamism in using a new social networking app and how it could result in emotional attachment. Results show initial use and initial disclosure influence the privacy trade-off from pre-use to initial-use period. Finally, the effect of privacy concern and engagement on emotional attachment is demonstrated.
Explainable Deep Learning Methods in Medical Diagnosis: A Survey

The remarkable success of deep learning has prompted interest in its application to medical diagnosis. Even tough state-of-the-art deep learning models have achieved human-level accuracy on the classification of different types of medical data, these models are hardly adopted in clinical workflows, mainly due to their lack of interpretability. The black-box-ness of deep learning models has raised the need for devising strategies to explain the decision process of these models, leading to the creation of the topic of eXplainable Artificial Intelligence (XAI). In this context, we provide a thorough survey of XAI applied to medical diagnosis, including visual, textual, and example-based explanation methods. Moreover, this work reviews the existing medical imaging datasets and the existing metrics for evaluating the quality of the explanations . Complementary to most existing surveys, we include a performance comparison among a set of report generation-based methods. Finally, the major challenges in applying XAI to medical imaging are also discussed.
Understanding the Capability of PD Control for Uncertain Stochastic Systems

In this article, we focus on the global stabilizability problem for a class of uncertain stochastic control systems, where both the drift term and the diffusion term are nonlinear functions of the state variables and the control variables. We will show that the widely applied proportional–derivative (PD) control in engineering practice has the ability to globally stabilize such systems in the mean square sense, provided that the upper bounds of partial derivatives of the nonlinear functions satisfy a certain algebraic inequality. It will also be proved that the stabilizing PD parameters can only be selected from a two dimensional bounded convex set, which differs significantly from the existing results for PD controlled stochastic systems where the diffusion term does not depend on the control variables. Moreover, a particular polynomial on these bounds is introduced, which can be used to determine under what conditions the system is not stabilizable by the PD control, and thus demonstrating the fundamental limitations of PD control.
Spatio-Temporal Transformer for Dynamic Facial Expression Recognition in the Wild

Previous methods for dynamic facial expression in the wild are mainly based on Convolutional Neural Networks (CNNs), whose local operations ignore the long-range dependencies in videos. To solve this problem, we propose the spatio-temporal Transformer (STT) to capture discriminative features within each frame and model contextual relationships among frames. Spatio-temporal dependencies are captured and integrated by our unified Transformer. Specifically, given an image sequence consisting of multiple frames as input, we utilize the CNN backbone to translate each frame into a visual feature sequence. Subsequently, the spatial attention and the temporal attention within each block are jointly applied for learning spatio-temporal representations at the sequence level. In addition, we propose the compact softmax cross entropy loss to further encourage the learned features have the minimum intra-class distance and the maximum inter-class distance. Experiments on two in-the-wild dynamic facial expression datasets (i.e., DFEW and AFEW) indicate that our method provides an effective way to make use of the spatial and temporal dependencies for dynamic facial expression recognition. The source code and the training logs will be made publicly available.
Circumnuclear Medium around the Central AGN in a Cool-core Cluster, Abell 1644-South

We present the circumnuclear multiphase gas properties of the brightest cluster galaxy (BCG) in the center of Abell 1644-South (A1644-S). A1644-S is the main cluster in a merging system, which is well known for X-ray hot gas sloshing in its core. The sharply peaked X-ray profile of A1644-S implies the presence of a strongly cooling gas core. In this study, we analyze ALMA 12CO (1–0) data, JVLA H i data, and KaVA 22 GHz data for the central region of A1644-S to probe the potential origin of the cool gas and its role in (re)powering the central active galactic nucleus (AGN). We find CO clumps distributed in an arc shape along the X-ray gas sloshing, which is suggestive of a connection between the cold gas and the hot intracluster medium (ICM). H i and CN are detected in absorption against the AGN continuum emission. The absorption dip is observed at the systemic velocity of the BCG with an extended, redshifted tail. Based on the spatial and spectral configurations of the H i, CN, and CO gases, it is inferred that cool gas spirals into the core of the BCG, which is then fed to the central AGN. Indeed, our KaVA observation reveals a parsec-scale bipolar jet, implying that this AGN could have been (re)powered quite recently. Combining this, we suggest that some cold gas in A1644-S could have been formed from the cooling of the ICM, triggering the activity of the central AGN in the early development of a cool-core cluster.
Determinants of e-learning adoption among students of developing countries

Purpose – The purpose of this paper is to investigate the determinants of e-learning adoption (ELA) among students in the University of Ghana. Design/methodology/approach – A quantitative research approach comprising of a survey of 337 students was adopted. Data were collected using questionnaires designed in conjunction with 13 factors (computer self-efficacy (CSE), perceived ease of use (PEOU), perceived usefulness (PU) and attitude towards use (ATTU)) in the conceptual model of the study. Data analysis was conducted using structural equation modelling. Findings – The result revealed that PU and ATTU had a direct effect on ELA whilst, PU and PEOU also had a direct relationship on ATTU. Other variables such as CSE and PEOU had an indirect relationship on ELA though they were found to have an insignificant direct relationship on ELA. Practical implications – The level of significance of each construct identified in the study provide practical guidance to school administrators and instructors as to which factors to pay close attention to when implementing e-learning projects within their respective institutions. Originality/value – This study provides insight into ELA from the students’ perspective, through an extension of the TAM model in a developing country context given the existence of cultural differences and societal idiosyncrasies which exist in different contexts, particularly in Africa.
Controlling magnetic exchange and anisotropy by non-magnetic ligand substitution in layered MPX3 (M = Ni, Mn; X = S, Se)

Recent discoveries in two-dimensional (2D) magnetism have intensified the investigation of van der Waals (vdW) magnetic materials and further improved our ability to tune their magnetic properties. Tunable magnetism has been widely studied in antiferromagnetic metal thiophosphates M P X 3 . Substitution of metal ions M has been adopted as an important technique to engineer the magnetism in M P X 3 . In this work, we have studied the previously unexplored chalcogen X substitutions in M P X 3 ( M = Mn/Ni; X = S/Se). We synthesized the single crystals of MnPS 3- x Se x (0 ≤ x ≤ 3) and NiPS 3- x Se x (0 ≤ x ≤ 1.3) and investigated the systematic evolution of the magnetism with varying x . Our study reveals the effective tuning of magnetic interactions and anisotropies in both MnPS 3 and NiPS 3 upon Se substitution. Such efficient engineering of the magnetism provides a suitable platform to understand the low-dimensional magnetism and develop future magnetic devices.
Matrix Waring Problem -- II

We prove that for all integers $k \geq 1$, there exists a constant $C_k$ depending only on $k$ such that for all $q>C_k$ and for all $n \geq 1$ every matrix in $M_n(\mathbb F_q)$ is a sum of two $k$th powers.
Modeling Electron Acceleration and Transport in the Early Impulsive Phase of the 2017 September 10th Solar Flare

The X8.2-class limb flare on 2017 September 10 is among the best studied solar flare events owing to its great similarity to the standard flare model and the broad coverage by multiple spacecraft and ground-based observations. These multiwavelength observations indicate that electron acceleration and transport are efficient in the reconnection and flare looptop regions. However, there lacks a comprehensive model for explaining and interpreting the multi-faceted observations. In this work, we model the electron acceleration and transport in the early impulsive phase of this flare. We solve the Parker transport equation that includes the primary acceleration mechanism during magnetic reconnection in the large-scale flare region modeled by MHD simulations. We find that electrons are accelerated up to several MeV and fill a large volume of the reconnection region, similar to the observations shown in microwaves. The electron spatial distribution and spectral shape in the looptop region agree well with those derived from the microwave and hard X-ray emissions before magnetic islands grow large and dominate the acceleration. Future emission modelings using the electron maps will enable direct comparison with microwave and hard X-ray observations. These results shed new light on the electron acceleration and transport in a broad region of solar flares within a data-constrained realistic flare geometry.
A Tentative 114 minute Orbital Period Challenges the Ultracompact Nature of the X-Ray Binary 4U 1812–12

We present a detailed time-resolved photometric study of the ultracompact X-ray binary candidate 4U 1812–12. The multicolor light curves obtained with HiPERCAM on the 10.4 m Gran Telescopio Canarias show a ≃114 minute modulation similar to a superhump. Under this interpretation, this period should lie very close to the orbital period of the system. Contrary to what its other observational properties suggest (namely, persistent dim luminosity, low optical-to-X-ray flux ratio, and lack of hydrogen features in the optical spectrum), this implies that 4U 1812–12 is most likely not an ultracompact X-ray binary, which is usually defined as a system with an orbital period lower than 80 minutes. We discuss the nature of the system, showing that a scenario in which 4U 1812–12 is the progenitor of an ultracompact X-ray binary may reconcile all the observables.
Comparing value of travel time and value of travel time saving with heterogeneity in travelers

In research on the value of past time, the value of travel time and the value of saving travel time are two different concepts that have been vaguely distinguished over an extended period of time. This paper applies the theory of perspectives to discuss differences in the value of travel time and savings in travel time between different respondent groups and under different models. To this end, this paper designs an RP-SP questionnaire for urban travel behaviour, and collects data on the travel preferences of 409 Guangzhou residents. By introducing potential profiling to capture the effects of heterogeneity between classes, the respondent group is divided into three categories according to its individual socio-economic characteristics such as age, income, educational attainment, etc. After that, the utility function of travelers is established, and the MNL model, MIXL model, S-MNL model, G-MNL model, LCL model and MM-MNL model are selected for the total sample and three types of groups to estimate the time value. This paper emphasizes the definitions of VTT and VTTS, and presents a method for estimating VTT and VTTS using LPA to identify traveler heterogeneity.
Controlling Extra-Textual Attributes about Dialogue Participants: A Case Study of English-to-Polish Neural Machine Translation

Unlike English, morphologically rich languages can reveal characteristics of speakers or their conversational partners, such as gender and number, via pronouns, morphological endings of words and syntax. When translating from English to such languages, a machine translation model needs to opt for a certain interpretation of textual context, which may lead to serious translation errors if extra-textual information is unavailable. We investigate this challenge in the English-to-Polish language direction. We focus on the underresearched problem of utilising external metadata in automatic translation of TV dialogue, proposing a case study where a wide range of approaches for controlling attributes in translation is employed in a multi-attribute scenario. The best model achieves an improvement of +5.81 chrF++/+6.03 BLEU, with other models achieving competitive performance. We additionally contribute a novel attribute-annotated dataset of Polish TV dialogue and a morphological analysis script used to evaluate attribute control in models.
Long Dark Gaps in the Lyβ Forest at z < 6: Evidence of Ultra-late Reionization from XQR-30 Spectra

We present a new investigation of the intergalactic medium near reionization using dark gaps in the Lyβ forest. With its lower optical depth, Lyβ offers a potentially more sensitive probe to any remaining neutral gas compared to the commonly used Lyα line. We identify dark gaps in the Lyβ forest using spectra of 42 QSOs at z em > 5.5, including new data from the XQR-30 VLT Large Programme. Approximately 40% of these QSO spectra exhibit dark gaps longer than 10 h −1 Mpc at z ≃ 5.8. By comparing the results to predictions from simulations, we find that the data are broadly consistent both with models where fluctuations in the Lyα forest are caused solely by ionizing ultraviolet background fluctuations and with models that include large neutral hydrogen patches at z < 6 due to a late end to reionization. Of particular interest is a very long (L = 28 h −1 Mpc) and dark (τ eff ≳ 6) gap persisting down to z ≃ 5.5 in the Lyβ forest of the z = 5.85 QSO PSO J025−11. This gap may support late reionization models with a volume-weighted average neutral hydrogen fraction of 〈x H I〉 ≳ 5% by z = 5.6. Finally, we infer constraints on 〈x H I〉 over 5.5 ≲ z ≲ 6.0 based on the observed Lyβ dark gap length distribution and a conservative relationship between gap length and neutral fraction derived from simulations. We find 〈x H I〉 ≤ 0.05, 0.17, and 0.29 at z ≃ 5.55, 5.75, and 5.95, respectively. These constraints are consistent with models where reionization ends significantly later than z = 6.