Miracle-dz/UniFAR-FTUs huggingface dataset:
Graph Conditioned Sparse-Attention for Improved Source Code Understanding

Transformer architectures have been successfully used in learning source code representations. The fusion between a graph representation like Abstract Syntax Tree (AST) and a source code sequence makes the use of current approaches computationally intractable for large input sequence lengths. Source code can have long-range dependencies that require larger sequence lengths to model effectively. Current approaches have a quadratic growth in computational and memory costs with respect to the sequence length. Using such models in practical scenarios is difficult. In this work, we propose the conditioning of a source code snippet with its graph modality by using the graph adjacency matrix as an attention mask for a sparse self-attention mechanism and the use of a graph diffusion mechanism to model longer-range token dependencies. Our model reaches state-of-the-art results in BLEU, METEOR, and ROUGE-L metrics for the code summarization task and near state-of-the-art accuracy in the variable misuse task. The memory use and inference time of our model have linear growth with respect to the input sequence length as compared to the quadratic growth of previous works.
DPRK-BERT: The Supreme Language Model

Deep language models have achieved remarkable success in the NLP domain. The standard way to train a deep language model is to employ unsupervised learning from scratch on a large unlabeled corpus. However, such large corpora are only available for widely-adopted and high-resource languages and domains. This study presents the first deep language model, DPRK-BERT, for the DPRK language. We achieve this by compiling the first unlabeled corpus for the DPRK language and fine-tuning a preexisting the ROK language model. We compare the proposed model with existing approaches and show significant improvements on two DPRK datasets. We also present a cross-lingual version of this model which yields better generalization across the two Korean languages. Finally, we provide various NLP tools related to the DPRK language that would foster future research.
Consistent Analysis of the AGN LF in X-Ray and MIR in the XMM-LSS Field

The luminosity function of active galactic nuclei (AGN) probes the history of supermassive black hole assembly and growth across cosmic time. To mitigate selection biases, we present a consistent analysis of the AGN luminosity functions (LFs) derived for both X-ray and mid-infrared (MIR) selected AGN in the XMM-Large Scale Structure field. There are 4268 AGN used to construct the MIR luminosity function (IRLF) and 3427 AGN used to construct the X-ray luminosity function (XLF), providing the largest census of the AGN population out to z = 4 in both bands with significant reduction in uncertainties. We are able for the first time to see the knee of the IRLF at z > 2 and observe a flattening of the faint-end slope as redshift increases. The bolometric luminosity density, a proxy for the cosmic black hole accretion history, computed from our LFs, shows a peak at z ≈ 2.25, consistent with recent estimates of the peak in the star formation rate density (SFRD). However, at earlier epochs, the AGN luminosity density is flatter than the SFRD. If confirmed, this result suggests that the build up of black hole mass outpaces the growth of stellar mass in high-mass systems at z ≳ 2.5. This is consistent with observations of redshift z ∼ 6 quasars that lie above the local M − σ relationship. The luminosity density derived from the IRLF is higher than that from the XLF at all redshifts. This is consistent with the dominant role of obscured AGN activity in the cosmic growth of supermassive black holes.
Forward Operator Estimation in Generative Models with Kernel Transfer Operators

Generative models (e.g., variational autoencoders, flow-based generative models, GANs) usually involve finding a mapping from a known distribution, e.g. Gaussian, to an estimate of the unknown data-generating distribution. This process is often carried out by searching over a class of non-linear functions (e.g., representable by a deep neural network). While effective in practice, the associated runtime/memory costs can increase rapidly, and will depend on the performance desired in an application. We propose a much cheaper (and simpler) strategy to estimate this mapping based on adapting known results in kernel transfer operators. We show that if some compromise in functionality (and scalability) is acceptable, our proposed formulation enables highly efficient distribution approximation and sampling, and offers surprisingly good empirical performance which compares favorably with powerful baselines.
Push Stricter to Decide Better: A Class-Conditional Feature Adaptive Framework for Improving Adversarial Robustness

In response to the threat of adversarial examples, adversarial training provides an attractive option for improving robustness by training models on online-augmented adversarial examples. However, most existing adversarial training methods focus on improving the model’s robust accuracy by strengthening the adversarial examples but neglecting the increasing shift between natural data and adversarial examples, leading to a decrease in natural accuracy. To maintain the trade-off between natural and robust accuracy, we alleviate the shift from the perspective of feature adaption and propose a Feature Adaptive Adversarial Training (FAAT) optimizing the class-conditional feature adaption across natural data and adversarial examples. Specifically, we propose to incorporate a class-conditional discriminator to encourage the features to become (1) class-discriminative and (2) invariant to the change of adversarial attacks. The novel FAAT framework enables the trade-off between natural and robust accuracy by generating features with similar distribution across natural and adversarial data within the same class and achieves higher overall robustness benefiting from the class-discriminative feature characteristics. Experiments on various datasets demonstrate that FAAT produces more discriminative features and performs favorably against state-of-the-art methods.
Measurements of Proton High-Order Cumulants in sqrt[s_{NN}]=3  GeV Au+Au Collisions and Implications for the QCD Critical Point.

We report cumulants of the proton multiplicity distribution from dedicated fixed-target Au+Au collisions at sqrt[s_{NN}]=3.0 GeV, measured by the STAR experiment in the kinematic acceptance of rapidity (y) and transverse momentum (p_{T}) within -0.5<y<0 and 0.4<p_{T}<2.0 GeV/c. In the most central 0%-5% collisions, a proton cumulant ratio is measured to be C_{4}/C_{2}=-0.85±0.09 (stat)±0.82 (syst), which is 2σ below the Poisson baseline with respect to both the statistical and systematic uncertainties. The hadronic transport UrQMD model reproduces our C_{4}/C_{2} in the measured acceptance. Compared to higher energy results and the transport model calculations, the suppression in C_{4}/C_{2} is consistent with fluctuations driven by baryon number conservation and indicates an energy regime dominated by hadronic interactions. These data imply that the QCD critical region, if created in heavy-ion collisions, could only exist at energies higher than 3 GeV.
Collective discrete optimisation as judgment aggregation

Many important collective decision-making problems can be seen as multi-agent versions of discrete optimisation problems. Participatory budgeting, for instance, is the collective version of the knapsack problem; other examples include collective scheduling, and collective spanning trees. Rather than developing a specific model, as well as specific algorithmic techniques, for each of these problems, we propose to represent and solve them in the unifying framework of judgment aggregation with weighted issues. We provide a modular definition of collective discrete optimisation (CDO) rules based on coupling a set scoring function with an operator, and we show how they generalise several existing procedures developed for specific CDO problems. We also give an implementation based on integer linear programming (ILP) and test it on the problem of collective spanning trees.
Effectiveness of Fire Safety Components and Systems

A concept of fire safety system effectiveness incorporating two elements, efficacy and reliability, is introduced. Historical USA fire data is used to estimate the levels of effectiveness of several fire safety systems in terms of extent of flame spread, the rates of firefighter injuries and civilian fatalities and injuries, and estimated $ loss in fires reported to fire brigades. Based on the data analyzed, sprinklers appear to have greater effectiveness than detectors and protected construction for reducing extent of flame spread and generally (but not always) for reducing the rate of firefighter injuries, civilian fatalities and the estimated $ loss per fire.
Integration of visual and haptic feedback for teleoperation

﻿ Teleoperation systems are an important tool for performing tasks which require the sensori-motor coordination of an operator but where it is physically impossible for an operator to undertake such tasks in situ . The vast majority of these devices supply the operator with both visual and haptic sensory feedback in order that the operator can perform the task at hand as naturally and fluently as possible and as though physically present at the remote site. This thesis is concerned with overcoming the sensory limitations imposed by a fixed camera teleoperation system. The principal aim of this work is the extraction and redisplay of visual information to facilitate such a system. The thesis augments the Oxford teleoperation system with a virtual viewing module, where the operator is able to select his or her viewpoint and viewing direction onto the workcell by first tracking the locations of known objects in the workcell using a computer vision system, and then rendering them graphically on a display in front of the operator. This system, because the model-based object tracker is based around a Kalman filter, motivates the design of experiments to examine whether the operator's visuo-motor control loop maintains a state model of the manipulation process as in a Kalman filter. Experimental evidence is presented showing the latter to be false. A new operator model is then postulated using an adaptive gain controller, with the gain chosen to minimise the variance between desired and actual output. The experimental evidence supports this model. These findings support the hypothesis that the required bandwidth of the tracking filter is both i) sufficient that the tracker can robustly track hand manipulated objects and ii) matched to the visual needs of the operator.
Orbital-Selective High-Temperature Cooper Pairing Developed in the Two-Dimensional Limit.

For multiband superconductors, the orbital multiplicity yields orbital differentiation in normal-state properties and can lead to orbital-selective spin-fluctuation Cooper pairing. The orbital-selective phenomenon has become increasingly pivotal in clarifying the pairing "enigma", particularly for multiband high-temperature superconductors. Meanwhile, in one-unit-cell (1-UC) FeSe/SrTiO3, since the standard electron-hole Fermi pocket nesting scenario is inapplicable, the actual pairing mechanism is subject to intense debate. Here, by measuring high-resolution Bogoliubov quasiparticle interference, we report observations of highly anisotropic magnetic Cooper pairing in 1-UC FeSe. Theoretically, it is important to incorporate orbitally selective effects of electronic correlations within a spin-fluctuation pairing calculation, where the dxy orbital becomes coherence-suppressed. The resulting pairing gap is compatible with the experimental findings, which suggests that high-Tc Cooper pairing with orbital selectivity applies to 2D-limit 1-UC FeSe. Our findings imply the general existence of orbital selectivity in iron-based superconductors and the universal significance of electron correlations in high-Tc superconductors.
Effect of MHD Wind-driven Disk Evolution on the Observed Sizes of Protoplanetary Disks

It is still unclear whether the evolution of protoplanetary disks, a key ingredient in the theory of planet formation, is driven by viscous turbulence or magnetic disk winds. As viscously evolving disks expand outward over time, the evolution of disk sizes is a discriminant test for studying disk evolution. However, it is unclear how the observed disk size changes over time if disk evolution is driven by magnetic disk winds. Combining the thermo-chemical code DALI with the analytical wind-driven disk-evolution model presented in Tabone et al., we study the time evolution of the observed gas outer radius as measured from CO rotational emission (R CO,90%). The evolution of R CO,90% is driven by the evolution of the disk mass, as the physical radius stays constant over time. For a constant α DW , an extension of the α Shakura–Sunyaev parameter to wind-driven accretion, R CO,90% decreases linearly with time. Its initial size is set by the disk mass and the characteristic radius R c,0, but only R c,0 affects the evolution of R CO,90%, with a larger R c,0 resulting in a steeper decrease of R CO,90%. For a time-dependent α DW , R CO,90% stays approximately constant during most of the disk lifetime until R CO,90% rapidly shrinks as the disk dissipates. The constant α DW models are able to reproduce the observed gas disk sizes in the ∼1–3 Myr old Lupus and ∼5–11 Myr old Upper Sco star-forming regions. However, they likely overpredict the gas disk size of younger (⪅0.7 Myr) disks.
Extrapolating from a Single Image to a Thousand Classes using Distillation

What can neural networks learn about the visual world from a single image? While it obviously cannot contain the multitudes of possible objects, scenes and lighting conditions that exist – within the space of all possible 256 3 · 224 · 224 224 -sized square images, it might still provide a strong prior for natural images. To analyze this hypothesis, we develop a framework for training neural networks from scratch us-ing a single image by means of knowledge distillation from a supervised pretrained teacher. With this, we ﬁnd that the answer to the above question is: ‘surprisingly, a lot’. In quantitative terms, we ﬁnd top-1 accuracies of 94% / 74% on CIFAR-10/100, 59 % on ImageNet and, by extending this method to audio, 84 % on SpeechCommands. In extensive analyses we disentangle the effect of augmentations, choice of source image and network architectures and also discover “panda neurons” in networks that have never seen a panda. This work shows that one image can be used to extrapolate to thousands of object classes and motivates a renewed research agenda on the fundamental interplay of augmentations and images. Webpage: 1
Public Data-Assisted Mirror Descent for Private Model Training

In this paper, we revisit the problem of using in-distribution public data to improve the privacy/utility trade-offs for differentially private (DP) model training. (Here, public data refers to auxiliary data sets that have no privacy concerns.) We design a natural variant of DP mirror descent, where the DP gradients of the private/sensitive data act as the linear term, and the loss generated by the public data as the mirror map. We show that, for linear regression with feature vectors drawn from a non-isotropic sub-Gaussian distribution, our algorithm, PDA-DPMD (a variant of mirror descent), provides population risk guarantees that are asymptotically better than the best known guarantees under DP (without having access to public data), when the number of public data samples ($n_{\sf pub}$) is sufficiently large. We further show that our algorithm has natural"noise stability"properties that control the variance due to noise added to ensure DP. We demonstrate the efficacy of our algorithm by showing privacy/utility trade-offs on four benchmark datasets (StackOverflow, WikiText-2, CIFAR-10, and EMNIST). We show that our algorithm not only significantly improves over traditional DP-SGD, which does not have access to public data, but to our knowledge is the first to improve over DP-SGD on models that have been pre-trained with public data.
Pocket-type prestressed brickwork retaining walls

This thesis presents the results of a study into the behaviour of posttensioned pocket type brickwork retaining walls. An analytical and experimental study was carried out to examine the behaviour of the wall up to failure. The programme of work considered the effect of the following parameters on the performance of the wall:j) Vertical concentrated eccentric load. Percentage area of steel. Pocket spacing and wall slenderness. Type of wall bond. The effect of the last three parameters on deflection, cracking and ultimate load were studied. The following experimental work was carried out to examine the above parameters I) Nine full-scale six course prisms consisting of three with open pocket, three with concrete infill and three solid brickwork to be tested in axial compression Two full scale beams with the bed joints running parallel to the direction of the induced prestressing load. Six full scale beams/slabs with the bed joints running perpendicular to the direction of the induced prestressing load. Six half-scale retaining walls. V) A comprehensive series of control tests, on mortar and grout specimens, single course prisms and small wallets, were undertaken to determine the non-linear deformation characteristics and compressive strengths of brickwork masonry. The theoretical investigation was based on the following theories:Finite Element Analysis Direct Method Stress Block Analysis Yield Line Analysis The results of the analyses were compared with those based on the Code of Practice, B.S. 5628, Part 2, 1985. A computer program was written in Fortran to predict the ultimate moment of the wall panels, using predicted equilibrium equations. Good agreement was found between the theoretical and experimental results. The results show that post-tensioned pocket type brickwork retaining walls have a large nominal strength, largely due to the presence of prestressing forces and the behaviour of the walls as homogenous cantilevers. The most effective pocket spacing was found to be h13, and the maximum spacing should be limited to give an aspect ratio which is greater than 1.15. The study confirms the applicability of prestressed brick masonry for structures such as slabs and retaining walls irrespective of the type of brickwork bond. VI
The Estimation of Fundamental Physics Parameters for Fermi-LAT Blazars

Aiming to delineate the physical framework of blazars, we present an effective method to estimate four important parameters based on the idea proposed by Becker & Kafatos, including the upper limit of central black hole mass M, the Doppler factor δ, the distance along the axis to the site of the γ-ray production d (which then can be transformed into the location of γ-ray-emitting region R γ ) and the propagation angle with respect to the axis of the accretion disk Φ. To do so, we adopt an identical sample with 809 Fermi-LAT-detected blazars which had been compiled in Pei et al. These four derived parameters stepping onto the stage may shed new light on our knowledge regarding γ-ray blazars. With regard to the paper of Becker & Kafatos, we obtain several new perspectives, mainly in (1) putting forward an updated demarcation between BL Lacs and FSRQs based on the relation between broad-line region luminosity and disk luminosity both measured in Eddington units, i.e., L disk/L Edd = 4.68 × 10−3, indicating that there are some differences between BL Lacs and FSRQs on the accretion power in the disk; (2) proposing that there is a so-called “appareling zone,” a potential transition field between BL Lacs and FSRQs where the changing-look blazars perhaps reside; (3) the location of γ-ray emission region is principally constrained outside the broad-line region, and for some BL Lacs are also away from the dusty molecular torus, which means the importance of emission components in the jet.
Dyna-bAbI: unlocking bAbI’s potential with dynamic synthetic benchmarking

While neural language models often perform surprisingly well on natural language understanding (NLU) tasks, their strengths and limitations remain poorly understood. Controlled synthetic tasks are thus an increasingly important resource for diagnosing model behavior. In this work we focus on story understanding, a core competency for NLU systems. However, the main synthetic resource for story understanding, the bAbI benchmark, lacks such a systematic mechanism for controllable task generation. We develop Dyna-bAbI, a dynamic framework providing fine-grained control over task generation in bAbI. We demonstrate our ideas by constructing three new tasks requiring compositional generalization, an important evaluation setting absent from the original benchmark. We tested both special-purpose models developed for bAbI as well as state-of-the-art pre-trained methods, and found that while both approaches solve the original tasks (99{% accuracy), neither approach succeeded in the compositional generalization setting, indicating the limitations of the original training data.We explored ways to augment the original data, and found that though diversifying training data was far more useful than simply increasing dataset size, it was still insufficient for driving robust compositional generalization (with 70{% accuracy for complex compositions). Our results underscore the importance of highly controllable task generators for creating robust NLU systems through a virtuous cycle of model and data development.
Transmission of 24-Gb/s PAM-4 Over 150-km SSMF using a Driverless Silicon Microring Modulator

A 24-Gb/s PAM-4 signal is generated using a driverless silicon microring modulator, bit error rate of 3.8e-4 is achieved after transmission over 150km SSMF, proving that the low cost and low power consumption silicon microring modulators may be candidate in high-capacity modules with advanced modulation.
Structure-Aware Label Smoothing for Graph Neural Networks

Representing a label distribution as a one-hot vector is a common practice in training node classification models. However, the one-hot representation may not adequately reflect the semantic characteristics of a node in different classes, as some nodes may be semantically close to their neighbors in other classes. It would cause over-confidence since the models are encouraged to assign full probabilities when classifying every node. While training models with label smoothing can ease this problem to some degree, it still fails to capture the nodes' semantic characteristics implied by the graph structures. In this work, we propose a novel SALS (\textit{Structure-Aware Label Smoothing}) method as an enhancement component to popular node classification models. SALS leverages the graph structures to capture the semantic correlations between the connected nodes and generate the structure-aware label distribution to replace the original one-hot label vectors, thus improving the node classification performance without inference costs. Extensive experiments on seven node classification benchmark datasets reveal the effectiveness of our SALS on improving both transductive and inductive node classification. Empirical results show that SALS is superior to the label smoothing method and enhances the node classification models to outperform the baseline methods.