Miracle-dz/UniFAR-FTUs huggingface dataset:
An Intelligent End-to-End Neural Architecture Search Framework for Electricity Forecasting Model Development

: Recent years have witnessed an exponential growth in developing deep learning (DL) models for the time-series electricity forecasting in power systems. However, most of the proposed models are designed based on the designers' inherent knowledge and experience without elaborating on the suitability of the proposed neural architectures. Moreover, these models cannot be self-adjusted to the dynamically changing data patterns due to an inflexible design of their structures. Even though several latest studies have considered application of the neural architecture search (NAS) technique for obtaining a network with an optimized structure in the electricity forecasting sector, their training process is quite time-consuming, computationally expensive and not intelligent, indicating that the NAS application in electricity forecasting area is still at an infancy phase. In this research study, we propose an intelligent automated architecture search (IAAS) framework for the development of time-series electricity forecasting models. The proposed framework contains two primary components, i.e., network function-preserving transformation operation and reinforcement learning (RL)-based network transformation control. In the first component, we introduce a theoretical function-preserving transformation of recurrent neural networks (RNN) to the literature for capturing the hidden temporal patterns within the time-series data. In the second component, we develop three RL-based transformation actors and a net pool to intelligently and effectively search a high-quality neural architecture. After conducting comprehensive experiments on two publicly-available electricity load datasets and two wind power datasets, we demonstrate that the proposed IAAS framework significantly outperforms the ten existing models or methods in terms of forecasting accuracy and stability. Finally, we perform an ablation experiment to showcase the importance of critical components of the proposed IAAS framework in improving the forecasting accuracy and reducing the structural complexity of the model. We performed our experiments based on the time-series data to build forecasting models of electricity load demand and wind power generation in power systems. For the electricity load forecasting
AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling

Neural fields such as implicit surfaces have recently enabled avatar modeling from raw scans without explicit temporal correspondences. In this work, we exploit autoregressive modeling to further extend this notion to capture dynamic effects, such as soft-tissue deformations. Although autoregressive models are naturally capable of handling dynamics, it is non-trivial to apply them to implicit representations, as explicit state decoding is infeasible due to prohibitive memory requirements. In this work, for the first time, we enable autoregressive modeling of implicit avatars. To reduce the memory bottleneck and efficiently model dynamic implicit surfaces, we introduce the notion of articulated observer points, which relate implicit states to the explicit surface of a parametric human body model. We demonstrate that encoding implicit surfaces as a set of height fields defined on articulated observer points leads to significantly better generalization compared to a latent representation. The experiments show that our approach outperforms the state of the art, achieving plausible dynamic deformations even for unseen motions. https://zqbai-jeremy.github. io/autoavatar
NuSTAR Observations of 52 Compton-thick Active Galactic Nuclei Selected by the Swift/Burst Alert Telescope All-sky Hard X-Ray Survey

We present systematic broadband X-ray spectral analysis of 52 Compton-thick ( 24≤logNHLOS/cm−2 ) active galactic nucleus (CTAGN) candidates selected by the Swift/Burst Alert Telescope all-sky hard X-ray survey observed with Chandra, X-ray Multi-Mirror Mission-Newton (XMM-Newton), Swift/X-Ray Telescope, Suzaku, and NuSTAR. The XMM-Newton data of 10 objects and the NuSTAR data of 15 objects are published for the first time. We use an X-ray spectral model from a clumpy torus (XClumpy) to determine the torus properties. As a result, the hydrogen column density along the line of sight NHLOS obtained from the XClumpy model indicates that 24 objects are Compton-thin AGNs and 28 objects are CTAGNs in a 90% confidence interval. The main reason is the difference in the torus model applied. The hydrogen column density along the equatorial direction NHEqu of CTAGNs inferred from the XClumpy model is larger than that of less obscured AGNs. The Compton-thin torus covering factor C 22 obtained from the XClumpy model is consistent with that of Ricci et al. in the low Eddington ratio ( logREdd≤−1.0 ), whereas C 22 inferred from the XClumpy model is larger than that of Ricci et al. in the high Eddington ratio ( −1.0≤logREdd ). The average value of the Compton-thick torus covering factor C 24 obtained from the XClumpy model is 36−4+4 %. This value is larger than that of Ricci et al. ( C24≃27−4+4 %) based on the assumption that all AGNs have intrinsically the same torus structure. These results suggest that the structure of CTAGNs may be intrinsically different from that of less obscured AGNs.
Apparent anomalous diffusion and non-Gaussian distributions in a simple mobile–immobile transport model with Poissonian switching

We analyse mobile–immobile transport of particles that switch between the mobile and immobile phases with finite rates. Despite this seemingly simple assumption of Poissonian switching, we unveil a rich transport dynamics including significant transient anomalous diffusion and non-Gaussian displacement distributions. Our discussion is based on experimental parameters for tau proteins in neuronal cells, but the results obtained here are expected to be of relevance for a broad class of processes in complex systems. Specifically, we obtain that, when the mean binding time is significantly longer than the mean mobile time, transient anomalous diffusion is observed at short and intermediate time scales, with a strong dependence on the fraction of initially mobile and immobile particles. We unveil a Laplace distribution of particle displacements at relevant intermediate time scales. For any initial fraction of mobile particles, the respective mean squared displacement (MSD) displays a plateau. Moreover, we demonstrate a short-time cubic time dependence of the MSD for immobile tracers when initially all particles are immobile.
CNN LEGO: Disassembling and Assembling Convolutional Neural Network

Convolutional Neural Network (CNN), which mimics human visual perception mechanism, has been successfully used in many computer vision areas. Some psychophysical studies show that the visual perception mechanism synchronously processes the form, color, movement, depth, etc., in the initial stage [7,20] and then integrates all information for final recognition [38]. What's more, the human visual system [20] contains different subdivisions or different tasks. Inspired by the above visual perception mechanism, we investigate a new task, termed as Model Disassembling and Assembling (MDA-Task), which can disassemble the deep models into independent parts and assemble those parts into a new deep model without performance cost like playing LEGO toys. To this end, we propose a feature route attribution technique (FRAT) for disassembling CNN classifiers in this paper. In FRAT, the positive derivatives of predicted class probability w.r. t. the feature maps are adopted to locate the critical features in each layer. Then, relevance analysis between the critical features and preceding/subsequent parameter layers is adopted to bridge the route between two adjacent parameter layers. In the assembling phase, class-wise components of each layer are assembled into a new deep model for a specific task. Extensive experiments demonstrate that the assembled CNN classifier can achieve close accuracy with the original classifier without any fine-tune, and excess original performance with one-epoch fine-tune. What's more, we also conduct massive experiments to verify the broad application of MDA-Task on model decision route visualization, model compression, knowledge distillation, transfer learning, incremental learning, and so on.
Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap

Recently, contrastive learning has risen to be a promising approach for large-scale self-supervised learning. However, theoretical understanding of how it works is still unclear. In this paper, we propose a new guarantee on the downstream performance without resorting to the conditional independence assumption that is widely adopted in previous work but hardly holds in practice. Our new theory hinges on the insight that the support of different intra-class samples will become more overlapped under aggressive data augmentations, thus simply aligning the positive samples (augmented views of the same sample) could make contrastive learning cluster intra-class samples together. Based on this augmentation overlap perspective, theoretically, we obtain asymptotically closed bounds for downstream performance under weaker assumptions, and empirically, we propose an unsupervised model selection metric ARC that aligns well with downstream accuracy. Our theory suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and the overlapped augmented views (i.e., the chaos) create a ladder for contrastive learning to gradually learn class-separated representations. The code for computing ARC is available at https://github.com/zhangq327/ARC.
GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models

Deep learning (DL) techniques involving fine-tuning large numbers of model parameters have delivered impressive performance on the task of discriminating between language produced by cognitively healthy individuals, and those with Alzheimer’s disease (AD). However, questions remain about their ability to generalize beyond the small reference sets that are publicly available for research. As an alternative to fitting model parameters directly, we propose a novel method by which a Transformer DL model (GPT-2) pre-trained on general English text is paired with an artificially degraded version of itself (GPT-D), to compute the ratio between these two models’ perplexities on language from cognitively healthy and impaired individuals. This technique approaches state-of-the-art performance on text data from a widely used “Cookie Theft” picture description task, and unlike established alternatives also generalizes well to spontaneous conversations. Furthermore, GPT-D generates text with characteristics known to be associated with AD, demonstrating the induction of dementia-related linguistic anomalies. Our study is a step toward better understanding of the relationships between the inner workings of generative neural language models, the language that they produce, and the deleterious effects of dementia on human speech and language characteristics.
Answering Should Self-Publishing Video Game Developers Market Their Game on Twitter

In the marketing of video games made by self-publishing independent developers, there is common advice to market a game using the social media platform Twitter. However, the advice that stems from industry sources is somewhat contradictory, and many self-publishing independent developers elect not to use Twitter at all. This presents an opportunity for researchers to investigate this tension and determine if using Twitter does have a causal influence on the successful release of a game by using relatively recent developments in causal data science techniques. In this sense, this paper highlights these causal inference developments while simultaneously informing self-publishing independent developers whether they should indeed market their games using Twitter. It was found that using Twitter results in an average increase of 85.4 reviews during release week, corresponding to a 314% positive difference. Using Twitter also doubles the chance of reaching a critical 10-review inflection point threshold on release week. These effects, however, are moderated by the characteristics of the game, e xpressed as ‘tags’, yet in no case is the effect of Twitter use reduced to 0. Based on these findings, it is advised that new, self-publishing video game developers do use Twitter to market their games.
A singular Yamabe problem on manifolds with solid cones

We study the existence of conformal metrics on non-compact Riemannian manifolds with non-compact boundary, which are complete as metric spaces and have negative constant scalar curvature in the interior and negative constant mean curvature on the boundary. These metrics are constructed on smooth manifolds obtained by removing d-dimensional submanifolds from certain n-dimensional compact spaces locally modelled on generalized solid cones. We prove the existence of such metrics if and only if d>(n-2)/2. Our main theorem is inspired by the classical results by Aviles-McOwen and Loewner-Nirenberg known in the literature as the"singular Yamabe problem".
Relative energies without electronic perturbations via integral transform Relative energies without electronic perturbations via integral transform

We show that the energy of a perturbed system can be fully recovered from the unperturbed electron density. We derive an alchemical integral transform using a parametrization of space in terms of external potential changes, the chain rule and integration by parts. Within the radius of convergence, the zeroth order alone generates the energy expansion at all orders, narrowing the textbook statement that the p -th order wave function derivative is necessary to describe the (2 p + 1)-th energy derivative. We demonstrate the validity and numerical accuracy of this approach for hydrogen-like atoms, and multi-electron atoms with up to ten electrons.
Efficient-VDVAE: Less is more

Hierarchical VAEs have emerged in recent years as a reliable option for maximum likelihood estimation. However, instability issues and demanding computational requirements have hindered research progress in the area. We present simple modifications to the Very Deep VAE to make it converge up to $2.6\times$ faster, save up to $20\times$ in memory load and improve stability during training. Despite these changes, our models achieve comparable or better negative log-likelihood performance than current state-of-the-art models on all $7$ commonly used image datasets we evaluated on. We also make an argument against using 5-bit benchmarks as a way to measure hierarchical VAE's performance due to undesirable biases caused by the 5-bit quantization. Additionally, we empirically demonstrate that roughly $3\%$ of the hierarchical VAE's latent space dimensions is sufficient to encode most of the image information, without loss of performance, opening up the doors to efficiently leverage the hierarchical VAEs' latent space in downstream tasks. We release our source code and models at https://github.com/Rayhane-mamah/Efficient-VDVAE .
Visual-based Safe Landing for UAVs in Populated Areas: Real-time Validation in Virtual Environments

—Safe autonomous landing for Unmanned Aerial Vehicles (UAVs) in populated areas is a crucial aspect for successful urban deployment, particularly in emergency landing situations. Nonetheless, validating autonomous landing in real scenarios is a challenging task involving a high risk of injuring people. In this work, we propose a framework for real-time safe and thorough evaluation of vision-based autonomous landing in populated scenarios, using photo-realistic virtual environments. We propose to use the Unreal graphics engine coupled with the AirSim plugin for drone ´ s simulation, and evaluate autonomous landing strategies based on visual detection of Safe Landing Zones (SLZ) in populated scenarios. Then, we study two different criteria for selecting the “best” SLZ, and evaluate them during autonomous landing of a virtual drone in different scenarios and conditions, under different distributions of people in urban scenes, including moving people. We evaluate different metrics to quantify the performance of the landing strategies, establishing a baseline for comparison with future works in this challenging task, and analyze them through an important number of randomized iterations. The study suggests that the use of the autonomous landing algorithms considerably helps to prevent accidents involving humans, which may allow to unleash the full potential of drones in urban environments near to people.
Multiwavelength Variability of Sagittarius A* in 2019 July

We report a timing analysis of near-infrared (NIR), X-ray, and submillimeter data during a 3 day coordinated campaign observing Sagittarius A*. Data were collected at 4.5 μm with the Spitzer Space Telescope, 2–8 keV with the Chandra X-ray Observatory, 3–70 keV with NuSTAR, 340 GHz with ALMA, and 2.2 μm with the GRAVITY instrument on the Very Large Telescope Interferometer. Two dates show moderate variability with no significant lags between the submillimeter and the infrared at 99% confidence. A moderately bright NIR flare (F K ∼ 15 mJy) was captured on July 18 simultaneous with an X-ray flare (F 2−10 keV ∼ 0.1 counts s−1) that most likely preceded bright submillimeter flux (F 340 GHz ∼ 5.5 Jy) by about +34−33+14 minutes at 99% confidence. The uncertainty in this lag is dominated by the fact that we did not observe the peak of the submillimeter emission. A synchrotron source cooled through adiabatic expansion can describe a rise in the submillimeter once the synchrotron self-Compton NIR and X-ray peaks have faded. This model predicts high GHz and THz fluxes at the time of the NIR/X-ray peak and electron densities well above those implied from average accretion rates for Sgr A*. However, the higher electron density postulated in this scenario would be in agreement with the idea that 2019 was an extraordinary epoch with a heightened accretion rate. Since the NIR and X-ray peaks can also be fit by a nonthermal synchrotron source with lower electron densities, we cannot rule out an unrelated chance coincidence of this bright submillimeter flare with the NIR/X-ray emission.
Long time well-posedness of Whitham–Boussinesq systems

Consideration is given to three different full dispersion Boussinesq systems arising as asymptotic models in the bi-directional propagation of weakly nonlinear surface waves in shallow water. We prove that, under a non-cavitation condition on the initial data, these three systems are well-posed on a time scale of order O(1ε) , where ɛ is a small parameter measuring the weak nonlinearity of the waves. For one of the systems, this result is new even for short time. The two other systems involve surface tension, and for one of them, the non-cavitation condition has to be sharpened when the surface tension is small. The proof relies on suitable symmetrizers and the classical theory of hyperbolic systems. However, we have to track the small parameters carefully in the commutator estimates to get the long time well-posedness. Finally, combining our results with the recent ones of Emerald provide a full justification of these systems as water wave models in a larger range of regimes than the classical (a, b, c, d)-Boussinesq systems.
A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training

Adversarial Training (AT) is known as an effective approach to enhance the robustness of deep neural networks. Recently researchers notice that robust models with AT have good generative ability and can synthesize realistic images, while the reason behind it is yet under-explored. In this paper, we demystify this phenomenon by developing a unified probabilistic framework, called Contrastive Energy-based Models (CEM). On the one hand, we provide the first probabilistic characterization of AT through a unified understanding of robustness and generative ability. On the other hand, our unified framework can be extended to the unsupervised scenario, which interprets unsupervised contrastive learning as an important sampling of CEM. Based on these, we propose a principled method to develop adversarial learning and sampling methods. Experiments show that the sampling methods derived from our framework improve the sample quality in both supervised and unsupervised learning. Notably, our unsupervised adversarial sampling method achieves an Inception score of 9.61 on CIFAR-10, which is superior to previous energy-based models and comparable to state-of-the-art generative models.
Design and implementation of a wireless passive sensing system for structural health monitoring

DESIGN AND IMPLEMENTATION OF A WIRELESS PASSIVE SENSING SYSTEM FOR STRUCTURAL HEALTH MONITORING Burak Özbey Ph.D. in Electrical and Electronics Engineering Advisors: Ayhan Altıntaş, Hilmi Volkan Demir, Vakur B. Ertürk June 2016 Structural health monitoring (SHM) aims to ensure detection and prevention of damage in structures and protection of human life via observation of certain damage indicators. In SHM, one of the most important damage indicators is the strain forming on the steel reinforcing bars (rebars) embedded inside concrete. This strain can slowly develop over time, or can suddenly occur due to an overload such as an earthquake. In this dissertation, a novel wireless passive sensing system is presented for detecting and measuring the level of strain and relative displacement in structures. The sensing system comprises a nested split-ring resonator (NSRR) probe along with a transceiver antenna. These two elements form an electromagnetically coupled system that yields very high sensitivity and resolution of displacement and strain sensing accompanied with a wide dynamic range of measurement. Using this wireless system, it is possible to track strain/displacement in both the elastic (reversible-linear) and plastic (irreversiblenonlinear) deformation regions of steel rebars. In the dissertation, the results of the following experiments are presented: Characterization experiments carried out on a translation stage in laboratory environment, tensile test experiments where a rebar is loaded with a pulling force until fracture, and simply supported beam experiments where a beam undergoes loading, which leads to tensile strains on rebars at the bottom of the beam. Especially, the simply supported beam experiments constitute a decisive step toward a real-life application of the proposed sensing system. The sensing system is shown to acquire accurate data until the end of the measurements in which the wired devices such as strain gages break down and fail to capture. Furthermore, the effects of the complex electromagnetic medium formed by the rebars and the concrete on sensing are investigated. In addition, a multi-point sensing capability via multiple probes and single antenna is proposed and experimentally demonstrated, which can be used in 2-D surface strain mapping with further improvements. Finally, an equivalent circuit model is