Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Hyperparameters in Stochastic Gradient Descent with Momentum

Following the same routine as [SSJ20], we continue to present the theoretical analysis for stochastic gradient descent with momentum (SGD with momentum) in this paper. Differently, for SGD with momentum, we demonstrate it is the two hyperparameters together, the learning rate and the momentum coefficient, that play the significant role for the linear rate of convergence in non-convex optimization. Our analysis is based on the use of a hyperparameters-dependent stochastic differential equation (hp-dependent SDE) that serves as a continuous surrogate for SGD with momentum. Similarly, we establish the linear convergence for the continuous-time formulation of SGD with momentum and obtain an explicit expression for the optimal linear rate by analyzing the spectrum of the Kramers-Fokker-Planck operator. By comparison, we demonstrate how the optimal linear rate of convergence and the final gap for SGD only about the learning rate varies with the momentum coefficient increasing from zero to one when the momentum is introduced. Then, we propose a mathematical interpretation why the SGD with momentum converges faster and more robust about the learning rate than the standard SGD in practice. Finally, we show the Nesterov momentum under the existence of noise has no essential difference with the standard momentum.
Semantic Segmentation and Object Detection Towards Instance Segmentation: Breast Tumor Identification

Breast cancer is one of the factors that cause the increase of mortality of women. The most widely used method for diagnosing this geological disease i.e. breast cancer is the ultrasound scan. Several key features such as the smoothness and the texture of the tumor captured through ultrasound scans encode the abnormality of the breast tumors (malignant from benign). However, ultrasound scans are often noisy and include irrelevant parts of the breast that may bias the segmentation of eventual tumors. In this paper, we are going to extract the region of interest ( i.e, bounding boxes of the tumors) and feed-forward them to one semantic segmentation encoder-decoder structure based on its classification (i.e, malignant or benign). the whole process aims to build an instance-based segmenter from a semantic segmenter and an object detector.
Directly Training Joint Energy-Based Models for Conditional Synthesis and Calibrated Prediction of Multi-Attribute Data

Multi-attribute classification generalizes classification, presenting new challenges for making accurate predictions and quantifying uncertainty. We build upon recent work and show that architectures for multi-attribute prediction can be reinterpreted as energy-based models (EBMs). While existing EBM approaches achieve strong discriminative performance, they are unable to generate samples conditioned on novel attribute combinations. We propose a simple extension which expands the capabilities of EBMs to generating accurate conditional samples. Our approach, combined with newly developed techniques in energy-based model training, allows us to directly maximize the likelihood of data and labels under the unnormalized joint distribution. We evaluate our proposed approach on high-dimensional image data with high-dimensional binary attribute labels. We find our models are capable of both accurate, calibrated predictions and high-quality conditional synthesis of novel attribute combinations.
Whittle Index for A Class of Restless Bandits with Imperfect Observations

We consider a class of restless bandit problems that ﬁnds a broad application area 4 in stochastic optimization, reinforcement learning and operations research. In our model, there are 5 N independent 2-state Markov processes that may be observed and accessed for accruing rewards. 6 The observation is error-prone, i
PatchRNN: A Deep Learning-Based System for Security Patch Identification

With the increasing usage of open-source software (OSS) components, vulnerabilities embedded within them are propagated to a huge number of underlying applications. In practice, the timely application of security patches in downstream software is challenging. The main reason is that such patches do not explicitly indicate their security impacts in the documentation, which would be difficult to recognize for software maintainers and users. However, attackers can still identify these “secret” security patches by analyzing the source code and generate corresponding exploits to compromise not only unpatched versions of the current software, but also other similar software packages that may contain the same vulnerability due to code cloning or similar design/implementation logic. Therefore, it is critical to identify these secret security patches to enable timely fixes. To this end, we propose a deep learning-based defense system called PatchRNN to automatically identify secret security patches in OSS. Besides considering descriptive keywords in the commit message (i.e., at the text level), we leverage both syntactic and semantic features at the source-code level. To evaluate the performance of our system, we apply it on a large-scale real-world patch dataset and conduct a case study on a popular open-source web server software - NGINX. Experimental results show that the PatchRNN can successfully detect secret security patches with a low false positive rate.
Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey

The rapid increase in the percentage of chronic disease patients along with the recent pandemic pose immediate threats on healthcare expenditure and elevate causes of death. This calls for transforming healthcare systems away from one-on-one patient treatment into intelligent health systems, to improve services, access and scalability, while reducing costs. Reinforcement Learning (RL) has witnessed an intrinsic breakthrough in solving a variety of complex problems for diverse applications and services. Thus, we conduct in this paper a comprehensive survey of the recent models and techniques of RL that have been developed/used for supporting Intelligent-healthcare (I-health) systems. This paper can guide the readers to deeply understand the state-of-the-art regarding the use of RL in the context of I-health. Specifically, we first present an overview for the I-health systems challenges, architecture, and how RL can benefit these systems. We then review the background and mathematical modeling of different RL, Deep RL (DRL), and multi-agent RL models. After that, we provide a deep literature review for the applications of RL in I-health systems. In particular, three main areas have been tackled, i.e., edge intelligence, smart core network, and dynamic treatment regimes. Finally, we highlight emerging challenges and outline future research directions in driving the future success of RL in I-health systems, which opens the door for exploring some interesting and unsolved problems.
Joint AP Probing and Scheduling: A Contextual Bandit Approach

We consider a set of APs with unknown data rates that cooperatively serve a mobile client. The data rate of each link is i.i. d. sampled from a distribution that is unknown a priori. In contrast to traditional link scheduling problems under uncertainty, we assume that in each time step, the device can probe a subset of links before deciding which one to use. We model this problem as a contextual bandit problem with probing (CBwP) and present an efficient algorithm. We further establish the regret of our algorithm for links with Bernoulli data rates. Our CBwP model is a novel extension of the classic contextual bandit model and can potentially be applied to a large class of sequential decision-making problems that involve joint probing and play under uncertainty.
Significantly enhanced polymer thermal conductivity by confining effect through bilayer MoS2 Surfaces

The present work refers to a simulation study on nanoconfined polymers in polymer-MoS2 nanocomposites as a function of MoS2-MoS2 interlayer separation. We indeed apply reverse nonequlibrium molecular dynamics simulations (RNEMD) to investigate thermal conductivity of polyamide oligomers which is confined by MoS2 bilayers. The polymer thermal conductivity can be considerably enhanced when polymer chains are confined by MoS2 sheets, in particular such behavior is more obvious for charged surfaces. The presence of the MoS2 surfaces leads to wellordering of polymer chains as well as denser packing which is an increase in λ in the polymer network via the use of MoS2 surface confinement. Polymer chains elongation as well as their preferential alignment parallel to the MoS2 surfaces indicates that λ in the polymer domain of the considered nanocomposites is larger than the one in the pure polymer phase. Additionally, our results of number of hydrogen bonds (HBs) in confined polymer chains suggest that a combined effect of the mentioned structural modification and enlarged values of HBs may cooperatively contribute to high polymer thermal conductivity facilitating phonon transport. Results reported here suggest a significant manner to design of confined polymer-MoS2 composites for a wide variety range of applications.
An educator’s perspective of the tidyverse

Computing makes up a large and growing component of data science and statistics courses. Many of those courses, especially when taught by faculty who are statisticians by training, teach R as the programming language. A number of instructors have opted to build much of their teaching around use of the tidyverse . The tidyverse, in the words of its developers, “is a collection of R packages that share a high-level † design philosophy and low-level grammar and data structures, so that learning one package makes it easier to learn the next” (Wickham et al. 2019). These shared principles have led to the widespread adoption of the tidyverse ecosystem. A large part of this usage is because the tidyverse tools have been intentionally designed to ease the learning process and make it easier for users to learn new functions as they engage with additional pieces of the larger ecosystem. Moreover, the functionality oﬀered by the packages within the tidyverse spans the entire data science cycle, which includes data import, visualisation, wrangling, modeling, and communication. We believe the tidyverse provides an eﬀective and eﬃcient pathway for undergraduate students at all levels and majors to gain computational skills and thinking needed throughout the data science cycle. In this paper, we introduce the tidyverse from an educator’s perspective. We provide a brief introduction to the tidyverse, demonstrate how foundational statistics and data science tasks are accomplished with the tidyverse, and discuss the strengths of the tidyverse, particularly in the context of teaching and learning.
Define of internal recirculation coefficient for biological wastewater treatment in anoxic and aerobic bioreactors

The biological wastewater treatment technologies in anoxic and aerobic bioreactors with recycle of sludge mixture are used for the effective removal of organic compounds from wastewater. The change rate of sludge mixture recirculation between bioreactors leads to a change and redistribution of concentrations of organic compounds in sludge mixture in bioreactors and change hydrodynamic regimes in bioreactors. Determination of the coefficient of internal recirculation of sludge mixture between bioreactors is important for the choice of technological parameters of biological treatment (wastewater treatment duration in anoxic and aerobic bioreactors, flow capacity of recirculation pumps). Determination of the coefficient of internal recirculation of sludge mixture requires integrated consideration of hydrodynamic parameter (flow rate), kinetic parameter (rate of oxidation of organic compounds) and physical-chemical parameter of wastewater (concentration of organic compounds). The conducted numerical experiment from the proposed mathematical equations allowed to obtain analytical dependences of the coefficient of internal recirculation sludge mixture between bioreactors on the concentration of organic compounds in wastewater, the duration of wastewater treatment in bioreactors.
The mysteries of the best approximation and Chebyshev expansion for the function with logarithmic regularities

The best polynomial approximation and Chebyshev approximation are both important in numerical analysis. In tradition, the best approximation is regarded as more better than the Chebyshev approximation, because it is usually considered in the uniform norm. However, it not always superior to the latter noticed by Trefethen \cite{Trefethen11sixmyths,Trefethen2020} for the algebraic singularity function. Recently Wang \cite{Wang2021best} have proved it in theory. In this paper, we find that for the functions with logarithmic regularities, the pointwise errors of Chebyshev approximation are smaller than the ones of the best approximations except only in the very narrow boundaries at the same degree. The pointwise error for Chebyshev series, truncated at the degree $n$ is $O(n^{-\kappa})$ ($\kappa = \min\{2\gamma+1, 2\delta + 1\}$), but is worse by one power of $n$ in narrow boundary layer near the weak singular endpoints. Theorems are given to explain this effect.
Asymmetry-aware scalable locking

The pursuit of power-efficiency is popularizing asymmetric multicore processors (AMP) such as ARM big.LITTLE, Apple M1 and recent Intel Alder Lake with big and little cores. However, we find that existing scalable locks fail to scale on AMP and cause collapses in either throughput or latency, or both, because their implicit assumption of symmetric cores no longer holds. To address this issue, we propose the first asymmetry-aware scalable lock named LibASL. LibASL provides a new lock ordering guided by applications' latency requirements, which allows big cores to reorder with little cores for higher throughput under the condition of preserving applications' latency requirements. Using LibASL only requires linking the applications with it and, if latency-critical, inserting few lines of code to annotate the coarse-grained latency requirement. We evaluate LibASL in various benchmarks including five popular databases on Apple M1. Evaluation results show that LibASL can improve the throughput by up to 5 times while precisely preserving the tail latency designated by applications.
Two-stream Convolutional Networks for Multi-frame Face Anti-spoofing

Face anti-spoofing is an important task to protect the security of face recognition. Most of previous work either struggle to capture discriminative and generalizable feature or rely on auxiliary information which is unavailable for most of industrial product. Inspired by the video classification work, we propose an efficient two-stream model to capture the key differences between live and spoof faces, which takes multi-frames and RGB difference as input respectively. Feature pyramid modules with two opposite fusion directions and pyramid pooling modules are applied to enhance feature representation. We evaluate the proposed method on the datasets of Siw, Oulu-NPU, CASIA-MFSD and Replay-Attack. The results show that our model achieves the state-of-the-art results on most of datasets' protocol with much less parameter size.
Notes on Coalgebras in Stylometry

The syntactic behaviour of texts can highly vary depending on their contexts (e.g. author, genre, etc.). From the standpoint of stylometry, it can be helpful to objectively measure this behaviour. In this paper, we discuss how coalgebras are used to formalise the notion of behaviour by embedding syntactic features of a given text into probabilistic transition systems. By introducing the behavioural distance, we are then able to quantitatively measure differences between points in these systems and thus, comparing features of different texts. Furthermore, the behavioural distance of points can be approximated by a polynomial-time algorithm.
Detecting False Data Injection Attacks in Smart Grids with Modeling Errors: A Deep Transfer Learning Based Approach

Most traditional false data injection attack (FDIA) detection approaches rely on a key assumption, i.e., the power system can be accurately modeled. However, the transmission line parameters are dynamic and cannot be accurately known during operation and thus the involved modeling errors should not be neglected. In this paper, an illustrative case has revealed that modeling errors in transmission lines significantly weaken the detection effectiveness of conventional FDIA approaches. To tackle this issue, we propose an FDIA detection mechanism from the perspective of transfer learning. Specifically, the simulated power system is treated as a source domain, which provides abundant simulated normal and attack data. The real world's running system whose transmission line parameters are unknown is taken as a target domain where sufficient real normal data are collected for tracking the latest system states online. The designed transfer strategy that aims at making full use of data in hand is divided into two optimization stages. In the first stage, a deep neural network (DNN) is built by simultaneously optimizing several well-designed objective terms with both simulated data and real data, and then it is fine-tuned via real data in the second stage. Several case studies on the IEEE 14-bus and 118-bus systems verify the effectiveness of the proposed mechanism.
A Non-Trivial Lasing Mode From a Parity-Time Symmetric Diffraction Grating

This work presents a theoretical investigation of an active diffraction grating of the parity-time (PT) symmetric architecture. The analytical study of the free-space mode propagation in the grating structure indicates the unique bifurcation property due to the PT-symmetry modulation. It is shown that both the gain/loss contrast and the lattice constant parameters are critical factors to modulate the photonic system in between the PT-symmetry to the symmetry-broken phases. Furthermore, numerical simulations via the Rigorous Coupled-Wave Analysis (RCWA) method discover the existence of an unique Spectral Singularity (SS) phenomenon in this PT grating structure which is corresponding to a non-trivial single-mode and near-zero bandwidth photonic resonant emission. Also, the guiding procedure for fulfilling SS modes is found to be related to the unique formation of the scattering matrix applied in the PT-symmetric diffraction gratings. This theoretical work takes a fresh look into the active PT-symmetric diffraction gratings focusing on the discovery of new free-space emission modes rather than the commonly studied unidirectional properties, which could contribute to the development of novel low-threshold and super-coherent laser devices.
Deterministic massively parallel connectivity

We consider the problem of designing fundamental graph algorithms on the model of Massive Parallel Computation (MPC). The input to the problem is an undirected graph G with n vertices and m edges, and with D being the maximum diameter of any connected component in G. We consider the MPC with low local space, allowing each machine to store only Θ(nδ) words for an arbitrary constant δ>0, and with linear global space (which is the number of machines times the local space available), that is, with optimal utilization. In a recent breakthrough, Andoni et al. (FOCS’18) and Behnezhad et al. (FOCS’19) designed parallel randomized algorithms that in O(logD + loglogn) rounds on an MPC with low local space determine all connected components of a graph, improving on the classic bound of O(logn) derived from earlier works on PRAM algorithms. In this paper, we show that asymptotically identical bounds can be also achieved for deterministic algorithms: we present a deterministic MPC low local space algorithm that in O(logD + loglogn) rounds determines connected components of the input graph. Our result matches the complexity of state of the art randomized algorithms for this task. The techniques developed in our paper can be also applied to several related problems, giving new deterministic MPC algorithms for problems like finding a spanning forest, minimum spanning forest, etc. We complement our upper bounds by extending a recent lower bound for connectivity on an MPC conditioned on the 1-vs-2-cycles conjecture (which requires D ≥ log1+Ω(1)n), by showing a related conditional hardness of Ω(logD) MPC rounds for the entire spectrum of D, covering a particularly interesting range when D ≤ O(logn).
From Domain-Specific Languages to Memory-Optimized Accelerators for Fluid Dynamics

Many applications are increasingly requiring numerical simulations for solving complex problems. Most of these numerical algorithms are massively parallel and often implemented on parallel high-performance computers. However, classic CPU-based platforms suffers due to the demand of higher resolutions and the exponential growth of data. FPGAs offer a powerful and flexible alternative that can host accelerators to complement such platforms. Developing such application-specific accelerators is still challenging because it is hard to provide efficient code for hardware synthesis. In this paper, we study the challenges for porting a numerical simulation kernels onto FPGA. We propose an automated tool flow from a domain-specific language (DSL) to generate accelerators for computational fluid dynamics on FPGA. Our DSL-based flow simplifies the exploration of parameters and constraints such as on-chip memory usage. We also propose a decoupled optimization of memory and logic resources, which allows us to better use the limited FPGA resources. In our preliminary evaluation, this enabled doubling the amount of parallel kernels, increasing the accelerator speedup versus ARM execution from 7 to 12 times.
Prescribed‐time regulation of nonlinear uncertain systems with unknown input gain and appended dynamics

The prescribed‐time stabilization problem for a general class of uncertain nonlinear systems with unknown input gain and appended dynamics (with unmeasured state) is addressed. Unlike the asymptotic stabilization problem, the prescribed‐time stabilization objective requires convergence of the state vector to the origin in a finite time interval that can be arbitrarily picked (i.e., “prescribed”) by the control system designer irrespective of the system's initial condition. The class of systems considered is allowed to have general nonlinear uncertain terms throughout the system dynamics as well as uncertain appended dynamics (that effectively generate a time‐varying non‐vanishing disturbance signal input into the nominal system). The control design is based on a nonlinear transformation of the time scale, dynamic high‐gain scaling, adaptation dynamics with temporal forcing terms, and a composite control law that includes two components. The first component in the composite control law is analogous to prior dynamic high‐gain scaling‐based control designs, but with a time‐dependent function in place of the unknown input gain, while the second component has a non‐smooth form with time‐dependent terms that ensure prescribed‐time convergence in spite of the unknown input gain and the disturbances. The efficacy of the proposed control design is illustrated through numerical simulation studies on two example systems (a “synthetic” fifth‐order system and a “real‐world” electromechanical system).