Miracle-dz/UniFAR-FTUs huggingface dataset:
White Matter Hyperintensities Segmentation Using Probabilistic TransUNet

White Matter Hyperintensities (WMH) are areas of the brain that have a higher intensity than other normal brain regions on Magnetic Resonance Imaging (MRI) scans. WMH is often associated with small vessel disease in the brain, making early detection of WMH important. However, there are two common issues in detecting WMH: high ambiguity and difficulty detecting small WMH. In this study, we propose a method called Probabilistic TransUNet to address the precision of small object segmentation and the high ambiguity of medical images. We conducted a k-fold cross-validation and cross-dataset robustness experiment to measure model performance. Based on the experiments, Transformer-based models (TransUNet and Probabilistic TransUNet) were found to provide more precise and better segmentation results, as demonstrated by the higher DSC scores obtained compared to CNN-based models (UNet and Probabilistic UNet) and their ability to segment small WMH objects. The addition of a probabilistic model and the use of a transformer-based approach was able to achieve better performance. Our proposed method achieved a DSC score of 0.742 in K-fold cross-validation outperforming the previous method. The use of CNN and Transformer-based probabilistic models increased the average DSC score by +0.016 and +0.001, respectively. In cross-dataset robustness, our proposed method achieved the second-best result with a DSC score of 0.605
Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts

We evaluated the capability of a state-of-the-art generative pretrained transformer (GPT) model to perform semantic annotation of short text snippets (one to few sentences) coming from legal documents of various types. Discussions of potential uses (e.g., document drafting, summarization) of this emerging technology in legal domain have intensified, but to date there has not been a rigorous analysis of these large language models' (LLM) capacity in sentence-level semantic annotation of legal texts in zero-shot learning settings. Yet, this particular type of use could unlock many practical applications (e.g., in contract review) and research opportunities (e.g., in empirical legal studies). We fill the gap with this study. We examined if and how successfully the model can semantically annotate small batches of short text snippets (10-50) based exclusively on concise definitions of the semantic types. We found that the GPT model performs surprisingly well in zero-shot settings on diverse types of documents (F1 = .73 on a task involving court opinions, .86 for contracts, and .54 for statutes and regulations). These findings can be leveraged by legal scholars and practicing lawyers alike to guide their decisions in integrating LLMs in wide range of workflows involving semantic annotation of legal texts.
Towards Prompt-robust Face Privacy Protection via Adversarial Decoupling Augmentation Framework

Denoising diffusion models have shown remarkable potential in various generation tasks. The open-source large-scale text-to-image model, Stable Diffusion, becomes prevalent as it can generate realistic artistic or facial images with personalization through fine-tuning on a limited number of new samples. However, this has raised privacy concerns as adversaries can acquire facial images online and fine-tune text-to-image models for malicious editing, leading to baseless scandals, defamation, and disruption to victims' lives. Prior research efforts have focused on deriving adversarial loss from conventional training processes for facial privacy protection through adversarial perturbations. However, existing algorithms face two issues: 1) they neglect the image-text fusion module, which is the vital module of text-to-image diffusion models, and 2) their defensive performance is unstable against different attacker prompts. In this paper, we propose the Adversarial Decoupling Augmentation Framework (ADAF), addressing these issues by targeting the image-text fusion module to enhance the defensive performance of facial privacy protection algorithms. ADAF introduces multi-level text-related augmentations for defense stability against various attacker prompts. Concretely, considering the vision, text, and common unit space, we propose Vision-Adversarial Loss, Prompt-Robust Augmentation, and Attention-Decoupling Loss. Extensive experiments on CelebA-HQ and VGGFace2 demonstrate ADAF's promising performance, surpassing existing algorithms.
On k-wave solutions of quasilinear systems of partial differential equations

In this paper, we establish a relation between two seemingly unrelated concepts for solving first-order hyperbolic quasilinear systems of partial differential equations in many dimensions. These concepts are based on a variant of the conditional symmetry method and on the generalized method of characteristics. We present the outline of recent results on multiple Riemann wave solutions of these systems. An auxiliary result concerning a modification of the Frobenius theorem for integration is used. We apply this result in order to show that the conditional symmetry method can deliver larger classes of multiple Riemann wave solutions, through a simpler procedure, than the one obtained from the generalized method of characteristics. We demonstrate that solutions can be interpreted physically as a superposition of k single waves. These theoretical considerations are illustrated by examples of hydrodynamic-type systems in (n+1) dimensions.
Learning Stochastic Dynamical System via Flow Map Operator

We present a numerical framework for learning unknown stochastic dynamical systems using measurement data. Termed stochastic flow map learning (sFML), the new framework is an extension of flow map learning (FML) that was developed for learning deterministic dynamical systems. For learning stochastic systems, we define a stochastic flow map that is a superposition of two sub-flow maps: a deterministic sub-map and a stochastic sub-map. The stochastic training data are used to construct the deterministic sub-map first, followed by the stochastic sub-map. The deterministic sub-map takes the form of residual network (ResNet), similar to the work of FML for deterministic systems. For the stochastic sub-map, we employ a generative model, particularly generative adversarial networks (GANs) in this paper. The final constructed stochastic flow map then defines a stochastic evolution model that is a weak approximation, in term of distribution, of the unknown stochastic system. A comprehensive set of numerical examples are presented to demonstrate the flexibility and effectiveness of the proposed sFML method for various types of stochastic systems.
Gaussian Beam Propagation in a Kerr Type Metamaterial Medium Using ABCD Matrix Method

In this paper, a split step ABCD matrix method is suggested to investigate Gaussian beam propagation in a Kerr type metamaterial medium. This method is based on dividing the medium interval into subsequent steps. Meanwhile, Gaussian beam profile in every step is obtained by finding the ABCD matrix of that particular step, and is used to find the ABCD matrix of the next step. Results of the suggested matrix method have been compared with the results of numerical split-step Fourier method for a Kerr medium, which indicates a good agreement. Then, we use the ABCD matrix to investigate Gaussian beams propagation in a Kerr type metamaterial, which is also in agreement with pervious results by other methods.
Documenting Bioinformatics Software Via Reverse Engineering

Documentation is one of the most neglected activities in Software Engineering, although it is an important method of assuring quality and understanding. Bioinformatics software is generally written by researchers from fields other than Computer Science who usually do not provide documentation. Documenting bioinformatics software may ease its adoption in multidisciplinary teams and expand its impact on the community. In this paper, we highlight how one can document software that is already finished, using reverse engineering and thinking of the end-user.
Exact Quantum Speed Limits

The traditional quantum speed limits are not attainable for many physical processes, as they tend to be loose and fail to determine the exact time taken by quantum systems to evolve. To address this, we derive exact quantum speed limits for the unitary dynamics of pure-state quantum system that outperform the existing quantum speed limits. Using these exact quantum speed limits, we can precisely estimate the evolution time for two- and higher-dimensional quantum systems. Additionally, for both finite- and infinite-dimensional quantum systems, we derive an improved Mandelstam-Tamm bound for pure states and show that this bound always saturates for any unitary generated by self-inverse Hamiltonians. Furthermore, we show that our speed limits establish an upper bound on the quantum computational circuit complexity. These results will have a significant impact on our understanding of quantum physics as well as rapidly developing quantum technologies, such as quantum computing, quantum control and quantum thermal machines.
The GitHub Development Workflow Automation Ecosystems

Large-scale software development has become a highly collaborative and geographically distributed endeavour, especially in open-source software development ecosystems and their associated developer communities. It has given rise to modern development processes (e.g., pull-based development) that involve a wide range of activities such as issue and bug handling, code reviewing, coding, testing, and deployment. These often very effort-intensive activities are supported by a wide variety of tools such as version control systems, bug and issue trackers, code reviewing systems, code quality analysis tools, test automation, dependency management, and vulnerability detection tools. To reduce the complexity of the collaborative development process, many of the repetitive human activities that are part of the development workflow are being automated by CI/CD tools that help to increase the productivity and quality of software projects. Social coding platforms aim to integrate all this tooling and workflow automation in a single encompassing environment. These social coding platforms gave rise to the emergence of development bots, facilitating the integration with external CI/CD tools and enabling the automation of many other development-related tasks. GitHub, the most popular social coding platform, has introduced GitHub Actions to automate workflows in its hosted software development repositories since November 2019. This chapter explores the ecosystems of development bots and GitHub Actions and their interconnection. It provides an extensive survey of the state-of-the-art in this domain, discusses the opportunities and threats that these ecosystems entail, and reports on the challenges and future perspectives for researchers as well as software practitioners.
Spectral gap and curvature of monotone Markov chains

We prove that the absolute spectral gap of any monotone Markov chain coincides with its optimal Ollivier-Ricci curvature, where the word `optimal' refers to the choice of the underlying metric. Moreover, we provide a new expression in terms of local variations of increasing functions, which has several practical advantages over the traditional variational formulation using the Dirichlet form. As an illustration, we explicitly determine the optimal curvature and spectral gap of the non-conservative exclusion process with heterogeneous reservoir densities on any network, despite the lack of reversibility.
Harmonic vector excitation coding of speech at 2.0 kbps

A low rate speech coding algorithm, harmonic vector excitation coding (HVXC) is proposed for MPEG-4 standardization, in which an efficient coding scheme based on harmonic and stochastic vector representation of linear predictive coding (LPC) residuals is employed. A combination of weighted vector quantization of harmonic spectral envelope of LPC residual signal for voiced segments and vector excitation coding for unvoiced segments provides good speech duality at very low bit rates. MPEG-4 formal listening tests in December 95 showed that the subjective speech quality of HVXC at 2.0 kbps was better than that of FS1016 4.8 kbps CELP.
Inferring Alzheimer’s disease pathologic traits from clinical measures in living adults

Background and Objectives: Develop imputation models using clinical measures to infer Alzheimer's disease neuropathologic changes (ADNC) in living adults to identify adults at risk for Alzheimer's disease (AD). Methods: We used clinical and postmortem data from two prospective cohort studies, Memory and Aging Project (MAP) and Religious Orders Study (ROS). We used generalized linear regression models with Elastic Net penalty to train imputation models of ADNC traits (beta Amyloid, tau tangles, global AD pathology, and NIA Reagan), in MAP decedents using clinical measures collected at the last visit as predictors. ROS cohort was used as an independent validation and test data. We validated these models in ROS decedents and applied the models to baseline clinical data of ROS participants to infer baseline ADNC traits. Baseline clinical data were collected an average of 8 years before the last follow up. We used Cox proportional hazard models to test if inferred baseline ADNC traits predicted incident AD dementia (ADD). In addition, two sample t tests were used to examine if inferred baseline ADNC traits predicted adults with a high risk of pathologic AD profiled at death. Results: By applying imputation models to clinical measures collected at the last visit in ROS to validate the imputation models, we obtained predicted R2 as 0.188 for beta Amyloid, 0.316 for tau tangles, and 0.262 for global AD pathology. The prediction area under the receiver operating characteristic curve (AUC) for the dichotomous NIA Reagan was 0.765. All four inferred ADNC traits at the last visit strongly discriminated postmortem NIA Reagan status (pvalues <10E-28). The inferred baseline levels of all four ADNC traits predicted ADD, with higher accuracies for predicting ADD in Year 3 (AUC ranging in 0.861 and 0.919) versus Year 5 (AUC from 0.842 to 0.896), and the highest accuracy was obtained using inferred NIA Reagan status. The inferred baseline levels of all four ADNC traits significantly discriminate individuals with postmortem pathologic AD (all pvalues < 1.5E-7). Conclusions: Inferred baseline levels of ADNC traits derived from clinical measures discriminate adults at risk for ADD and pathologic AD profiled at death. Further studies are needed to determine if repeated measures of inferred ADNC traits can be used to monitor the accumulation of ADNC traits during the prolonged course of AD.
ALCAP: Alignment-Augmented Music Captioner

Music captioning has gained significant attention in the wake of the rising prominence of streaming media platforms. Traditional approaches often prioritize either the audio or lyrics aspect of the music, inadvertently ignoring the intricate interplay between the two. However, a comprehensive understanding of music necessitates the integration of both these elements. In this study, we delve into this overlooked realm by introducing a method to systematically learn multimodal alignment between audio and lyrics through contrastive learning. This not only recognizes and emphasizes the synergy between audio and lyrics but also paves the way for models to achieve deeper cross-modal coherence, thereby producing high-quality captions. We provide both theoretical and empirical results demonstrating the advantage of the proposed method, which achieves new state-of-the-art on two music captioning datasets.
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning

Prompt tuning is a parameter-efficient approach to adapting pre-trained language models to downstream tasks. Although prompt tuning has been shown to match the performance of full model tuning when training data is sufficient, it tends to struggle in few-shot learning settings. In this paper, we present Multi-task Pre-trained Modular Prompt (MP2) to boost prompt tuning for few-shot learning. MP2 is a set of combinable prompts pre-trained on 38 Chinese tasks. On downstream tasks, the pre-trained prompts are selectively activated and combined, leading to strong compositional generalization to unseen tasks. To bridge the gap between pre-training and fine-tuning, we formulate upstream and downstream tasks into a unified machine reading comprehension task. Extensive experiments under two learning paradigms, i.e., gradient descent and black-box tuning, show that MP2 significantly outperforms prompt tuning, full model tuning, and prior prompt pre-training methods in few-shot settings. In addition, we demonstrate that MP2 can achieve surprisingly fast and strong adaptation to downstream tasks by merely learning 8 parameters to combine the pre-trained modular prompts.
On the predictive validity of the National Football League combine: does it forecast future success?

ABSTRACT The National Football League (NFL) Combine provides NFL teams the ability to assess prospective athletes’ medical histories and physical and psychological abilities. Using this information, NFL personnel must then decide whether an athlete is a good fit to their team. Given the combine’s 40-year history and the availability of peer reviewed articles on the function and efficacy of the combine, the purpose of this systematic review was to synthesize the literature evaluating the predictive validity of the combine according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). Full-text, peer-reviewed articles containing information relevant to the NFL combine and at least one measure of future success were retained. The search yielded 1954 articles and after screening, 68 articles remained. These remaining articles focused on measures of success pertaining to (a) medical testing and future performance (n = 25), (b) combine tests (n = 12), (c) draft position (n = 10), (d) draft and future performance (n = 8), (e) medical testing (n = 7), (f) career longevity (n = 3), (g) draft and salary (n = 2), and (h) playing performance (n = 1). Due to the mixed results of combine measures on future success, this review highlights the need for more research to investigate the combine’s influence on long-term performance and success.
Spectral condition for spanning $k$-ended trees in $t$-connected graphs

For any integer $k\geq 2$, a spanning $k$-ended tree is a spanning tree with at most $k$ leaves. In this paper, we provide a tight spectral radius condition for the existence of a spanning $k$-ended tree in $t$-connected graphs, which generalizes a result of Ao, Liu and Yuan (2023).
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch

Gradient preconditioning is a key technique to integrate the second-order information into gradients for improving and extending gradient-based learning algorithms. In deep learning, stochasticity, nonconvexity, and high dimensionality lead to a wide variety of gradient preconditioning methods, with implementation complexity and inconsistent performance and feasibility. We propose the Automatic Second-order Differentiation Library (ASDL), an extension library for PyTorch, which offers various implementations and a plug-and-play unified interface for gradient preconditioning. ASDL enables the study and structured comparison of a range of gradient preconditioning methods.
Hierarchical Visual Localization Based on Sparse Feature Pyramid for Adaptive Reduction of Keypoint Map Size

Visual localization is a fundamental task for a wide range of applications in the field of robotics. Yet, it is still a complex problem with no universal solution, and the existing approaches are difficult to scale: most state-of-the-art solutions are unable to provide accurate localization without a significant amount of storage space. We propose a hierarchical, low-memory approach to localization based on keypoints with different descriptor lengths. It becomes possible with the use of the developed unsupervised neural network, which predicts a feature pyramid with different descriptor lengths for images. This structure allows applying coarse-to-fine paradigms for localization based on keypoint map, and varying the accuracy of localization by changing the type of the descriptors used in the pipeline. Our approach achieves comparable results in localization accuracy and a significant reduction in memory consumption (up to 16 times) among state-of-the-art methods.
Dose, Content, and Context of Usual Care in Stroke Upper Limb Motor Interventions: A Systematic Review

Objective The objectives of this systematic review were to describe the current dose and content of usual care upper limb motor intervention for inpatients following stroke and examine if context factors alter dose and content. Data sources A systematic search (EMBASE, MEDLINE) was completed from January 2015 to February 2023 (PROSPERO CRD42021281986). Methods Studies were eligible if they reported non-protocolised usual care upper limb motor intervention dose data for stroke inpatients. Studies were rated using the Johanna Briggs Institute critical appraisal tool. Data were descriptively reported for dose dimensions of time (on task or, in therapy) and intensity (repetitions, repetition/minute), content (intervention type/mode), and context (e.g., severity strata). Results Eight studies were included from four countries, largely reflecting inpatient rehabilitation. Time in therapy ranged from 23 to 121 min/day. Time on task ranged from 8 to 44 min/day. Repetitions ranged from 36 to 57/session, and 15 to 282/day. Time on task was lowest in the stratum of people with severe upper limb impairment (8 min/day), the upper limit for this stratum was 41.5 min/day. There was minimal reporting of usual care content across all studies. Conclusion Upper limb motor intervention dose appears to be increasing in usual care compared to prior reports (e.g., average 21 min/day and 23 to 32 repetitions/session). Context variability suggests that doses are lowest in the stratum of patients with a severely impaired upper limb. Consistent reporting of the multiple dimensions of dose and content is necessary to better understand usual care offered during inpatient rehabilitation.
BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning

The ability of deep neural networks to continually learn and adapt to a sequence of tasks has remained challenging due to catastrophic forgetting of previously learned tasks. Humans, on the other hand, have a remarkable ability to acquire, assimilate, and transfer knowledge across tasks throughout their lifetime without catastrophic forgetting. The versatility of the brain can be attributed to the rehearsal of abstract experiences through a complementary learning system. However, representation rehearsal in vision transformers lacks diversity, resulting in overfitting and consequently, performance drops significantly compared to raw image rehearsal. Therefore, we propose BiRT, a novel representation rehearsal-based continual learning approach using vision transformers. Specifically, we introduce constructive noises at various stages of the vision transformer and enforce consistency in predictions with respect to an exponential moving average of the working model. Our method provides consistent performance gain over raw image and vanilla representation rehearsal on several challenging CL benchmarks, while being memory efficient and robust to natural and adversarial corruptions.