Miracle-dz/UniFAR-FTUs huggingface dataset:
DoseDiff: Distance-aware Diffusion Model for Dose Prediction in Radiotherapy

Treatment planning, which is a critical component of the radiotherapy workflow, is typically carried out by a medical physicist in a time-consuming trial-and-error manner. Previous studies have proposed knowledge-based or deep-learning-based methods for predicting dose distribution maps to assist medical physicists in improving the efficiency of treatment planning. However, these dose prediction methods usually fail to effectively utilize distance information between surrounding tissues and targets or organs-at-risk (OARs). Moreover, they are poor at maintaining the distribution characteristics of ray paths in the predicted dose distribution maps, resulting in a loss of valuable information. In this paper, we propose a distance-aware diffusion model (DoseDiff) for precise prediction of dose distribution. We define dose prediction as a sequence of denoising steps, wherein the predicted dose distribution map is generated with the conditions of the computed tomography (CT) image and signed distance maps (SDMs). The SDMs are obtained by distance transformation from the masks of targets or OARs, which provide the distance from each pixel in the image to the outline of the targets or OARs. We further propose a multi-encoder and multi-scale fusion network (MMFNet) that incorporates multi-scale and transformer-based fusion modules to enhance information fusion between the CT image and SDMs at the feature level. We evaluate our model on two in-house datasets and a public dataset, respectively. The results demonstrate that our DoseDiff method outperforms state-of-the-art dose prediction methods in terms of both quantitative performance and visual quality.
DCT: Dual Channel Training of Action Embeddings for Reinforcement Learning with Large Discrete Action Spaces

The ability to learn robust policies while generalizing over large discrete action spaces is an open challenge for intelligent systems, especially in noisy environments that face the curse of dimensionality. In this paper, we present a novel framework to efficiently learn action embeddings that simultaneously allow us to reconstruct the original action as well as to predict the expected future state. We describe an encoder-decoder architecture for action embeddings with a dual channel loss that balances between action reconstruction and state prediction accuracy. We use the trained decoder in conjunction with a standard reinforcement learning algorithm that produces actions in the embedding space. Our architecture is able to outperform two competitive baselines in two diverse environments: a 2D maze environment with more than 4000 discrete noisy actions, and a product recommendation task that uses real-world e-commerce transaction data. Empirical results show that the model results in cleaner action embeddings, and the improved representations help learn better policies with earlier convergence.
Generalized Core Spanner Inexpressibility via Ehrenfeucht-Fraïssé Games for FC

Despite considerable research on document spanners, little is known about the expressive power of generalized core spanners. In this paper, we use Ehrenfeucht-Fra\"iss\'e games to obtain general inexpressibility lemmas for the logic FC (a finite-model variant of the theory of concatenation). Applying these lemmas give inexpressibility results for FC that we lift to generalized core spanners. In particular, we give several relations that cannot be selected by generalized core spanners, thus demonstrating the effectiveness of the inexpressibility lemmas. As an immediate consequence, we also gain new insights into the expressive power of core spanners.
Pb-Hash: Partitioned b-bit Hashing

Many hashing algorithms including minwise hashing (MinHash), one permutation hashing (OPH), and consistent weighted sampling (CWS) generate integers of $B$ bits. With $k$ hashes for each data vector, the storage would be $B\times k$ bits; and when used for large-scale learning, the model size would be $2^B\times k$, which can be expensive. A standard strategy is to use only the lowest $b$ bits out of the $B$ bits and somewhat increase $k$, the number of hashes. In this study, we propose to re-use the hashes by partitioning the $B$ bits into $m$ chunks, e.g., $b\times m =B$. Correspondingly, the model size becomes $m\times 2^b \times k$, which can be substantially smaller than the original $2^B\times k$. Our theoretical analysis reveals that by partitioning the hash values into $m$ chunks, the accuracy would drop. In other words, using $m$ chunks of $B/m$ bits would not be as accurate as directly using $B$ bits. This is due to the correlation from re-using the same hash. On the other hand, our analysis also shows that the accuracy would not drop much for (e.g.,) $m=2\sim 4$. In some regions, Pb-Hash still works well even for $m$ much larger than 4. We expect Pb-Hash would be a good addition to the family of hashing methods/applications and benefit industrial practitioners. We verify the effectiveness of Pb-Hash in machine learning tasks, for linear SVM models as well as deep learning models. Since the hashed data are essentially categorical (ID) features, we follow the standard practice of using embedding tables for each hash. With Pb-Hash, we need to design an effective strategy to combine $m$ embeddings. Our study provides an empirical evaluation on four pooling schemes: concatenation, max pooling, mean pooling, and product pooling. There is no definite answer which pooling would be always better and we leave that for future study.
Exploring Fully Offloaded GPU Stream-Aware Message Passing

Modern heterogeneous supercomputing systems are comprised of CPUs, GPUs, and high-speed network interconnects. Communication libraries supporting efficient data transfers involving memory buffers from the GPU memory typically require the CPU to orchestrate the data transfer operations. A new offload-friendly communication strategy, stream-triggered (ST) communication, was explored to allow offloading the synchronization and data movement operations from the CPU to the GPU. A Message Passing Interface (MPI) one-sided active target synchronization based implementation was used as an exemplar to illustrate the proposed strategy. A latency-sensitive nearest neighbor microbenchmark was used to explore the various performance aspects of the implementation. The offloaded implementation shows significant on-node performance advantages over standard MPI active RMA (36%) and point-to-point (61%) communication. The current multi-node improvement is less (23% faster than standard active RMA but 11% slower than point-to-point), but plans are in progress to purse further improvements.
Secure and Fast Asynchronous Vertical Federated Learning via Cascaded Hybrid Optimization

Vertical Federated Learning (VFL) attracts increasing attention because it empowers multiple parties to jointly train a privacy-preserving model over vertically partitioned data. Recent research has shown that applying zeroth-order optimization (ZOO) has many advantages in building a practical VFL algorithm. However, a vital problem with the ZOO-based VFL is its slow convergence rate, which limits its application in handling modern large models. To address this problem, we propose a cascaded hybrid optimization method in VFL. In this method, the downstream models (clients) are trained with ZOO to protect privacy and ensure that no internal information is shared. Meanwhile, the upstream model (server) is updated with first-order optimization (FOO) locally, which significantly improves the convergence rate, making it feasible to train the large models without compromising privacy and security. We theoretically prove that our VFL framework converges faster than the ZOO-based VFL, as the convergence of our framework is not limited by the size of the server model, making it effective for training large models with the major part on the server. Extensive experiments demonstrate that our method achieves faster convergence than the ZOO-based VFL framework, while maintaining an equivalent level of privacy protection. Moreover, we show that the convergence of our VFL is comparable to the unsafe FOO-based VFL baseline. Additionally, we demonstrate that our method makes the training of a large model feasible.
Structure in Reinforcement Learning: A Survey and Open Problems

Reinforcement Learning (RL), bolstered by the expressive capabilities of Deep Neural Networks (DNNs) for function approximation, has demonstrated considerable success in numerous applications. However, its practicality in addressing various real-world scenarios, characterized by diverse and unpredictable dynamics, noisy signals, and large state and action spaces, remains limited. This limitation stems from issues such as poor data efficiency, limited generalization capabilities, a lack of safety guarantees, and the absence of interpretability, among other factors. To overcome these challenges and improve performance across these crucial metrics, one promising avenue is to incorporate additional structural information about the problem into the RL learning process. Various sub-fields of RL have proposed methods for incorporating such inductive biases. We amalgamate these diverse methodologies under a unified framework, shedding light on the role of structure in the learning problem, and classify these methods into distinct patterns of incorporating structure. By leveraging this comprehensive framework, we provide valuable insights into the challenges of structured RL and lay the groundwork for a design pattern perspective on RL research. This novel perspective paves the way for future advancements and aids in developing more effective and efficient RL algorithms that can potentially handle real-world scenarios better.
Identifying downregulated hub genes and key pathways in HBV-related hepatocellular carcinoma using systems biology approach

Chronic Hepatitis B (CHB) is an independent risk factor for hepatocellular carcinoma (HCC) initiation without cirrhosis occurrence. Apart from the favorable effects of some antiviral drugs following tumor resection on the survival of HCC patients, the use of these agents is essential lifelong. Thus, designing the target-oriented therapeutic strategies to increase life expectancy in HCC patients would be very important. The present study aimed to identify downregulated hub genes and enriched pathways in HB-related HCC using a systems biology-based approach. Microarray data of GSE121248 were downloaded from gene expression omnibus (GEO) database. The differentially expressed genes (DEGs) with the cut-off criteria of adjusted p<0.05 and Log Fold-change (FC)<-1.5 were selected. Then, the genes with the highest centrality were detected. Finally, the prognostic values of the hub genes were assessed. Six under-expressed hub genes with the highest interaction degree, Betweenness and Eigenvector centrality were including IGF-1, PTGS2, PLG, HGF, ESR-1 and CYP2B6. Among genes with high centrality, several genes including CYP2C9, ESR-1, CXCL2, CYP2C8, IGF-1, CYP3A4, CYP2E1, CERPINE-1 and PXR were prognostic in HCC. The important repressed pathways were including metabolic pathways and PI3K-Akt and chemokine signaling pathways. The under-expression of several genes implicated in metabolism, differentiation and chemotaxis might be a hallmark of the progression of HCC that can be considered as diagnostic and therapeutic targets.
Symmetries and Covering Maps for the Minimal Tension String on $\mathbf{AdS_3\times S^3\times T^4}$

This paper considers a recently-proposed string theory on $AdS_3\times S^3\times T^4$ with one unit of NS-NS flux ($k=1$). We discuss interpretations of the target space, including connections to twistor geometry and a more conventional spacetime interpretation via the Wakimoto representation. We propose an alternative perspective on the role of the Wakimoto formalism in the $k=1$ string, for which no large radius limit is required by the inclusion of extra operator insertions in the path integral. This provides an exact Wakimoto description of the worldsheet CFT. We also discuss an additional local worldsheet symmetry, $Q(z)$, that emerges when $k=1$ and show that this symmetry plays an important role in the localisation of the path integral to a sum over covering maps. We demonstrate the emergence of a rigid worldsheet translation symmetry in the radial direction of the $AdS_3$, for which again the presence of $Q(z)$ is crucial. We conjecture that this radial symmetry plays a key role in understanding, in the case of the $k=1$ string, the encoding of the bulk physics on the two-dimensional boundary.
Separable Pathway Effects of Semi-Competing Risks via Multi-State Models

Semi-competing risks refer to the phenomenon where a primary outcome event (such as mortality) can truncate an intermediate event (such as relapse of a disease), but not vice versa. Under the multi-state model, the primary event is decomposed to a direct outcome event and an indirect outcome event through intermediate events. Within this framework, we show that the total treatment effect on the cumulative incidence of the primary event can be decomposed into three separable pathway effects, corresponding to treatment effects on population-level transition rates between states. We next propose estimators for the counterfactual cumulative incidences of the primary event under hypothetical treatments by generalized Nelson-Aalen estimators with inverse probability weighting, and then derive the consistency and asymptotic normality of these estimators. Finally, we propose hypothesis testing procedures on these separable pathway effects based on logrank statistics. We have conducted extensive simulation studies to demonstrate the validity and superior performance of our new method compared with existing methods. As an illustration of its potential usefulness, the proposed method is applied to compare effects of different allogeneic stem cell transplantation types on overall survival after transplantation.
RETROSPECTIVE: Corona: System Implications of Emerging Nanophotonic Technology

The 2008 Corona effort was inspired by a pressing need for more of everything, as demanded by the salient problems of the day. Dennard scaling was no longer in effect. A lot of computer architecture research was in the doldrums. Papers often showed incremental subsystem performance improvements, but at incommensurate cost and complexity. The many-core era was moving rapidly, and the approach with many simpler cores was at odds with the better and more complex subsystem publications of the day. Core counts were doubling every 18 months, while per-pin bandwidth was expected to double, at best, over the next decade. Memory bandwidth and capacity had to increase to keep pace with ever more powerful multi-core processors. With increasing core counts per die, inter-core communication bandwidth and latency became more important. At the same time, the area and power of electrical networks-on-chip were increasingly problematic: To be reliably received, any signal that traverses a wire spanning a full reticle-sized die would need significant equalization, re-timing, and multiple clock cycles. This additional time, area, and power was the crux of the concern, and things looked to get worse in the future. Silicon nanophotonics was of particular interest and seemed to be improving rapidly. This led us to consider taking advantage of 3D packaging, where one die in the 3D stack would be a photonic network layer. Our focus was on a system that could be built about a decade out. Thus, we tried to predict how the technologies and the system performance requirements would converge in about 2018. Corona was the result this exercise; now, 15 years later, it's interesting to look back at the effort.
Probing Galaxy structure with VHE gamma rays

As an observer from within the Milky Way, it is difficult to determine its global structure. Despite extensive observational data from surveys at different wavelengths, we have no conclusive description of the structure of our own Galaxy. For very-high-energy (VHE) $\gamma$ rays, the most comprehensive catalogue of Galactic sources resulting from the H.E. S.S. Galactic Plane Survey (HGPS) shows a striking asymmetry in the distribution of the sources in the latitudinal direction. This could be the result of a local feature in the spatial distribution of the sources or it could be due to the position of the Sun above the Galactic plane. In this contribution, we estimate the position of the Sun based on the latitudinal flux profile of VHE $\gamma$-ray sources, assuming three mirror-symmetric models for the spatial distribution of the sources in three-dimensional space and taking into account the observational bias of the HGPS. We verify our method using simulations and find values for $z_{\odot}$ between $-6\,\mathrm{pc}$ and $94\,\mathrm{pc}$ depending on the considered model. Our results show that the position of the Sun has a significant impact on the observed source distribution and must therefore be taken into account when modelling the population of Galactic VHE $\gamma$ sources. However, it is not conclusive whether the Sun's offset from the Galactic plane is the only factor leading to the asymmetry in the latitudinal profile.
Text-Driven Talking Face Synthesis by Reprogramming Audio-Driven Models

In this paper, we present a method for reprogramming pre-trained audio-driven talking face synthesis models to operate in a text-driven manner. Consequently, we can easily generate face videos that articulate the provided textual sentences, eliminating the necessity of recording speech for each inference, as required in the audio-driven model. To this end, we propose to embed the input text into the learned audio latent space of the pre-trained audio-driven model, while preserving the face synthesis capability of the original pre-trained model. Specifically, we devise a Text-to-Audio Embedding Module (TAEM) which maps a given text input into the audio latent space by modeling pronunciation and duration characteristics. Furthermore, to consider the speaker characteristics in audio while using text inputs, TAEM is designed to accept a visual speaker embedding. The visual speaker embedding is derived from a single target face image and enables improved mapping of input text to the learned audio latent space by incorporating the speaker characteristics inherent in the audio. The main advantages of the proposed framework are that 1) it can be applied to diverse audio-driven talking face synthesis models and 2) we can generate talking face videos with either text inputs or audio inputs with high flexibility.
On the Extremal Functions of Acyclic Forbidden 0-1 Matrices

The extremal theory of forbidden 0-1 matrices studies the asymptotic growth of the function $\mathrm{Ex}(P,n)$, which is the maximum weight of a matrix $A\in\{0,1\}^{n\times n}$ whose submatrices avoid a fixed pattern $P\in\{0,1\}^{k\times l}$. This theory has been wildly successful at resolving problems in combinatorics, discrete and computational geometry, structural graph theory, and the analysis of data structures, particularly corollaries of the dynamic optimality conjecture. All these applications use acyclic patterns, meaning that when $P$ is regarded as the adjacency matrix of a bipartite graph, the graph is acyclic. The biggest open problem in this area is to bound $\mathrm{Ex}(P,n)$ for acyclic $P$. Prior results have only ruled out the strict $O(n\log n)$ bound conjectured by Furedi and Hajnal. It is consistent with prior results that $\forall P. \mathrm{Ex}(P,n)\leq n\log^{1+o(1)} n$, and also consistent that $\forall \epsilon>0. \exists P. \mathrm{Ex}(P,n) \geq n^{2-\epsilon}$. In this paper we establish a stronger lower bound on the extremal functions of acyclic $P$. Specifically, we give a new construction of relatively dense 0-1 matrices with $\Theta(n(\log n/\log\log n)^t)$ 1s that avoid an acyclic $X_t$. Pach and Tardos have conjectured that this type of result is the best possible, i.e., no acyclic $P$ exists for which $\mathrm{Ex}(P,n)\geq n(\log n)^{\omega(1)}$.
No Transfers Required: Integrating Last Mile with Public Transit Using Opti-Mile

Public transit is a popular mode of transit due to its affordability, despite the inconveniences due to the necessity of transfers required to reach most areas. For example, in the bus and metro network of New Delhi, only 30% of stops can be directly accessed from any starting point, thus requiring transfers for most commutes. Additionally, first/last mile services like rickshaws, tuk-tuks or shuttles are commonly used as feeders to the nearest public transit access points, which leads to subpar utilisation of the transit network. Ultimately, users often face a trade-off between coverage and transfers to reach their destination, regardless of the mode of transit. To address the problem of limited accessibility and inefficiency due to transfers in public transit systems, we propose “optimile,” a novel trip planning approach that combines first/last mile services with public transit such that no transfers are required11Our work is based on the assumption that the transfers from first/last mile are necessary for any public transit commute, whether it be through walking or taking a first/last mile service. Thus any further mention of transfers specifically refers to transfers within public transit, unless otherwise mentioned, Opti-mile allows users to customise trip parameters such as maximum walking distance, and acceptable fare range. We analyse the transit network of New Delhi, evaluating the efficiency, feasibility and advantages of opti-mile multi-modal trips between randomly selected source-destination pairs. We demonstrate that opti-mile trips lead to a 10% reduction in distance travelled for 18% increase in average price compared to traditional shortest paths using just public transit. We also show that opti-mile trips provide better coverage of the city than public transit, without a significant fare increase.
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution

Genomic (DNA) sequences encode an enormous amount of information for gene regulation and protein synthesis. Similar to natural language models, researchers have proposed foundation models in genomics to learn generalizable features from unlabeled genome data that can then be fine-tuned for downstream tasks such as identifying regulatory elements. Due to the quadratic scaling of attention, previous Transformer-based genomic models have used 512 to 4k tokens as context (<0.001% of the human genome), significantly limiting the modeling of long-range interactions in DNA. In addition, these methods rely on tokenizers to aggregate meaningful DNA units, losing single nucleotide resolution where subtle genetic variations can completely alter protein function via single nucleotide polymorphisms (SNPs). Recently, Hyena, a large language model based on implicit convolutions was shown to match attention in quality while allowing longer context lengths and lower time complexity. Leveraging Hyenas new long-range capabilities, we present HyenaDNA, a genomic foundation model pretrained on the human reference genome with context lengths of up to 1 million tokens at the single nucleotide-level, an up to 500x increase over previous dense attention-based models. HyenaDNA scales sub-quadratically in sequence length (training up to 160x faster than Transformer), uses single nucleotide tokens, and has full global context at each layer. We explore what longer context enables - including the first use of in-context learning in genomics for simple adaptation to novel tasks without updating pretrained model weights. On fine-tuned benchmarks from the Nucleotide Transformer, HyenaDNA reaches state-of-the-art (SotA) on 12 of 17 datasets using a model with orders of magnitude less parameters and pretraining data. On the GenomicBenchmarks, HyenaDNA surpasses SotA on all 8 datasets on average by +9 accuracy points.