SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For all baselines, we use the Transformer as the backbone encoder. The number of layers is 3, the embedding dimension is 128, the number of attention heads is 2. The event embedding look-up table is initialized with ClinicalBERT [3] embeddings of the event name and then project it down to 128 dimension with a linear layer. Patient demographics features (age, gender, and ethnicity) are separately embedded with another embedding look-up table. We also embed the timestamps with sinusoidal positional encoding. The medical, patient demographics, and temporal embeddings are added together to form the overall sequence embedding. For SLDG, UMAP [30] from UMAP-learn [41] is used with 2 components, 10 neighbors, and 0 minimum distance; and k-Means from Scikit-learn [35] is used with the default hyper-parameter. We apply a dropout of rate 0.2. We use Adam as the optimizer with a learning rate of 1e-4 and a weight decay of 1e-5. All models are trained for 100 epochs. The batch size is 256. We select the best model by monitoring the AUPRC score on the source validation set (except for the Oracle baseline, where we directly use the target validation set). We implement SLDG using PyTorch [34] 1.11 and Python 3.8. The model is trained on a CentOS Linux 7 machine with 128 AMD EPYC 7513 32-Core Processors, 512 GB memory, and eight NVIDIA RTX A6000 GPUs.
C Limitations and Broader Impacts
In terms of limitations, it is important to acknowledge that our work operates under the assumption that the target testing data still exhibit some similarities with the source training data. If there is
a significant distribution shift, the knowledge acquired from the source training data may become irrelevant. In such cases, neither the DG baselines nor our proposed method can effectively address the problem. It would be more appropriate to explore transfer learning or train a new model to obtain a better solution. Further, we propose SLDG to tackle two main challenges: (1) unknown domain IDs and (2) distinct characteristics across domains. In the scenario when the domain IDs are given and clearly separable (e.g., photo, art painting, cartoon, and sketch in the PACS [21] dataset), SLDG's domain discovery approach might be unnecessary. Existing DG methods directly utilizing the domain IDs could be a better solution.
In terms of broader impacts, our work tackles a practical and prevalent issue in healthcare known as the domain shift problem. We aim to inspire future research in this area: both by investigating the existence of domain shift under various scenarios, and by contributing to the development of effective solutions for this real-world challenge.
D Notations
Notation	Meaning
x	a patient's hospital visit
[e1, ..., em]	sequence of m events
t	type of an event
T(·)	mapping function from event to its type
E	set of all events
T	set of all event types
y ∈ {+, -}	label, i.e., the occurrence of a certain future event
fφ(·)	overall clinical predictive model
φ	model parameter
l(·)	loss function
Ptr, Pte	training and testing data distribution
Et(·)	feature-specific patient encoder for event type t
ht	patient representation in latent space of type t
h	hidden dimension
{ht(i)}Ntr	all patient representations in latent space of type t
Ntr	total number of training samples
Kt	number of discovered domains in the latent space of type t
Mt	domain assignment matrix
[el, ..., em]	contextualized representation for event sequence [e1, ..., em]
E(·)	embedding function
{ht}t∈T	multi-vector representations for a single patient
Ct,k(·)	customized classifier for the discovered domain k in the latent space of type t
Gt,k(·)	the gating weight for the customized classifier Ct,k(·)
o	model output
wt,k+, wt-k-	learnable prototype weight vectors of the positive and negative classes for the k-th discovered domain in the latent space of type t
d(·, ·)	Euclidean distance
wt,k	learnable prototypical weight vector for the discovered domain k in the latent space of type t
E Additional Illustrations
Figure 6: An illustration of the patient visit as input.
1. Whether the patient will decease at the end of current ICU stay?
2. Whether the patient will be readmitted to ICU within 14 days?
Figure 7: An illustration of the task definitions in the eICU and the MIMIC-IV datasets.
1. Whether the patient will decease within 90 days?
2. Whether the patient will be readmitted to hospital within 14 days?
Figure 8: The architecture of the feature-specific patient encoder.