kenchenxingyu/joint-fmri-text-model2

🧠 Joint fMRI-Text Model This model jointly predicts cognitive response type, trial type, and generates a 4D fMRI-like brain activation tensor based on natural language input and user-level metadata.

🧩 Inputs Text: a belief or statement in natural language (e.g., "Handwashing reduces disease risk."), processed using distilbert-base-uncased.

Metadata: a vector of 14 user features including:

Scaled continuous inputs: Age_scaled, Openness_scaled, Conscientiousness_scaled, Extraversion_scaled, Agreeableness_scaled, Neuroticism_scaled, ICAR_Total_scaled, MOCA_scaled, VMN_Sum_scaled

Encoded categorical features: Gender_encoded, Education_encoded, Ethnicity_fused_encoded, Income_level_encoded, VCBS_cat_encoded

🎯 Outputs Response Type: probabilities over 4 possible response categories

Trial Type: probabilities over 4 trial categories

fMRI Tensor: synthetic output of shape (74 × 74 × 52 × 4) representing brain activity across four timepoints

🚀 Example Usage python Copy Edit from transformers import AutoTokenizer from joint_fmri_model import JointFMRIModelWithHub import torch

model = JointFMRIModelWithHub.from_pretrained("kenchenxingyu/joint-fmri-text-model") tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

text = "I think regular hand washing reduces disease risk." inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)

with torch.no_grad(): text_vec = model.text_encoder(inputs["input_ids"]).squeeze(0)

meta_input = torch.rand(1, 14) # Replace with realistic metadata

output = model(text_vec.unsqueeze(0), meta_input)

print("Response:", output["response_probs"]) print("Trial:", output["trial_probs"]) print("fMRI shape:", output["fmri"].shape) 📂 Files pytorch_model.bin: trained model weights

config.json: model configuration

README.md: this file

🏷 License This model is released under an open academic research license. For other use cases, please contact the author.