File size: 1,945 Bytes
b542ffd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | 🧠 Joint fMRI-Text Model
This model jointly predicts cognitive response type, trial type, and generates a 3D fMRI-like brain activation tensor based on natural language input and user-level metadata.
🧩 Inputs
Text: a belief or statement in natural language (e.g., "Handwashing reduces disease risk."), processed using distilbert-base-uncased.
Metadata: a vector of 14 user features including:
Scaled continuous inputs: Age_scaled, Openness_scaled, Conscientiousness_scaled, Extraversion_scaled, Agreeableness_scaled, Neuroticism_scaled, ICAR_Total_scaled, MOCA_scaled, VMN_Sum_scaled
Encoded categorical features: Gender_encoded, Education_encoded, Ethnicity_fused_encoded, Income_level_encoded, VCBS_cat_encoded
🎯 Outputs
Response Type: probabilities over 4 possible response categories
Trial Type: probabilities over 3 trial categories
fMRI Tensor: synthetic output of shape (74 × 74 × 52) representing brain activity across four timepoints
🚀 Example Usage
from transformers import AutoTokenizer
from joint_fmri_model import JointFMRIModelWithHub
import torch
model = JointFMRIModelWithHub.from_pretrained("kenchenxingyu/joint-fmri-text-model")
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
text = "I think regular hand washing reduces disease risk."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
text_vec = model.text_encoder(inputs["input_ids"]).squeeze(0)
meta_input = torch.rand(1, 14) # Replace with realistic metadata
output = model(text_vec.unsqueeze(0), meta_input)
print("Response:", output["response_probs"])
print("Trial:", output["trial_probs"])
print("fMRI shape:", output["fmri"].shape)
📂 Files
pytorch_model.bin: trained model weights
config.json: model configuration
README.md: this file
🏷 License
This model is released under an open academic research license. For other use cases, please contact the author.
|