metadata

language:
  - en
pipeline_tag: text-classification
metrics:
  - f1
  - accuracy
  - recall
  - precision
library_name: transformers
widget:
  - text: >-
      The past 25 years have seen a strong increase in the number of
      publications related to criticality in different areas of neuroscience.
      The potential of criticality to explain various brain properties,
      including optimal information processing, has made it an increasingly
      exciting area of investigation for neuroscientists. Recent reviews on this
      topic, sometimes termed brain criticality, make brief mention of clinical
      applications of these findings to several neurological disorders such as
      epilepsy, neurodegenerative disease, and neonatal hypoxia. Other
      clinicallyrelevant domains - including anesthesia, sleep medicine,
      developmental-behavioral pediatrics, and psychiatry - are seldom discussed
      in review papers of brain criticality. Thorough assessments of these
      application areas and their relevance for clinicians have also yet to be
      published. In this scoping review, studies of brain criticality involving
      human data of all ages are evaluated for their current and future clinical
      relevance. To make the results of these studies understandable to a more
      clinical audience, a review of the key concepts behind criticality (e.g.,
      phase transitions, long-range temporal correlation, self-organized
      criticality, power laws, branching processes) precedes the discussion of
      human clinical studies. Open questions and forthcoming areas of
      investigation are also considered.

MIReAD Neuro

This model is a fine-tuned version of arazd/MIReAD on a dataset of Neuroscience papers from 200 journals collected from various sources for a journal classification task. It achieves the following results on the evaluation set:

Loss: 2.7117
Accuracy: 0.4011
F1: 0.3962
Precision: 0.4066
Recall: 0.3999

Model description

This model was trained on a journal classification task.

Intended uses & limitations

The intended use of this model is to create abstract embeddings for semantic similarity search for neuroscience-related articles.

Model Usage

To load the model:

from transformers import BertForSequenceClassification, AutoTokenizer
model_path = "biodatlab/MIReAD-Neuro"
model = BertForSequenceClassification.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)

To create embeddings and for classification:

# sample abstract & title text
title = "Why Brain Criticality Is Clinically Relevant: A Scoping Review."
abstract = "The past 25 years have seen a strong increase in the number of publications related to criticality in different areas of neuroscience..."
text = title + tokenizer.sep_token + abstract
tokens = tokenizer(
    text,
    max_length=512,
    padding=True,
    truncation=True,
    return_tensors="pt"
)

# to generate an embedding from a given title and abstract
with torch.no_grad():
  output = model.bert(**tokens)
  embedding = output.last_hidden_state[:, 0, :]

# to classify (200 journals) a given title and abstract
output = model(**tokens)
class = output.logits

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 16
eval_batch_size: 16
num_epochs: 6