MembraneBERT: A Domain-Tailored NLP Model for Membrane-Based Gas Separation Data Extraction

Separation process is the core link of chemical production, widely used in purification, resource recovery and environmental protection, etc., and the energy consumption accounts for about 45% of the total energy consumption of the industry. Membrane separation technology is getting more and more attention due to its advantages of low energy consumption, simple operation and easy regulation, in which the performance of membrane material is the key to determine the separation efficiency.

At present, a large amount of literature has reported the experimental data and industrial application cases related to membrane separation, which can truly reflect the performance of materials under different conditions, with wide coverage. The automatic extraction of unstructured information in the literature by natural language processing (NLP) technology can not only significantly improve the processing efficiency and reduce the labor cost, but also accelerate the data structuring and integration, which can help the research and application of membrane separation.

MembraneBERT is a model based on the pre-trained SciBERT model for extracting membrane separation–related information from the literature.

Requirements

• Python(>=3)

Python modules（version used in this work）

• pandas (2.0.3)
• numpy (1.24.3)
• scipy (1.10.1)
• transformers (4.46.3)
• datasets (2.19.1)
• torch (2.4.1)
• bs4 (0.0.2)
• requests (2.32.3)

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 8
• num_epochs: 5.0
• lr_scheduler_type="cosine"

How to Use:

Here is how to use this model to get the features of a given text in PyTorch:

from transformers import AutoTokenizer, AutoModelForTokenClassification
tokenizer = AutoTokenizer.from_pretrained('yinyincao/MembraneBERT')
model = AutoModelForTokenClassification.from_pretrained('yinyincao/MembraneBERT')
ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer, aggregation_strategy="simple")
text = "Enter any text you like"
entities = ner_pipeline(text)