UKP-SQuARE/bert-base-uncased-pf-race-onnx

ONNX export of Adapter AdapterHub/bert-base-uncased-pf-race for bert-base-uncased

Conversion of AdapterHub/bert-base-uncased-pf-race for UKP SQuARE

Usage

onnx_path = hf_hub_download(repo_id='UKP-SQuARE/bert-base-uncased-pf-race-onnx', filename='model.onnx') # or model_quant.onnx for quantization
onnx_model = InferenceSession(onnx_path, providers=['CPUExecutionProvider'])

context = 'ONNX is an open format to represent models. The benefits of using ONNX include interoperability of frameworks and hardware optimization.'
question = 'What are advantages of ONNX?'
choices = ["Cat", "Horse", "Tiger", "Fish"]tokenizer = AutoTokenizer.from_pretrained('UKP-SQuARE/bert-base-uncased-pf-race-onnx')

raw_input = [[context, question +  + choice] for choice in choices]
inputs = tokenizer(raw_input, padding=True, truncation=True, return_tensors="np")
inputs['token_type_ids'] = np.expand_dims(inputs['token_type_ids'], axis=0)
inputs['input_ids'] =  np.expand_dims(inputs['input_ids'], axis=0)
inputs['attention_mask'] =  np.expand_dims(inputs['attention_mask'], axis=0)
outputs = onnx_model.run(input_feed=dict(inputs), output_names=None)

Architecture & Training

The training code for this adapter is available at https://github.com/adapter-hub/efficient-task-transfer. In particular, training configurations for all tasks can be found here.

Evaluation results

Refer to the paper for more information on results.

Citation

If you use this adapter, please cite our paper "What to Pre-Train on? Efficient Intermediate Task Selection":

@inproceedings{poth-etal-2021-what-to-pre-train-on,
    title={What to Pre-Train on? Efficient Intermediate Task Selection},
    author={Clifton Poth and Jonas Pfeiffer and Andreas Rücklé and Iryna Gurevych},
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/2104.08247",
    pages = "to appear",
}