metadata

datasets: []
language: []
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - UniHGKR-base
widget: []

UniHGKR-base

Our paper: UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrievers.

Please see github repository UniHGKR to know how to use this model.

We recommend using the sentence-transformers package to load our model and to perform embedding for paragraphs and sentences.

It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Training Details

Framework Versions

Python: 3.8.10
Sentence Transformers: 3.0.1
Transformers: 4.44.2
PyTorch: 2.0.0+cu118
Accelerate: 0.34.0
Datasets: 2.21.0
Tokenizers: 0.19.1

Citation

If you find this resource useful in your research, please consider giving a like and citation.

@article{min2024unihgkr,
  title={UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrievers},
  author={Min, Dehai and Xu, Zhiyang and Qi, Guilin and Huang, Lifu and You, Chenyu},
  journal={arXiv preprint arXiv:2410.20163},
  year={2024}
}