Package: simplerepresentations
Simple RepresentationsThis library is based on theTransformerslibrary by HuggingFace. Using this library, you can quickly extract text representations from Transformer models. Only two lines of code are needed to initialize the required model and extract the text representations from it.Table of contentsInstallationWithpipFrom sourceUsageMinimal StartDefault SettingsCurrent Pretrained ModelsAcknowledgementsInstallationThis repository is tested on Python 3.6.8 and PyTorch 1.2.0WithpipFirst you need to install PyTorch. Please refer toPyTorch installation pageregarding the specific install command for your platform.When PyTorch has been installed, Simple Representation can be installed using pip as follows:pip install simplerepresentationFrom sourceHere also, you first need to install PyTorch. Please refer toPyTorch installation pageregarding the specific install command for your platform.When PyTorch has been installed, you can install from source by cloning the repository and running:pip install .UsageMinimal StartThe following example extracts the text representations fromBERT Base Uncasedmodel for the sentencesHello Transformers!andIt's very simple..fromsimplerepresentationsimportRepresentationModeldefload_data():return['Hello Transformers!','It\'s very simple.']if__name__=='__main__':model_type='bert'model_name='bert-base-uncased'representation_model=RepresentationModel(model_type=model_type,model_name=model_name,batch_size=32,max_seq_length=10,# truncate sentences to be less than or equal to 10 tokenscombination_method='cat',# concatenate the last `last_hidden_to_use` hidden stateslast_hidden_to_use=4# use the last 4 hidden states to build tokens representations)text_a=load_data()all_sentences_representations,all_tokens_representations=representation_model(text_a=text_a)print(all_sentences_representations.shape)# (2, 768) => (number of sentences, hidden size)print(all_tokens_representations.shape)# (2, 10, 3072) => (number of sentences, number of tokens, hidden size)You can change the code inload_datafunction to load your own data from any source you want (e.g. a CSV file).Default SettingsThe default settings forRepresentationModelclass are given below:batch_size (32): integerThe batch size will be used while extracting representations.max_seq_length (128): integerMaximum sequence length the model will support.last_hidden_to_use (1): integerThe number of the last hidden states that will be used to build the representations.combination_method ('sum'): string ('sum', 'cat')The method that will be used to combine thelast_hidden_to_use.use_cuda (True): booleanWhether to useCUDAor not.process_count (cpu_count() - 2 if cpu_count() > 2 else 1): integerNumber of CPU cores (processes) to use when converting examples to features. Default is (number of cores - 2) or 1 if (number of cores <= 2).chunksize (500): integerThe number of chunks that the examples will be divided to when converting them to features.Current Pretrained ModelsYou can find the complete list of the current pretrained models from Transformers librarydocumentation.AcknowledgementsNone of this would have been possible without the hard work by the HuggingFace team in developing theTransformerslibrary.Also, a lot of ideas used in this repository inspired from theSimple Transformerslibrary.