Saravanan666/pypi-org huggingface dataset:
Package: bert-document-classification
:book: BERT Long Document Classification :book:an easy-to-use interface to fully trained BERT based models for multi-class and multi-label long document classification.pre-trained models are currently available for two clinical note (EHR) phenotyping tasks: smoker identification and obesity detection.To sustain future development and improvements, we interfacepytorch-transformersfor all language model components of our architectures. Additionally, their is ablog postdescribing the architecture.ModelDataset# LabelsEvaluation F1n2c2_2006_smoker_lstmI2B2 2006: Smoker Identification40.981n2c2_2008_obesity_lstmI2B2 2008: Obesity and Co-morbidities Identification150.997InstallationInstall with pip:pip install bert_document_classificationor directly:pip install git+https://github.com/AndriyMulyar/bert_document_classificationUseMaps text documents of arbitrary length to binary vectors indicating labels.frombert_document_classification.modelsimportSmokerPhenotypingBertfrombert_document_classification.modelsimportObesityPhenotypingBertsmoking_classifier=SmokerPhenotypingBert(device='cuda',batch_size=10)#defaults to GPU predictionobesity_classifier=ObesityPhenotypingBert(device='cpu',batch_size=10)#or CPU if you would like.smoking_classifier.predict(["I'm a document! Make me long and the model can still perform well!"])Moreexamples.NotesFor training you will need a GPU.For bulk inference where speed is not of concern lots of available memory and CPU cores will likely work.Model downloads are cached in~/.cache/torch/bert_document_classification/. Try clearing this folder if you have issues.AcknowledgementIf you found this project useful, consider citing our extended abstract accepted at NeurIPS 2019 ML4Health .Format bibtex citationImplementation, development and training in this project were supported by funding from the Mark Dredze Lab at Johns Hopkins University.
Package: bert-embedding
Bert EmbeddingsBERT, published byGoogle, is new way to obtain pre-trained language model word representation. Many NLP tasks are benefit from BERT to get the SOTA.The goal of this project is to obtain the token embedding from BERT's pre-trained model. In this way, instead of building and do fine-tuning for an end-to-end NLP model, you can build your model by just utilizing or token embedding.This project is implemented with@MXNet. Special thanks to@gluon-nlpteam.Installpip install bert-embedding
# If you want to run on GPU machine, please install `mxnet-cu92`.
pip install mxnet-cu92Usagefrombert_embeddingimportBertEmbeddingbert_abstract="""We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context in all layers.As a result, the pre-trained BERT representations can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications.BERT is conceptually simple and empirically powerful.It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE benchmark to 80.4% (7.6% absolute improvement), MultiNLI accuracy to 86.7 (5.6% absolute improvement) and the SQuAD v1.1 question answering Test F1 to 93.2 (1.5% absolute improvement), outperforming human performance by 2.0%."""sentences=bert_abstract.split('\n')bert_embedding=BertEmbedding()result=bert_embedding(sentences)If you want to use GPU, please import mxnet and set contextimportmxnetasmxfrombert_embeddingimportBertEmbedding...ctx=mx.gpu(0)bert=BertEmbedding(ctx=ctx)This result is a list of a tuple containing (tokens, tokens embedding)For example:first_sentence=result[0]first_sentence[0]# ['we', 'introduce', 'a', 'new', 'language', 'representation', 'model', 'called', 'bert', ',', 'which', 'stands', 'for', 'bidirectional', 'encoder', 'representations', 'from', 'transformers']len(first_sentence[0])# 18len(first_sentence[1])# 18first_token_in_first_sentence=first_sentence[1]first_token_in_first_sentence[1]# array([ 0.4805648 ,  0.18369392, -0.28554988, ..., -0.01961522,#        1.0207764 , -0.67167974], dtype=float32)first_token_in_first_sentence[1].shape# (768,)OOVThere are three ways to handle oov, avg (default), sum, and last. This can be specified in encoding....bert_embedding=BertEmbedding()bert_embedding(sentences,'sum')...Available pre-trained BERT modelsbook_corpus_wiki_en_uncasedbook_corpus_wiki_en_casedwiki_multilingualwiki_multilingual_casedwiki_cnbert_12_768_12✓✓✓✓✓bert_24_1024_16x✓xxxExample of using the large pre-trained BERT model from Googlefrombert_embeddingimportBertEmbeddingbert_embedding=BertEmbedding(model='bert_24_1024_16',dataset_name='book_corpus_wiki_en_cased')Source:gluonnlp
Package: bert-embeddings
Bert EmbeddingsUse this library to really easily embed text using Bert Models.Github:https://github.com/sorcely/BertEmbeddingsChange Log0.0.10 (30/04/2021)Removed print statement0.0.9 (17/04/2021)0.0.8 (24/01/2021)Removed torch from requirements.txt0.0.7 (24/01/2021)Exception stuff still didn’t work, so I changed it to just all exceptions0.0.6 (24/01/2021)Changed ImportError to ModuleNotFoundError0.0.5 (24/01/2021)Assert error if torch is not installed. This is a non-permanent solution to error when installing PyTorch0.0.3 (23/01/2021)Fixed Installment issue0.0.2 (23/01/2021)Fixed not finding package0.0.1 (22/01/2021)First release
Package: bert-experimental
bert_experimentalContains code and supplementary materials for a series of Medium articles about the BERT model.pretraining:https://towardsdatascience.com/pre-training-bert-from-scratch-with-cloud-tpu-6e2f71028379feature_extraction:https://medium.com/@gaphex/building-a-search-engine-with-bert-and-tensorflow-c6fdc0186c8afinetuning:https://medium.com/@gaphex/fine-tuning-bert-with-keras-and-tf-module-ed24ea91cff2representation:https://towardsdatascience.com/improving-sentence-embeddings-with-bert-and-representation-learning-dfba6b444f6b
Package: bert-for-sequence-classification
bert-for-sequence-classificationPipeline for easy fine-tuning of BERT architecture for sequence classificationQuick StartInstallationInstall the librarypip install bert-for-sequence-classificationIf you want to train you model on GPU, please install pytorch version compatible with your device.To find the version compatible with the cuda installed on your GPU, checkPytorch website.
You can learn CUDA version installed on your device by typingnvidia-smiin console or!nvidia-smiin a notebook cell.CLI Usebert-clf-train --path_to_config <path to yaml file>Example config file can be foundhereJupyter notebookExample notebook can be foundhereInference modeWhen using your trained model for inference it depends on how you saved your modelif path_to_state_dict inconfigis equal to false,
then if you have the library installed:importtorchimportpandasaspddevice=torch.device("cuda"iftorch.cuda.is_available()else"cpu")model=torch.load("path_to_saved_model",map_location=device)model.eval()df=pd.read_csv("path_to_some_df")df["target_column"]=df["text_column"].apply(model.predict)Otherwise:importtorchimportjsonimportpandasaspdfrombert_clf.src.models.BertCLFimportBertCLFfromtransformersimportAutoModel,AutoTokenizerdevice=torch.device("cuda"iftorch.cuda.is_available()else"cpu")tokenizer=AutoTokenizer.from_pretrained(pretrained_model_name_or_path="pretrained_model_name_or_path")model_bert=AutoModel.from_pretrained(pretrained_model_name_or_path="pretrained_model_name_or_path").to(device)id2label=json.load(open("path/to/saved/mapper"))# mapper is saved with the state dictmodel=BertCLF(pretrained_model=model_bert,tokenizer=tokenizer,id2label=id2label,dropout="some number",device=device)model.load_state_dict(torch.load("path_to_state_dict",map_location=device),strict=False)model.eval()df=pd.read_csv("path_to_some_df")df["target_column"]=df["text_column"].apply(model.predict)
Package: bert-for-tf2e
BERT for TensorFlow 2.11.0This is a modification version of the originalbert-for-tf2created bykpe. I made a minor change to the code
to make it work with never version of TensorFlow, following the solution that i found in github community.
Resolving the TypeError issue. Last time checked at 1/23/2023 - it worked fine.This repo contains aTensorFlow 2.11.0_Keras_ implementation ofgoogle-research/bert_
with support for loading of the originalpre-trained weights_,
and producing activationsnumerically identicalto the one calculated by the original model.ALBERT_ andadapter-BERT_ are also supported by setting the corresponding
configuration parameters (shared_layer=True,embedding_sizeforALBERT_
andadapter_sizeforadapter-BERT_). Setting both will result in an adapter-ALBERT
by sharing the BERT parameters across all layers while adapting every layer with layer specific adapter.The implementation is build from scratch using only basic tensorflow operations,
following the code ingoogle-research/bert/modeling.py_
(but skipping dead code and applying some simplifications). It also utilizeskpe/params-flow_ to reduce
common Keras boilerplate code (related to passing model and layer configuration arguments).bert-for-tf2e_ should work with bothTensorFlow 2.11.0_ andTensorFlow 1.14_ or newer.Installbert-for-tf2ebert for tensorflow 2.0 (extended) is on the Python Package Index (PyPI):::pip install bert-for-tf2eFor more detail please check the original version:SOURCE_ -https://github.com/kpe/bert-for-tf2
Package: berth
Berth is a tool to help you automate the building of packages for an array of operating system and package systems, while not putting a big load on the local dependencies where you build the packages.It does this by using a Docker container of your choice to build your project inside, and then use another container with theFPMpackage creator inside to package up the project.Read more onGitHub.Changelog1.3.5 - 2017-05-05Updated to support version 1.24 of the Docker API.Updated to work with new version oftenzer/fpm.1.3.4 - 2015-10-05Somehow the fix in 1.3.2 got lost, so we are adding that back in again.1.3.3 - 2015-10-05Bumped dependencies on docker-py and click. The former has a bug in < 1.3.1 where the streaming output from a container can suddenly stop while the container keeps running.1.3.2 - 2015-05-17Fixed packaging problem about missing CHANGELOG.md at installation time.1.3.1 - 2015-04-27Bumped dependencies on the docker-py and click libraries.Replaceddockerfiles/fpmwithtenzer/fpmas the default image to use for packaging as the dockerfiles image no longer exists.1.3.0 - 2015-03-09Added--build-onlyand--package-onlyflags to tell Berth to only do one of the two steps. This can be useful when setting up a configuration for a new package together with:New--keep-containersflag which makes the containers Berth create stay around after the jobs have ran. This will make you able todocker runthem afterwards and inspect the files inside of them.Berth now provides streaming output of the build scripts if either the verbose or debug output levels are enabled, instead of only getting the output when the container stops.When builds took longer time to execute than 100 seconds, the run time of the container was outputted in scientific format, this has now been corrected so it will always be outputted as the number of seconds with 1 digit precision.1.2.0 - 2015-02-12Made the “build” part of the configuration optional so you also can use Berth to just build Python and Ruby packages directly, since FPM supports that as input.1.1.0 - 2015-02-01Enable the use of environment variables during build and packaging steps. This makes it easier to create an updated package by just increasing the version number one place.When a local path specified as a source for a volume doesn’t exist, we now try to create a directory at that path and make use of that. If the directory creation fails then we complain like before.Minor improvements to configuration file checks.1.0.1 - 2015-01-29Stop the package generation flow if the build script fails (exits with non-zero exit code)Disable hostname verification for Docker client, since the hostname and certificate doesn’t match when using boot2docker - Fig does this as wellPlace the build script file in the current working directory of the host system instead of letting Python decide. It is only possible to mount paths inside/Userswhen using boot2docker on OS XMinor code changes in relation to Pylint code check1.0.0 - 2015-01-28Initial release
Package: ber-tlv
BER-TLV PackageSummaryThis is a BER-TLV (described in EMV Book 3) format encoder/decoder library for Python.
BER is Basic Encoding Rules.
TLV stands for Tag + Length + Value.
Tag is 1-4 byte long (integer) identifier.
Length is 1-4 byte integer representing value (or entire branch) size.
Value is binary data, which can also be a nested list of sub-items.
BER-TLV is a binary data storage tree, which is widely used in electonic payment industry and in EMV in particular.Installpip3 install ber-tlvUsageParse TLV:>>> from ber_tlv.tlv import *
>>> Tlv.parse(binascii.unhexlify("7F100DF303414243F4038A0135100100"))
[(32528, [(243, b'ABC'), (244, [(138, b'5')]), (16, b'\x00')])]Build TLV:>>> from ber_tlv.tlv import *
>>> Tlv.build({0x9F10:[(0x8A,b"ABC"),(0x8B,{0x10:b"\xf0\x0d"})]})
b'\x9f\x10\x0b\x8a\x03ABC\x8b\x04\x10\x02\xf0\r'Convert HEX bytes to string:>>> from ber_tlv.tlv import *
>>> Tlv.hexify_bytes(b"\x01\x23\x45\x67\x89\xab\xcd\xef")
'0123456789ABCDEF'Tag class checking:>>> from ber_tlv.tlv import *
>>> Tag.isConstructed(0x7f10)
True
>>> Tag.tagClass(0x1f10) == Tag.UNIVERSAL
True
>>> Tag.tagClass(0x7f10) == Tag.APPLICATION
True
>>> Tag.tagClass(0x9f10) == Tag.CONTEXT_SPECIFIC
True
>>> Tag.tagClass(0xdf10) == Tag.PRIVATE
TrueTechnical descriptionBelow is a brief technical description of BER-TLV format.Tag fieldTag has variable size of 1-4 bytes.
If first tag byte, bits 1-5 are set to 1, there are more tag bytes follow. In subsequent bytes, bit 8 indicates if this is a last byte. Other bits identify unique tag number.Length fieldLength field has variable size of 1-4 bytes.
When bit b8 of the most significant byte of the length field is set to 0, the length field consists of only one byte. Bits b7 to b1 code the number of bytes of the value field. The length field is within the range 1 to 127.When bit b8 of the most significant byte of the length field is set to 1, the subsequent bits b7 to b1 of the most significant byte code the number of subsequent bytes in the length field. The subsequent bytes code an integer representing the number of bytes in the value field.Value fieldValue is binary bytes array of specified length. Value may represent recursive sub-tree (branch).PaddingBefore, between, or after TLV-coded data objects, zero bytes without any meaning may occur (for example, due to erased or modified TLV-coded data objects).
Package: bertmoticon
BertmoticonThe Bertmoticon package is fine-tuned from theBERTmodel, to the emoji prediction task. It can predict emojis in 102 languages. In this package we include two functions that enable the use of it:bertmoticon.inferandbertmoticon.infer_mappings. The number of emojis available for this model are 80; and are listed inbertmoticon.emojis.InstallationInstalling the Bertmoticon package fromPyPIusing:pip3 install bertmoticonImporting in pythonImporting the package can be done as:import bertmoticonIf the model is not already downloaded; upon first run it will download and extract the model automatically as such:Downloading bermoticon model
[=                                                          ]
...
[==================                                         ]
...
[===========================================================]
Extracting the modelThe model is not included with the pypi installation. It requires 1.34 GB. Loads it either into CUDA or CPU based on CUDA availability.Usagebertmoticon.emojisThe model can predict up to 80 emojis. Acceessing the emojis can be done by calling the global variableemojiscalled asbertmoticon.emojis.>>> print(bertmoticon.emojis)
['😂', '😭', '😍', '😊', '🙏', '😅', '😁', '🙄', '😘', '😔', '😩', '😉', '😎', '😢', '😆', '😋', '😌', '😳', '😏', '🙂', '😃', '🙃', '😒', '😜', '😀', '😱', '🙈', '😄', '😡', '😬', '🙌', '😴', '😫', '😪', '😤', '😇', '😈', '😞', '😷', '😣', '😥', '😝', '😑', '😓', '😕', '😹', '😐', '😻', '😖', '😛', '😠', '🙊', '😰', '😚', '😲', '😶', '😮', '🙁', '😵', '😗', '😟', '😨', '🙇', '🙋', '😙', '😯', '🙆', '🙉', '😧', '😿', '😸', '🙀', '😦', '😽', '😺', '😼', '🙅', '😾', '🙍', '🙎']bertmoticon.inferTakes in alistofstringsand anintnumber of guesses. It returns a list of dictionaries, where each dictionary contains an emoji and a corresponding percentage.>>> ls_of_strings =  ["Vote #TRUMP2020ToSaveAmerica from corrupt Joe Biden and the radical left.","Je veux aller dormir. #fatigué"]
>>> print(bertmoticon.infer(ls_of_strings,3))
    [{'😂': '0.1938', '😡': '0.1866', '🙄': '0.0847'}, {'😴': '0.1547', '😭': '0.1507', '😩': '0.0892'}]bertmoticon.infer_mappingsTakes in alistofstrings, a dictionarydictof the emoji mappings, and anintnumber of guesses. It returns the number of occurences of each key value. We define the dictionary and the list as follows:>>> mappings = {"Anger":['😡'], "Other":['😂','😭']}
>>> ls_of_strings =  ["Vote #TRUMP2020ToSaveAmerica from corrupt Joe Biden and the radical left.","Je veux aller dormir. #fatigué"]The key values are the category names and the values are lists of the emojis contained in that category. Then parsed into thebertmoticon.infer_mappingsreturns:>>>print(bertmoticon.infer_mappings(ls_of_strings,mappings,3))
{'Anger': 1, 'Other': 2}
Package: bert-multitask-client
bert-multitask-as-serviceA serving service forbert-multitask-learningInstallpipinstallbert-multitask-server
pipinstallbert-multitask-clientGetting StartedTrain and export model.A typical trained checkpoint dir looks like below.bert_serving_ckpt/
├── *_label_encoder.pkl
├── bert_config.json
├── export_model
├── params.json
└── vocab.txtStart server using CLIbert-multitask-serving-start-model_dir~/CWS_NER_POS_ckpt/-num_worker=4-problem"CWS|NER|POS"Use Client to Get Predictionfrombert_serving.clientimportBertClientbc=BertClient()bc.encode(['我爱北京天安门'])Bert多任务学习服务一个部署Bert多任务学习的服务安装pipinstallbert-multitask-server
pipinstallbert-multitask-client开始使用训练模型并导出模型.导出后的模型目录应该有以下文件bert_serving_ckpt/
├── *_label_encoder.pkl
├── bert_config.json
├── export_model
├── params.json
└── vocab.txt用CLI启动服务bert-multitask-serving-start-model_dirmodels/-num_worker=4-problem"fake_problem"用客户端获取预测结果frombert_serving.clientimportBertClientbc=BertClient()bc.encode(['我爱北京天安门'])
Package: bert-multitask-learning
Bert for Multi-task Learningpython中文文档Note: Since 0.4.0, tf version >= 2.1 is required.Installpip install bert-multitask-learningWhat is itThis a project that uses transformers(based on huggingface transformers) to domulti-modal multi-task learning.Why do I need thisIn the original BERT code, neither multi-task learning or multiple GPU training is possible. Plus, the original purpose of this project is NER which dose not have a working script in the original BERT code.To sum up, compared to the original bert repo, this repo has the following features:Multimodal multi-task learning(major reason of re-writing the majority of code).Multiple GPU trainingSupport sequence labeling (for example, NER) and Encoder-Decoder Seq2Seq(with transformer decoder).What type of problems are supported?Masked LM and next sentence prediction Pre-train(pretrain)Classification(cls)Sequence Labeling(seq_tag)Multi-Label Classification(multi_cls)Multi-modal Mask LM(mask_lm)How to run pre-defined problemsThere are two types of chaining operations can be used to chain problems.&. If two problems have the same inputs, they can be chained using&. Problems chained by&will be trained at the same time.|. If two problems don't have the same inputs, they need to be chained using|. Problems chained by|will be sampled to train at every instance.For example,cws|NER|weibo_ner&weibo_cws, one problem will be sampled at each turn, sayweibo_ner&weibo_cws, thenweibo_nerandweibo_cwswill trained for this turn together. Therefore, in a particular batch, some tasks might not be sampled, and their loss could be 0 in this batch.Please see the examples innotebooksfor more details about training, evaluation and export models.Bert多任务学习注意：版本0.4.0后要求tf>=2.1安装pip install bert-multitask-learning这是什么这是利用transformer(基于huggingface transformers)进行多模态多任务学习的项目.我为什么需要这个项目在原始的BERT代码中, 是没有办法直接用多GPU进行多任务学习的. 另外, BERT并没有给出序列标注和Seq2seq的训练代码.因此, 和原来的BERT相比, 这个项目具有以下特点:多任务学习多GPU训练序列标注以及Encoder-decoder seq2seq的支持(用transformer decoder)目前支持的任务类型Masked LM和next sentence prediction预训练(pretrain)单标签分类(cls)序列标注(seq_tag)多标签分类(multi_cls)多模态Mask LM(mask_lm)如何运行预定义任务可以用两种方法来将多个任务连接起来.&. 如果两个任务有相同的输入, 不同标签的话, 那么他们可以用&来连接. 被&连接起来的任务会被同时训练.|. 如果两个任务为不同的输入, 那么他们必须用|来连接. 被|连接起来的任务会被随机抽取来训练.例如, 我们定义任务cws|NER|weibo_ner&weibo_cws, 那么在生成每一条数据时, 一个任务块会被随机抽取出来, 例如在这一次抽样中,weibo_ner&weibo_cws被选中. 那么这次weibo_ner和weibo_cws会被同时训练. 因此, 在一个batch中, 有可能某些任务没有被抽中, loss为0.训练, eval和导出模型请见notebooks
Package: bert-multitask-server
bert-multitask-as-serviceA serving service forbert-multitask-learningInstallpipinstallbert-multitask-server
pipinstallbert-multitask-clientGetting StartedTrain and export model.A typical trained checkpoint dir looks like below.bert_serving_ckpt/
├── *_label_encoder.pkl
├── bert_config.json
├── export_model
├── params.json
└── vocab.txtStart server using CLIbert-multitask-serving-start-model_dir~/CWS_NER_POS_ckpt/-num_worker=4-problem"CWS|NER|POS"Use Client to Get Predictionfrombert_serving.clientimportBertClientbc=BertClient()bc.encode(['我爱北京天安门'])Bert多任务学习服务一个部署Bert多任务学习的服务安装pipinstallbert-multitask-server
pipinstallbert-multitask-client开始使用训练模型并导出模型.导出后的模型目录应该有以下文件bert_serving_ckpt/
├── *_label_encoder.pkl
├── bert_config.json
├── export_model
├── params.json
└── vocab.txt用CLI启动服务bert-multitask-serving-start-model_dirmodels/-num_worker=4-problem"fake_problem"用客户端获取预测结果frombert_serving.clientimportBertClientbc=BertClient()bc.encode(['我爱北京天安门'])
Package: bertopic-base-chinese
About一个基于bertopic对中文文档进行主题建模的包。Install$ pip3 install -U bertopic_base_chineseDirectorbertopic_base_chinese_model.py_model.pyBERTopic类
重写了__init__()，设置embedding_model为"paraphrase-multilingual-MiniLM-L12-v2"，以及选取tokenizer为jieba.lcut，初始化类参数。Usagefrombertopic_base_chineseimportBERTopicdocs=["我爱北京天安门","我家大门常打开，开放怀抱等你"]topic_model=BERTopic()topics,probs=topic_model.fit_transform(docs)Contact usmay.xiaoya.zhang@gmail.com