Saravanan666/pypi-org huggingface dataset:
Package: torch-geometric-edge
PyTorch-Geometric EdgePyTorch-Geometric Edge (PyGE) is a library that implements models for learning vector
representations of graph edges. It is build upon the popularPyGlibrary and allows to mix layers and models from both libraries in the same code.
Note that PyGE is still under development and model APIs may change in future revisions.InstallationCurrently, we test our code using Python 3.8, but newer versions should work
fine as well. We plan to extend our test suite in upcoming releases. The same
applies for the PyTorch version – we currently run our tests using PyTorch 1.10.0.For a detailed list of required packages, please take a look at thesetup.pyfile, where library versions are constrained.You can install PyGE using pip:$pipinstalltorch-geometric-edgeAvailable modelsCurrently, we implement the following edge-centric models:non-trainable node pair operators: Average, Hadamard, L1, L2node2edge and edge2node layersLine2vecAttrE2vecPairEDual Hypergraph TransformationLicenseMIT
Package: torchgeometry
ThePyTorch Geometrypackage is a geometric computer vision library forPyTorch.It consists of a set of routines and differentiable modules to solve generic geometry computer vision problems. At its core, the package usesPyTorchas its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions.InstallationFrom pip:pipinstalltorchgeometryFrom source:pythonsetup.pyinstallFrom source using pip:pipinstallgit+https://github.com/arraiyopensource/torchgeometryQuick Usageimporttorchimporttorchgeometryastgmx_rad=tgm.pi*torch.rand(1,3,3)x_deg=tgm.rad2deg(x_rad)torch.allclose(x_rad,tgm.deg2rad(x_deg))# TrueExamplesRun our Jupyter notebooksexamplesto learn to use the library.CiteIf you are using torchgeometry in your research-related documents, it is recommended that you cite the poster.@misc{Arraiy2018,author={E.Riba,M.Fathollahi,W.Chaney,E.RubleeandG.Bradski}title={torchgeometry:whenPyTorchmeetsgeometry},booktitle={PyTorchDeveloperConference},year={2018},url={https://drive.google.com/file/d/1xiao1Xj9WzjJ08YY_nYwsthE-wxfyfhG/view?usp=sharing}}ContributingWe appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us. Please, consider to read theCONTRIBUTINGnotes.
Package: torchgpipe
AGPipeimplementation inPyTorch.fromtorchgpipeimportGPipemodel=nn.Sequential(a,b,c,d)model=GPipe(model,balance=[1,1,1,1],chunks=8)forinputindata_loader:output=model(input)What is GPipe?GPipe is a scalable pipeline parallelism library published by Google Brain,
which allows efficient training of large, memory-consuming models. According to
the paper, GPipe can train a 25x larger model by using 8x devices (TPU), and
train a model 3.5x faster by using 4x devices.GPipe: Efficient Training of Giant Neural Networks using Pipeline ParallelismGoogle trained AmoebaNet-B with 557M parameters over GPipe. This model has
achieved 84.3% top-1 and 97.0% top-5 accuracy on ImageNet classification
benchmark (the state-of-the-art performance as of May 2019).LinksSource Code:https://github.com/kakaobrain/torchgpipeDocumentation:https://torchgpipe.readthedocs.io/Original Paper:https://arxiv.org/abs/1811.06965
Package: torch-gpt-2
PyTorch GPT-2Installpipinstalltorch-gpt-2Demoimportosimportsysfromtorch_gpt_2importload_trained_model_from_checkpoint,get_bpe_from_files,generateiflen(sys.argv)!=2:print('python3 demo.py MODEL_FOLDER')sys.exit(-1)model_folder=sys.argv[1]config_path=os.path.join(model_folder,'hparams.json')checkpoint_path=os.path.join(model_folder,'model.ckpt')encoder_path=os.path.join(model_folder,'encoder.json')vocab_path=os.path.join(model_folder,'vocab.bpe')print('Load net from checkpoint...')net=load_trained_model_from_checkpoint(config_path,checkpoint_path)print('Load BPE from files...')bpe=get_bpe_from_files(encoder_path,vocab_path)print('Generate text...')output=generate(net,bpe,['From the day forth, my arm'],length=20,top_k=1)# If you are using the 117M model and top_k equals to 1, then the result would be:# "From the day forth, my arm was broken, and I was in a state of pain. I was in a state of pain,"print(output[0])
Package: torch-grammar
torch-grammarAlpha Quality: This might do what you want. It's likely to not do what you
want. Please open issues!Torch-Grammar restricts a model to output a token sequence that conforms to a
provided EBNF grammar.For example:importtorchfromtorch_grammarimportGrammarSamplerfromtransformersimportLlamaTokenizertokenizer=LlamaTokenizer.from_pretrained("huggyllama/llama-7b")withopen("examples/grammar.ebnf","r")asfile:input_text=file.read()grammar=GrammarSampler(input_text,"root",tokenizer)ids=[[1]]logits_processor=grammar.logits_processor()vocab_size=len(tokenizer.get_vocab())foriinrange(10):logits=torch.randn((1,vocab_size))logits=logits_processor(ids,logits)token=torch.argmax(logits).item()# logits_processor.accept_token(token)ids[0].append(token)print(f"\x1b[1mfirst 10 tokens:\x1b[1;35m{tokenizer.decode(ids[0])}\x1b[0m")logits_processoris meant to be passed tomodel.generatein a HuggingFace
transformers model but this integration is not yet super clean.TODO / possible featuresUTF-8 support... a bit of fiddling but not terribly hardMore expressive grammars... lookahead would be challenging.Easier integration with various tokenizers... LLaMA works well; T5 presents
significant challenges; haven't even tried others.Testing and automatic benchmarking.Binary parse is probably not carrying its weight with all the caching and
precomputation we're doing now; it should be rewritten to something less
confusing. In fact it might work to just hijack Lark or something?Broken seeds11833882144218229242Related WorkThe code was originally adapted fromhttps://github.com/ggerganov/llama.cpp/pull/1773. In particular, the grammar
parser is a pretty straightforward mechanical translation and the binary grammar
format is identical.
Package: torchgs
torchgsPytorch wrapper for grid search of hyperparameters
[https://github.com/danny-1k/torch-gs]Install$ pip install torchgsExampleFinding the best set of hyper-parameters and models for a classification problemfromsklearn.datasetsimportmake_classificationimporttorchimporttorch.nnasnnfromtorch.utils.dataimportTensorDatasetfromtorchgsimportGridSearchfromtorchgs.metricsimportLossX,Y=make_classification(n_samples=200,n_features=20,n_informative=10,n_classes=2,shuffle=True,random_state=42)X=torch.Tensor(X).float()Y=torch.Tensor(Y).long()traindata=TensorDataset(X,Y)net1=nn.Sequential(nn.Linear(20,10),nn.ReLU(),nn.Linear(10,2))net2=nn.Sequential(nn.Linear(20,10),nn.Tanh(),nn.Linear(10,2))net3=nn.Sequential(nn.Linear(20,20),nn.ReLU(),nn.Linear(20,10),nn.ReLU(),nn.Linear(10,2))net4=nn.Sequential(nn.Linear(20,20),nn.Tanh(),nn.Linear(20,10),nn.Tanh(),nn.Linear(10,2))search_space={'trainer':{'net':[net1,net2,net3,net4],'optimizer':[torch.optim.Adam],'lossfn':[torch.nn.CrossEntropyLoss()],'epochs':list(range(11)),'metric':[Loss(torch.nn.CrossEntropyLoss())],},'train_loader':{'batch_size':[32,64],},'optimizer':{'lr':[1e-1,1e-2,1e-3,1e-4],},}searcher=GridSearch(search_space)results=searcher.fit(traindata)best=searcher.best(results,using='mean',topk=10,should_print=True)OutputtorchgsTrainerGridSearchmetricsoptimizerstorchgs.metricsMetricLossAccuracyRecallPrecisionF1torchgs.optimizersOptimizerLRschedulerTodoParallel Training on multiple GPUSTensorboard IntegrationPull requests are welcome, let's collab ðŸ¤².
Package: torchhk
pytorch-custom-utilsThis is a lightweight repository to help PyTorch users.Usage:clipboard: Dependenciestorch 1.4.0torchvision 0.5.0python 3.6matplotlib 2.2.2numpy 1.14.3seaborn 0.9.0sklearnplotly:hammer: Installationpip install torchhkorgit clone https://github.com/Harry24k/pytorch-custom-utilsfromtorchhkimport*:rocket: DemosRecordManager(code,markdown):
RecordManager will help you to keep tracking training records.Datasets(code,markdown):
Dataset will help you to use torch datasets including split and label-filtering.Supported datasets# CIFAR10datasets=Datasets("CIFAR10",root='./data')# CIFAR100datasets=Datasets("CIFAR100",root='./data')# STL10datasets=Datasets("STL10",root='./data')# MNISTdatasets=Datasets("MNIST",root='./data')# FashionMNISTdatasets=Datasets("FashionMNIST",root='./data')# SVHNdatasets=Datasets("SVHN",root='./data')# MNISTMdatasets=Datasets("MNISTM",root='./data')# ImageNetdatasets=Datasets("ImageNet",root='./data')# USPSdatasets=Datasets("USPS",root='./data')# TinyImageNetdatasets=Datasets("TinyImageNet",root='./data')# CIFAR with Unsuperviseddatasets=Datasets("CIFAR10U",root='./data')datasets=Datasets("CIFAR100U",root='./data')# Corrupted CIFAR (Only test data will be corrupted)# CORRUPTIONS = [#    'gaussian_noise', 'shot_noise', 'impulse_noise', 'defocus_blur',#    'glass_blur', 'motion_blur', 'zoom_blur', 'snow', 'frost', 'fog',#    'brightness', 'contrast', 'elastic_transform', 'pixelate',#    'jpeg_compression'#]datasets=Datasets("CIFAR10",root='./data',corruption='gaussian_noise')Vis(code,markdown):
Vis will help you to visualize torch tensors.Transform(code):
Transform will help you to change specific layers.ContributionContribution is always welcome! Usepull requests:blush:
Package: torch-householder
This package implements the Householder transformation algorithm for calculating orthogonal matrices and orthonormal
frames with differentiable bindings to PyTorch. In particular, the package provides an enhanced drop-in replacement for
thetorch.orgqrfunction, which was renamed intotorch.linalg.householder_productas of PyTorch 1.9.APIs for orthogonal transformations have been around since LAPACK; however, their support in the deep learning
frameworks is lacking. Recently, orthogonal constraints have become popular in deep learning as a way to regularize
models and improve training dynamics, and hence the need to backpropagate through orthogonal transformations arised.PyTorch 1.7 implements matrix exponential functiontorch.matrix_exp, which can be repurposed to performing the
orthogonal transformation when the input matrix is skew-symmetric. This is the baseline we use in Speed and Precision
evaluation.PyTorch 1.9 renamedtorch.orgqrintotorch.linalg.householder_product, and added support of autograd, batching, and
GPU execution.Compared totorch.matrix_exp, the Householder transformation implemented in this package has the following advantages:Orders of magnitude lower memory footprintAbility to transform non-square matrices (orthonormal frames)A significant speed-up for non-square matricesBetter numerical precision for all matrix and batch sizesFind more details and the most up-to-date information on the project webpage:https://www.github.com/toshas/torch-householder
Package: torchhydro
torchhydrodatasets, samplers, transforms, and pre-trained models for hydrology and water resourcesFree software: BSD licenseDocumentation:https://OuyangWenyu.github.io/torchhydroNOTE: THIS REPOSITORY ISSTILL UNDERDEVELOPMENT!!!FeaturesTODOFor developersTo install the environment, run the following code in the terminal:condaenvcreate-fenv-dev.yml
condaactivatetorchhydroTo use this repository of dev or other branches in your existing environment:you can fork it to your GitHub account. Don't choose "only fork the main branch" when forking in the Github page.run the following code in the terminal:# xxxxxx is your github account; here we choose to use dev branchpipinstallgit+ssh://git@github.com/xxxxxx/torchhydro.git@devFor the dataset we set a unified data path in settings.txt in the.hydrodatasetdirectory which is located in the user directory (for example,C:\Users\username\.hydrodatasetin Windows). You can change the data path in this file.Then we have some conventions for the dataset:Public datasets such as CAMELS is put in thewaterism/datasets-origindirectory.The processed datasets are put in thewaterism/datasets-interimdirectory.You can specify by yourself, but some changes are needed. We will optimize this part in the future.CreditsThis package is inspired byTorchGeo.It was created withCookiecutterand thegiswqs/pypackageproject template.
Package: torchility
torchilityA Pytorch-lightning based tool for training pytorch deep learning model more simply.Installpip install torchilityDependencypytorch>=2.0pytorch-lightning>=2.0,<2.1torchmetrics>=0.11,<0.12matplotlib>=3.3pyyaml>=5.4tensorboardX>=2.6UsageExampleMNISTfromtorchilityimportTrainerimporttorchfromtorchimportnnfromtorch.nnimportfunctionalasFfromtorchvision.datasetsimportMNISTfromtorchvisionimporttransformsfromtorch.utils.dataimportDataLoader,random_splitimportwarningswarnings.simplefilter("ignore")# ignore annoying warnings# datasetsdata_dir='./datasets'transform=transforms.Compose([transforms.ToTensor(),transforms.Normalize((0.1307,),(0.3081,))])mnist_full=MNIST(data_dir,train=True,transform=transform,download=True)train_ds,val_ds=random_split(mnist_full,[55000,5000])test_ds=MNIST(data_dir,train=False,transform=transform,download=True)# dataloaderstrain_dl=DataLoader(train_ds,batch_size=32)val_dl=DataLoader(val_ds,batch_size=32)test_dl=DataLoader(test_ds,batch_size=32)# pytorch modelchannels,width,height=(1,28,28)model=nn.Sequential(nn.Flatten(),nn.Linear(channels*width*height,64),nn.ReLU(),nn.Dropout(0.1),nn.Linear(64,64),nn.ReLU(),nn.Dropout(0.1),nn.Linear(64,10))# optimizeropt=torch.optim.Adam(model.parameters(),lr=2e-4)# trainertrainer=Trainer(model,F.cross_entropy,opt,epochs=2)# train and validatetrainer.fit(train_dl,val_dl)# testtrainer.test(test_dl)See theexamplesfor more examplesData Flow
Package: torchimize
torchimize contains implementations of the Gradient Descent, Gauss-Newton and Levenberg-Marquardt optimization algorithms using the PyTorch library. The main motivation for this project is to enable convex optimization on GPUs based on the torch.Tensor class, which (as of 2022) is widely used in the deep learning field. This package features the capability to minimize several least-squares optimization problems at each loop iteration in parallel.
Package: torchinstaller
PyTorch Install HelperSuper simple helper to install pytorch stuff without having to check cuda versions and go to websites for the installer URLs.
Installs based on requested or detected CUDA version, doesn't check python versions.Only Linux and macOS SupportedDetects what cuda version is available and runs the pip command to install latest pytorch and compatible cuda versionInstallationpipinstalltorchinstallerUsage$torchinstall-h
usage:torchinstall[-h][--pytorch[PYTORCH]][--pyg][--pyg-lib-source][--compute-platform{+,cu102,cu111,cu113,cu116,cu117,cu118,cu121,rocm4.0.1,rocm4.1,rocm4.2,rocm4.5.2,rocm5.1.1,rocm5.2,rocm5.4.2,rocm5.6,rocm5.7}][--lightning][--use{pip,conda,mamba}][-install][--sync]options:-h,--helpshowthishelpmessageandexit--pytorch[PYTORCH],-pt[PYTORCH]Flagtoinstallpytorch,canoptionallyspecifyadesiredversion.Mustbefullsemanticversion,e.g.1.13.1,not1.13,defaultsto`latest`--pyg,-pygFlagtoinstallpytorch-geometric--pyg-lib-source,-pyg-srcFlagtoinstallPyGfromsource.i.e.PyGdoesn'tsupportwheelsforM1/M2macs.Theyrecommendinstallingfromsource--compute-platform{+,cu102,cu111,cu113,cu116,cu117,cu118,cu121,rocm4.0.1,rocm4.1,rocm4.2,rocm4.5.2,rocm5.1.1,rocm5.2,rocm5.4.2,rocm5.6,rocm5.7},-c{+,cu102,cu111,cu113,cu116,cu117,cu118,cu121,rocm4.0.1,rocm4.1,rocm4.2,rocm4.5.2,rocm5.1.1,rocm5.2,rocm5.4.2,rocm5.6,rocm5.7}Manuallyspecifyplatformversion(cudaorrocm)insteadofauto-detect(usefulforclusterinstallations).--lightning,-lFlagtoinstalllightning(lightning.ai)--use{pip,conda,mamba},-u{pip,conda,mamba}setcommandtoinstallwith.-install,-iRuninstallation(defaultistodryruncommands)--sync,-supdateinstallationcommandsbyparsingthepytorchwebsiteNote: pytorch-geometric can be problematic to install. Installing from source has been added to facilitate installation, but referring to their documentation may be necessary to address errors if they occur.
Package: torch-intermediate-layer-getter
Simple easy to use module to get the intermediate results from chosen submodules. Supports submodule annidation. Inspired inthisbut does not assume that submodules are executed sequentially.Installationpipinstalltorch_intermediate_layer_getterUsageExampleimporttorchimporttorch.nnasnnfromtorch_intermediate_layer_getterimportIntermediateLayerGetterasMidGetterclassModel(nn.Module):def__init__(self):super().__init__()self.fc1=nn.Linear(2,2)self.fc2=nn.Linear(2,2)self.nested=nn.Sequential(nn.Sequential(nn.Linear(2,2),nn.Linear(2,3)),nn.Linear(3,1),)self.interaction_idty=nn.Identity()# Simple trick for operations not performed as modulesdefforward(self,x):x1=self.fc1(x)x2=self.fc2(x)interaction=x1*x2self.interaction_idty(interaction)x_out=self.nested(interaction)returnx_outmodel=Model()return_layers={'fc2':'fc2','nested.0.1':'nested','interaction_idty':'interaction',}mid_getter=MidGetter(model,return_layers=return_layers,keep_output=True)mid_outputs,model_output=mid_getter(torch.randn(1,2))print(model_output)>>tensor([[0.3219]],grad_fn=<AddmmBackward>)print(mid_outputs)>>OrderedDict([('fc2',tensor([[-1.5125,0.9334]],grad_fn=<AddmmBackward>)),('interaction',tensor([[-0.0687,-0.1462]],grad_fn=<MulBackward0>)),('nested',tensor([[-0.1697,0.1432,0.2959]],grad_fn=<AddmmBackward>))])# model_output is None if keep_ouput is False# if keep_output is True the model_output contains the final model's output
Package: torchist
NumPy-style histograms in PyTorchThetorchistpackage implements NumPy'shistogramandhistogramddfunctions in PyTorch with CUDA support. The package also features implementations ofravel_multi_index,unravel_indexand some useful functionals likeentropyorkl_divergence.InstallationThetorchistpackage is available onPyPI, which means it is installable withpip.pip install torchistAlternatively, if you need the latest features, you can install it from the repository.pip install git+https://github.com/francois-rozet/torchistGetting Startedimporttorchimporttorchistx=torch.rand(100,3).cuda()hist=torchist.histogramdd(x,bins=10,low=0.0,upp=1.0)print(hist.shape)# (10, 10, 10)BenchmarkThe implementations oftorchistare on par or faster than those ofnumpyon CPU and benefit greately from CUDA capabilities.$pythontorchist/__init__.pyCPU---np.histogram : 1.2559 snp.histogramdd : 20.7816 snp.histogram (non-uniform) : 5.4878 snp.histogramdd (non-uniform) : 17.3757 storchist.histogram : 1.3975 storchist.histogramdd : 9.6160 storchist.histogram (non-uniform) : 5.0883 storchist.histogramdd (non-uniform) : 17.2743 sCUDA----torchist.histogram : 0.1363 storchist.histogramdd : 0.3754 storchist.histogram (non-uniform) : 0.1355 storchist.histogramdd (non-uniform) : 0.5137 s
Package: torchiva
A package for blind source separation and beamforming inpytorch.supports many BSS and beamforming methodssupports memory efficient gradient computation for training neural source modelssupports batched computationscan run on GPU via pytorchQuick StartThe package can be installed via pip:pip install torchivaSeparation using Pre-trained ModelWe provide a pre-trained model intrained_models/tiss.
You can easily try separation with the pre-trained model:# Separation
python -m torchiva.separation INPUT OUTPUTwhereINPUTis either a multichannel wav file or a folder containing
multichannel wav files.  If a folder, then all the files inside are separted.
The output is saved toOUTPUT.
The model stored intrained_models/tissis automatically downloaded to$HOME/.torchiva_models. The path or url to the model can also be
manually provided via the--modeloption.
The model was trained on theWSJ1-mix datasetwith the same configuration
as./examples/configs/tiss.json.TrainingWe provide some simple training scripts.
We support training ofT-ISS,MWF,MVDR,GEV:cd examples

# install some modules necessary for training
pip install -r requirements.txt

# training
python train.py PATH_TO_CONFIG PATH_TO_DATASETNote that our example scripts assumes using WSJ1-mix dataset.
If you want to use other datasets, please change the script in the part that loads audios.Test your trained model with checkpoint from epoch 128:# python ./test.py --dataset ../wsj1_6ch --n_fft 2048 --hop 512 --n_iter 40 --iss-hparams checkpoints/tiss_delay1tap5_2ch/lightning_logs/version_0/hparams.yaml --epoch 128 --testExport the trained model for later use:python ./export_model.py ../trained_models/tiss checkpoints/tiss_delay1tap5_2ch/lightning_logs/version_0 128 146 148 138 122 116 112 108 104 97Run the example script using the exported model:python ./example_dnn.py ../wsj1_6ch ../trained_models/tiss -m 2 -r 100AuthorsRobin ScheiblerKohei SaijoLicense2022 (c) Robin Scheibler, Kohei Saijo, LINE Corporation.All of this code is released underMIT License
Package: torchjpeg
TorchJPEGThis package contains a C++ extension for pytorch that interfaces with libjpeg to allow for manipulation of low-level JPEG data.
By using libjpeg, quantization results are guaranteed to be consistent with other applications, like image viewers or MATLAB,
which use libjpeg to compress and decompress images. This is useful because JPEG images can be effected by round-off
errors or slight differences in the decompression procedure. Besides this, this library can be used to read and write
DCT coefficients, functionality which is not available from other python interfaces.Besides this, the library includes many utilities related to JPEG compression, many of which are written using native pytorch code meaning
they can be differentiated or GPU accelerated. The library currently includes packages related to the DCT, quantization, metrics, and dataset
transformations.LIBJPEGCurrently builds against:libjpeg-9d. libjpeg is statically linked during the build process. Seehttp://www.ijg.org/files/for libjpeg source.
The full libjpeg source is included with the torchjpeg source code and will be built during the install process (for a source or sdist install).InstallPackages are hosted onpypi. Install using pip, note that only Linux builds are supported at the moment.pip install torchjpegIf there is demand for builds on other platforms it may happen in the future. Also note that the wheel is intended to be compatible with manylinux2014
which means it should work on modern Linux systems, however, because of they way pytorch works, we can't actually build it using all of the manylinux2014
tools. So compliance is not guaranteed and YMMV.torchjpeg is currently in pre-beta development and consists mostly of converted research code. The public facing API, including any and all names of
parameters and functions, is subject to change at any time. We follow semver for versioning and will adhere to that before making and breaking
changes.CitationIf you use our code in a publication, we ask that you cite the following paper (bibtex):Max Ehrlich, Larry Davis, Ser-Nam Lim, and Abhinav Shrivastava. "Quantization Guided JPEG Artifact Correction." In Proceedings of the European Conference on Computer Vision, 2020
Package: torchKbert
torchKbertOur customized version of bert for pytorch说明这是笔者基于 Meelfy 的 pytorch_pretrained_BERT 库进行部分定制化修改的模型库。本项目的初衷是为了满足个人实验的方便，因此不会经常更新。功能原始的模型库pytorch_pretrained_BERT中的功能仍然支持；支持层次分解位置编码。支持基于词颗粒度的woBERT。
Pytorch 权重（这里提供的是WoBERT Plus模型）：chinese_wobert_plus.zip（提取码: fg6j)使用安装：pipinstalltorchKbert典型的使用例子请参考官方examples目录。若想使用层次分解位置编码，使 BERT 可以处理长文本，在model中传入参数is_hierarchical=True即可。示例如下：model = BertModel(config)
encoder_outputs, _ = model(input_ids, token_ids, input_mask, is_hierarchical=True)若想使用基于词颗粒度的中文WoBERT，只需在构建BertTokenizer对象时传入新参数：from torchKbert.tokenization import BertTokenizer

tokenizer = BertTokenizer(
    vocab_file=vocab_path, 
    pre_tokenizer=lambda s: jieba.cut(s, HMM=False))不传入时，默认为None。分词时默认以词为单位，若想恢复使用以字为单位，只需在tokenize时传入新参数pre_tokenize=False：tokenzier.tokenize(text, pre_tokenize=False)背景之前一直在用 Meelfy 编写的pytorch_pretrained_BERT，调用预训练模型或进行微调已经十分方便。后来因个人的需求，所以就想改写一个支持层次分解位置编码的版本。苏神的bert4keras已经实现了这样的功能。但因个人惯于使用 pytorch，已经很久不用 keras 了，所以才打算自己改写一个。更新2021.03.07: 添加层次分解位置编码。2021.05.27: 添加基于词颗粒度的中文WoBERT。2022.03.27: 参照pytorch_transformers对 BertPretrainedModel 代码实现进行了重构。参考感谢 Meelfy 实现的pytorch_pretrained_BERT，本实现完全基于 pytorch_pretrained_BERT 的源码。感谢苏神的 insight 和无私分享：层次分解位置编码，让BERT可以处理超长文本。WoBERT: Word-based Chinese BERT model - ZhuiyiAI。