Package: fastseq
FastSeqIntroductionFastSeq provides efficient implementations of the popular sequence models with high performance for text generation, summarization, and translation tasks. It can automatically optimize the performance of the pupular NLP toolkits (e.g.FairSeq) by simplyimport fastseq.Supported ModelsSupported models infairseqProphetNetBARTScaling Neural Machine Translation (Ott et al., 2018)Mixture Models for Diverse Machine Translation: Tricks of the Trade (Shen et al., 2019)Pay Less Attention with Lightweight and Dynamic Convolutions (Wu et al., 2019)Supported models inHuggingFace-TransformersBARTT5GPT-2UniLM-V1UniLM-V2ProphetNetBenchmarksProphetNetCNN daily mail val data, NVIDIA-V100-16GBBatchSize3264128prophetnet2.7 samples/s3.1 samples/sOOMprophetnet + fastseq5.5 samples/s8.4 samples/s10.3 samples/swith setting:$fastseq-generate-for-fairseq\cnn_dm_bert.1k/len-1024.bin\--pathprophetnet/model.pt\--fp16\--tasktranslation_prophetnet\--batch-sizeBATCH_SIZE\--beam4\--num-workers4\--min-len55\--max-len-b140\--no-repeat-ngram-size3\--lenpen2.0\--remove-bpe\--gen-subsetvalid\BART from FairseqCNN daily mail val data, NVIDIA-V100-16GBBatchSize3264128fairseq-0.9.02.7 samples/sOOMOOMabove + fastseq9.0 samples/s12.5 samples/s14.5 samples/swith setting:$fastseq-generate-for-fairseq\cnn_dm.1k/len-1024.bin\--pathbart.large.cnn/model.pt\--fp16\--tasktranslation\--batch-sizeBATCH_SIZE\--gen-subsetvalid\--truncate-source\--bpegpt2\--beam4\--num-workers4\--min-len55\--max-len-b140\--no-repeat-ngram-size3\--lenpen2.0To get the baseline fairseq's speed number, replacefastseq-generate-for-fairseqbyfairseq-generate.BART from TransformersCNN daily mail val data, NVIDIA-V100-16GBBatchSize3264128transformers-3.0.23.4 samples/sOOMOOMabove + fastseq5.2 samples/s6.2 samples/s6.4 samples/stransformers-2.11.02.5 samples/sOOMOOMabove + fastseq4.4 samples/s5.3 samples/s>5.3 samples/s(numbers for 2.11.0 needs to be updated based on docker env.)with setting:$fastseq-generate-for-transformers\facebook/bart-large-cnn\cnn_dm.1k/val.source\out.summary\--reference_pathcnn_dm/val.target\--devicecuda\--bs128\--fp16\--score_pathout.score\--tasksummarizationTo get the baseline transformers' speed number, we can either add option--without_fastseq_optor usetoolprovided in Transformers GitHub repository.WMT from FairseqWMT16 En-DemodelBatchSize2565121024fairseq-0.9.084 samples/sOOMOOMabove + fastseq129 samples/s131 samples/s135 samples/swith setting:$fastseq-generate-for-fairseq\wmt14.en-fr.joined-dict.newstest2014/\--pathwmt14.en-fr.joined-dict.transformer/model.pt\--beam4\--lenpen0.6\--remove-bpe\--batch-size32To get the fairseq's speed number, replacefastseq-generate-for-fairseqbyfairseq-generate.InstallationRequirementsPython version >= 3.6torch>= 1.4.0fairseq>= 0.9.0transformers>= 3.0.2requets>= 2.24.0absl-py>= 0.9.0rouge-scoreIf you use fairseq or transformers, you only need to install one of them. If you use both, you need to install both.Python packagefastseqPython package can be directly installed with pip using$pipinstallfastseqInstall from the source$gitclonehttps://github.com/microsoft/fastseq
$cdfastseq
$pipinstall--editable./UsageExampleOnly one line of code change is needed to use the optimizations provided byFastSeq.# import fastseq at the beginning of your programimportfastseqimporttorch# Download bart.large.cnnbart=torch.hub.load('pytorch/fairseq','bart.large.cnn')bart.cuda()# use GPUbart.eval()# disable dropout for evaluationbart.half()slines=['FastSeq provides efficient implementations of the popular sequence models. Please visit https://github.com/microsoft/fastseq for more details.']hypotheses=bart.sample(slines,beam=4,lenpen=2.0,max_len_b=140,min_len=55,no_repeat_ngram_size=3)print(hypotheses)Command line tool for fairseq modelsExample$fastseq-generate-for-fairseq\cnn_dnn/bin\--pathbart.large.cnn/model.pt\--fp16\--tasktranslation\--batch-size128\--gen-subsetvalid\--truncate-source\--bpegpt2\--beam4\--num-workers4\--min-len55\--max-len-b140\--no-repeat-ngram-size3\--lenpen2.0Command line tool for transformers modelsExample$fastseq-generate-for-transformers\facebook/bart-large-cnn\cnn_dm/val.source\out.summary\--reference_pathcnn_dm/val.target\--devicecuda\--bs128\--fp16\--score_pathout.score\--tasksummarizationRun tests# run a single test.$pythontests/optimizer/fairseq/test_fairseq_optimizer.py# run benchmark.$pythontests/optimizer/fairseq/benchmark_fairseq_optimizer.py# run all the tests.$python-munittestdiscover-stests/-p'*.py'# run all the benchmarks.$cdbenchmarks&&bashrun_all_benchmarks.shBuild# build package$pythonsetup.pysdistbdist_wheelCode StylePython coding styleChanges to Python code should conform toPEP 8.yapfcan be used to help format the python code, and usepylintto check your Python changes.# format the code by yapf$yapf--stylepep8-i-rPYTHON_FILE/PACKAGE# run pylint check$pylint--rcfile=.pylintrcPYTHON_FILE/PACKAGEContributingThis project welcomes contributions and suggestions.  Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visithttps://cla.opensource.microsoft.com.When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct.
For more information see theCode of Conduct FAQor
contactopencode@microsoft.comwith any additional questions or comments.