spacy/en_core_web_sm
Token Classification
•
Updated
Tasks
Fill-Mask
Question Answering
Summarization
Table Question Answering
Text Classification
Text Generation
Text2Text Generation
Token Classification
Translation
Zero-Shot Classification
Sentence Similarity
Conversational
Feature Extraction
Text-to-Speech
Automatic Speech Recognition
Audio Source Separation
Voice Activity Detection
Image Classification
Object Detection
Image Segmentation
Libraries
Datasets
wikiann
common_voice
wikipedia
indic_glue
dcep europarl jrc-acquis
squad
bookcorpus
c4
CLUECorpusSmall
parsinlu
squad_v2
oscar
librispeech_asr
cnn_dailymail
conll2003
imagenet
xsum
jrc-acquis
natural_questions
PropBank.Br
multi_nli
imdb
OSIAN
1.5B Arabic Corpus
gigaword
ontonotes
imagenet-21k
mnli
OSCAR Arabic Unshuffled
mc4
brWaC
blended_skill_talk
openslr
wikisql
mustc
CoNLL-2012
twitter
openwebtext
xnli
sep_clean
xtreme
wmt19
coco
covost2
samsum
Indo4B
arabic_billion_words
open_subtitles
lince
qqp
enh_single
cc100
msr_sqa
emotion
code_search_net
OPUS
snli
the Pile
librispeech
sep_noisy
race
ai-soco
OpenLegalData
tab_fact
wtq
glue
common_crawl
Libri1Mix
ag_news
arabic_speech_corpus
trivia_qa
ljspeech
mlsum
fever
ms_marco
piaf
Libri2Mix
Libri3Mix
NST Swedish ASR Database
openbookqa
DAGW
opus100
scientific_papers
PubMed
libri-light
flaubert
OpenSLR
web_questions
voxceleb
dihard
quoref
docred
gap
winograd_wsc
winogender
squad2
biomedical literature from Scielo and Pubmed
Farasa
SAIL 2017
stsb
commonvoice
xsum_nl
yelp_polarity
ComVE
wmt16
id_liputan6
FQuAD
SQuAD-FR
wiki_dpr
WebNLG
Dart
common_gen
interspeech_2021_asr
array of dataset identifiers
wham
jsut
Universal Dependencies
MLSUM
aesdd
pn_summary
Wikipedia
tydiqa
DRCD
LJSpeech
commonsenseqa
arc
pib
cola
ax
mrpc
qnli
rte
sst2
wnli
reddit singapore, malaysia
hardwarezone
sst-2
hatexplain
klue
fquad
ccnews-titles-es
mlsum - es
MS MARCO document ranking
dindebat.dk
hestenettet.dk
danish OpenSubtitles
tweets_hate_speech_detection
CSS10
NSC2018
BFD
mlqa
sts
STSbenchmark
emo
AI4Bharat IndicNLP Corpora
scancode-rules
trec
Twitter
IndianPolitics
MIMIC-III
pubmed
id_newspapers_2018
Indonesian Wikipedia
parlament_parla
germeval_14
Wikihow
timit_asr
Squad
XQuad
Tydiqa
conll2000
wmt14
cc_news
wikipedia-turkish
MNLI
go_emotions
wer
LibriMix
muchocine
break_data
fairface
TQUAD
indosum
wikiann-conll2003
arcd
mulit_nli
aishell
Voicebank
DEMAND
WHAMR!
Baker
KSS
Thorsten
BALANCED_TRAIN_SNORKEL_V2
anli
NQ
Trivia
SQuAD
MLQA
cnn_dm_nl
sick
arxiv_dataset
common_voice, infore_25h
Oscar Corpus, News, Stories
masakhaner
augmented_codesearchnet
pytorrent
DKHate - OffensEval2020
Wsj0MixVar
ai2_arc
cc-news-es
XSUM
Gigaword
HARD-Arabic-Dataset
opusbook
JW300
iapp-wiki-qa-dataset
XQuAD
Uniref100
UniRef50
legal entity recognition
VoxLingua107
cnndm
Finnish parliament session 2
100GB Chinese corpus
CC100
covid_qa_deepset
kazakh_speech_corpus
Arabic poetry from several eras
Arabic Wikipedia
custom danish dataset
SciDocs
ALFFA,Gamayun & IWSLT
BC5CDR-chemicals
BC4CHEMD
BC5CDR-diseases
ncbi_disease
JNLPBA
BC2GM
msmarco
tweets
nadi
common_voice mn
RuTweetCorp
RuReviews
RuSentiment
COCA
fon_dataset
Tesserae
Phi5
Thomas Aquinas
BembaSpeech
algebra_linear_1d
algebra_linear_1d_composed
measurement_time
numbers_gcd
triviaqa
kowiki
news
OpenSLR 77
Arabic Poetry Dataset (6th - 21st century)
cnn_dailymail, orange_sum, pn_summary, samsum, xsum, wiki_summary
DaNE
legal
sail
big_patent
discofuse
custom-book-corpus
movies
DAMP-VSEP
Indic TTS Malayalam Speech Corpus
Openslr Malayalam Speech Corpus
SMC Malayalam Speech Corpus
IIIT-H Indic Speech Databases
220M words (IndoWiki, IndoWC, News)
google
cord19
yahoo-answers
larazonpublico
es
tatoeba
setimes
csmsc
AI4D Baamtu Datamation - Automatic Speech Recognition in WOLOF
coqa
L3CubeMahaSent
allocine
SUC 3.0
sqa
malromur
shemo
wiki-sh
wiki-bs
wiki-gr
wiki-hr
wiki-mk
time-mk-news-2010-2015
wiki-sl
wiki-sr
marefa-mt
Marefa-NER
CORD-19
test xxx
another cool dataset
libri1mix
zaion-april-annotated-dataset
squad_v1_pt
sms_spam
codexglue
bible_para
event2Mind
qasc
quarel
quartz
quora
CC-aligned
quotes-500K
cifar10
Icelandic portion of the OSCAR corpus from INRIA
IndicGlue
Oscar
drop
vivos
LibriTTS
mgb5
bioASQ
Spotify Podcasts Dataset
oscar (NL)
dbrd
lassy-ud
europarl-mono
conll2002
TACDataset
hellaswag
ami
voxconverse
socian
bangla-sentiment-benchmark
google_wellformed_query
xquad
CommonCrawl
openslr_hindi
urdu-text-news
indic tts
iiith
google speech commands
WHAM!
WSJ0-2Mix
WSJ0-3Mix
Fluent Speech Commands
Timers and Such
Urbansound8k
Wikipedia (Hindi, Sanskrit, Gujarati)
baker
kss
cornell_movie_dialogue
squad-v1
wikimovies
Interspeech 2021
imagenet_21k
laborotv
squad_v1
NST Estonian ASR Database
billsum
webqa
dureader
eli5
multi_nli_mismatch
Languages
en
bn
es
fr
de
sv
fi
zh
ar
ru
ja
tr
it
pt
fa
id
nl
uk
da
eo
pl
he
el
ro
bg
hi
lt
ko
cs
ca
et
no
vi
hu
af
is
sl
ms
hr
mr
ht
tl
rw
mk
mt
ig
lg
yo
lv
gl
sg
gu
ur
eu
sn
ee
ts
ln
ny
xh
or
jv
as
si
ha
rn
nb
mn
fy
ta
th
be
ga
ty
to
sm
am
fj
te
nn
lb
yi
sq
tn
cy
ka
br
fo
co
wa
mh
sk
st
ti
zu
kg
hy
an
rm
oc
tw
bi
gv
ml
lu
km
so
gd
kn
se
ps
os
wo
kw
ho
mg
tt
sw
az
sc
cv
ve
ne
tk
mi
ba
sh
dv
sr
su
ch
lo
na
my
ky
bs
sa
io
ab
ce
kl
bo
pa
kk
kj
gn
nv
ng
om
ss
ia
la
py
vn
Go
nr
al
gr
ks
sd
ku
tg
ug
uz
Licenses
apache-2.0
mit
cc-by-nc-4.0
cc-by-sa-3.0
cc-by-4.0
gpl-3.0
cc-by-sa-4.0
apache 2.0
cc by-nc-sa 4.0
gnu-gpl-3.0
cc-by-nc-sa-3.0
cc-by 4.0
public domain notice
lgpl-lr
any valid license identifier
apache license 2.0
apache
apache-2
cc-by-nc-sa 4.0
cc0
mit license
gnu
apache-40
gnu gplv3
attribution-sharealike 4.0 international