Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
Spaces:
huggingface
/
text-data-filtering
like
33
Runtime error
App
Files
Files
Community
4
b37a555
text-data-filtering
6 contributors
History:
57 commits
HugoLaurencon
HF staff
new kenlm models
b37a555
over 2 years ago
.gitattributes
1.39 kB
chinese visu
over 2 years ago
.gitignore
25 Bytes
new tool to analyse our own doc
over 2 years ago
LICENSE
11.4 kB
Create LICENSE
over 2 years ago
README.md
909 Bytes
initial commit
over 2 years ago
app.py
42.4 kB
add register information
over 2 years ago
en.arpa.bin
4.44 GB
LFS
new kenlm models
over 2 years ago
en.sp.model
931 kB
LFS
new kenlm models
over 2 years ago
en_examples_with_stats.json
276 MB
LFS
new kenlm models
over 2 years ago
explanation_filtering_pipeline.pdf
217 kB
new filter on word repetition ratio
over 2 years ago
filtering.py
30.6 kB
rename badwords to flagged words + new flagged words list of 68 words
over 2 years ago
flagged_words.py
51.8 kB
rename badwords to flagged words + new flagged words list of 68 words
over 2 years ago
languages_id.py
5.61 kB
rename badwords to flagged words + new flagged words list of 68 words
over 2 years ago
lid.176.bin
131 MB
LFS
test
over 2 years ago
normalization.py
941 Bytes
test
over 2 years ago
parameters_filtering.py
31.7 kB
rename badwords to flagged words + new flagged words list of 68 words
over 2 years ago
requirements.txt
76 Bytes
fix requirements
over 2 years ago
stopwords.py
99.2 kB
test
over 2 years ago
zh.arpa.bin
3.64 GB
LFS
new kenlm models
over 2 years ago
zh.sp.model
876 kB
LFS
new kenlm models
over 2 years ago
zh_examples_with_stats.json
74.9 MB
LFS
new kenlm models
over 2 years ago
