Commit
•
2b89df8
1
Parent(s):
71966ed
Upload model files
Browse files- README.md +55 -0
- language_model.arpa.gz +3 -0
- lexicon.txt +117 -0
- model +0 -0
- syms.txt +117 -0
- tokens.txt +117 -0
- weights.ckpt +3 -0
README.md
CHANGED
@@ -1,3 +1,58 @@
|
|
1 |
---
|
|
|
2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
library_name: PyLaia
|
3 |
license: mit
|
4 |
+
tags:
|
5 |
+
- PyLaia
|
6 |
+
- PyTorch
|
7 |
+
- Handwritten text recognition
|
8 |
+
metrics:
|
9 |
+
- CER
|
10 |
+
- WER
|
11 |
+
language:
|
12 |
+
- 'fr'
|
13 |
+
datasets:
|
14 |
+
- Teklia/Belfort
|
15 |
---
|
16 |
+
|
17 |
+
# Belfort handwritten text recognition
|
18 |
+
|
19 |
+
This model performs Handwritten Text Recognition in French on historical documents.
|
20 |
+
|
21 |
+
## Model description
|
22 |
+
|
23 |
+
The model was trained using the PyLaia library on the [Belfort dataset](https://zenodo.org/records/8041668).
|
24 |
+
|
25 |
+
For training, text-lines were resized with a fixed height of 128 pixels, keeping the original aspect ratio. Vertical lines are discarded.
|
26 |
+
|
27 |
+
| split | N lines |
|
28 |
+
| ----- | ------: |
|
29 |
+
| train | 25,800 |
|
30 |
+
| val | 3,102 |
|
31 |
+
| test | 3,819 |
|
32 |
+
|
33 |
+
An external 6-gram character language model can be used to improve recognition. The language model is trained on the text from the NorHand v2 training set.
|
34 |
+
|
35 |
+
## Evaluation results
|
36 |
+
|
37 |
+
The model achieves the following results:
|
38 |
+
|
39 |
+
| set | Language model | CER (%) | WER (%) | N lines |
|
40 |
+
|:------|:---------------|:----------:|:-------:|----------:|
|
41 |
+
| test | no | 10.54 | 28.12 | 3,819 |
|
42 |
+
| test | yes | 9.52 | 23.73 | 3,819 |
|
43 |
+
|
44 |
+
## How to use
|
45 |
+
|
46 |
+
Please refer to the [documentation](https://atr.pages.teklia.com/pylaia/).
|
47 |
+
|
48 |
+
## Cite us
|
49 |
+
|
50 |
+
```bibtex
|
51 |
+
@inproceedings{pylaia-lib,
|
52 |
+
author = "Tarride, Solène and Schneider, Yoann and Generali, Marie and Boillet, Melodie and Abadie, Bastien and Kermorvant, Christopher",
|
53 |
+
title = "Improving Automatic Text Recognition with Language Models in the PyLaia Open-Source Library",
|
54 |
+
booktitle = "Submitted at ICDAR2024",
|
55 |
+
year = "2024"
|
56 |
+
}
|
57 |
+
```
|
58 |
+
|
language_model.arpa.gz
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:cbb30d655c189a8f13e3a87c47252a65198493ee6d0859dcd289f97cee4ee6fa
|
3 |
+
size 11272705
|
lexicon.txt
ADDED
@@ -0,0 +1,117 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
<ctc> <ctc>
|
2 |
+
! !
|
3 |
+
" "
|
4 |
+
% %
|
5 |
+
& &
|
6 |
+
' '
|
7 |
+
( (
|
8 |
+
) )
|
9 |
+
* *
|
10 |
+
+ +
|
11 |
+
, ,
|
12 |
+
- -
|
13 |
+
. .
|
14 |
+
/ /
|
15 |
+
0 0
|
16 |
+
1 1
|
17 |
+
2 2
|
18 |
+
3 3
|
19 |
+
4 4
|
20 |
+
5 5
|
21 |
+
6 6
|
22 |
+
7 7
|
23 |
+
8 8
|
24 |
+
9 9
|
25 |
+
: :
|
26 |
+
; ;
|
27 |
+
< <
|
28 |
+
= =
|
29 |
+
> >
|
30 |
+
? ?
|
31 |
+
A A
|
32 |
+
B B
|
33 |
+
C C
|
34 |
+
D D
|
35 |
+
E E
|
36 |
+
F F
|
37 |
+
G G
|
38 |
+
H H
|
39 |
+
I I
|
40 |
+
J J
|
41 |
+
K K
|
42 |
+
L L
|
43 |
+
M M
|
44 |
+
N N
|
45 |
+
O O
|
46 |
+
P P
|
47 |
+
Q Q
|
48 |
+
R R
|
49 |
+
S S
|
50 |
+
T T
|
51 |
+
U U
|
52 |
+
V V
|
53 |
+
W W
|
54 |
+
X X
|
55 |
+
Y Y
|
56 |
+
Z Z
|
57 |
+
[ [
|
58 |
+
] ]
|
59 |
+
_ _
|
60 |
+
a a
|
61 |
+
b b
|
62 |
+
c c
|
63 |
+
d d
|
64 |
+
e e
|
65 |
+
f f
|
66 |
+
g g
|
67 |
+
h h
|
68 |
+
i i
|
69 |
+
j j
|
70 |
+
k k
|
71 |
+
l l
|
72 |
+
m m
|
73 |
+
n n
|
74 |
+
o o
|
75 |
+
p p
|
76 |
+
q q
|
77 |
+
r r
|
78 |
+
s s
|
79 |
+
t t
|
80 |
+
u u
|
81 |
+
v v
|
82 |
+
w w
|
83 |
+
x x
|
84 |
+
y y
|
85 |
+
z z
|
86 |
+
{ {
|
87 |
+
| |
|
88 |
+
} }
|
89 |
+
§ §
|
90 |
+
« «
|
91 |
+
° °
|
92 |
+
² ²
|
93 |
+
º º
|
94 |
+
» »
|
95 |
+
À À
|
96 |
+
É É
|
97 |
+
Ü Ü
|
98 |
+
à à
|
99 |
+
â â
|
100 |
+
ç ç
|
101 |
+
è è
|
102 |
+
é é
|
103 |
+
ê ê
|
104 |
+
ë ë
|
105 |
+
î î
|
106 |
+
ï ï
|
107 |
+
ô ô
|
108 |
+
ù ù
|
109 |
+
û û
|
110 |
+
ü ü
|
111 |
+
ÿ ÿ
|
112 |
+
Œ Œ
|
113 |
+
œ œ
|
114 |
+
’ ’
|
115 |
+
… …
|
116 |
+
<unk> <unk>
|
117 |
+
<space> <space>
|
model
ADDED
Binary file (1.52 kB). View file
|
|
syms.txt
ADDED
@@ -0,0 +1,117 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
<ctc> 0
|
2 |
+
! 1
|
3 |
+
" 2
|
4 |
+
% 3
|
5 |
+
& 4
|
6 |
+
' 5
|
7 |
+
( 6
|
8 |
+
) 7
|
9 |
+
* 8
|
10 |
+
+ 9
|
11 |
+
, 10
|
12 |
+
- 11
|
13 |
+
. 12
|
14 |
+
/ 13
|
15 |
+
0 14
|
16 |
+
1 15
|
17 |
+
2 16
|
18 |
+
3 17
|
19 |
+
4 18
|
20 |
+
5 19
|
21 |
+
6 20
|
22 |
+
7 21
|
23 |
+
8 22
|
24 |
+
9 23
|
25 |
+
: 24
|
26 |
+
; 25
|
27 |
+
< 26
|
28 |
+
= 27
|
29 |
+
> 28
|
30 |
+
? 29
|
31 |
+
A 30
|
32 |
+
B 31
|
33 |
+
C 32
|
34 |
+
D 33
|
35 |
+
E 34
|
36 |
+
F 35
|
37 |
+
G 36
|
38 |
+
H 37
|
39 |
+
I 38
|
40 |
+
J 39
|
41 |
+
K 40
|
42 |
+
L 41
|
43 |
+
M 42
|
44 |
+
N 43
|
45 |
+
O 44
|
46 |
+
P 45
|
47 |
+
Q 46
|
48 |
+
R 47
|
49 |
+
S 48
|
50 |
+
T 49
|
51 |
+
U 50
|
52 |
+
V 51
|
53 |
+
W 52
|
54 |
+
X 53
|
55 |
+
Y 54
|
56 |
+
Z 55
|
57 |
+
[ 56
|
58 |
+
] 57
|
59 |
+
_ 58
|
60 |
+
a 59
|
61 |
+
b 60
|
62 |
+
c 61
|
63 |
+
d 62
|
64 |
+
e 63
|
65 |
+
f 64
|
66 |
+
g 65
|
67 |
+
h 66
|
68 |
+
i 67
|
69 |
+
j 68
|
70 |
+
k 69
|
71 |
+
l 70
|
72 |
+
m 71
|
73 |
+
n 72
|
74 |
+
o 73
|
75 |
+
p 74
|
76 |
+
q 75
|
77 |
+
r 76
|
78 |
+
s 77
|
79 |
+
t 78
|
80 |
+
u 79
|
81 |
+
v 80
|
82 |
+
w 81
|
83 |
+
x 82
|
84 |
+
y 83
|
85 |
+
z 84
|
86 |
+
{ 85
|
87 |
+
| 86
|
88 |
+
} 87
|
89 |
+
§ 88
|
90 |
+
« 89
|
91 |
+
° 90
|
92 |
+
² 91
|
93 |
+
º 92
|
94 |
+
» 93
|
95 |
+
À 94
|
96 |
+
É 95
|
97 |
+
Ü 96
|
98 |
+
à 97
|
99 |
+
â 98
|
100 |
+
ç 99
|
101 |
+
è 100
|
102 |
+
é 101
|
103 |
+
ê 102
|
104 |
+
ë 103
|
105 |
+
î 104
|
106 |
+
ï 105
|
107 |
+
ô 106
|
108 |
+
ù 107
|
109 |
+
û 108
|
110 |
+
ü 109
|
111 |
+
ÿ 110
|
112 |
+
Œ 111
|
113 |
+
œ 112
|
114 |
+
’ 113
|
115 |
+
… 114
|
116 |
+
<unk> 115
|
117 |
+
<space> 116
|
tokens.txt
ADDED
@@ -0,0 +1,117 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
<ctc>
|
2 |
+
!
|
3 |
+
"
|
4 |
+
%
|
5 |
+
&
|
6 |
+
'
|
7 |
+
(
|
8 |
+
)
|
9 |
+
*
|
10 |
+
+
|
11 |
+
,
|
12 |
+
-
|
13 |
+
.
|
14 |
+
/
|
15 |
+
0
|
16 |
+
1
|
17 |
+
2
|
18 |
+
3
|
19 |
+
4
|
20 |
+
5
|
21 |
+
6
|
22 |
+
7
|
23 |
+
8
|
24 |
+
9
|
25 |
+
:
|
26 |
+
;
|
27 |
+
<
|
28 |
+
=
|
29 |
+
>
|
30 |
+
?
|
31 |
+
A
|
32 |
+
B
|
33 |
+
C
|
34 |
+
D
|
35 |
+
E
|
36 |
+
F
|
37 |
+
G
|
38 |
+
H
|
39 |
+
I
|
40 |
+
J
|
41 |
+
K
|
42 |
+
L
|
43 |
+
M
|
44 |
+
N
|
45 |
+
O
|
46 |
+
P
|
47 |
+
Q
|
48 |
+
R
|
49 |
+
S
|
50 |
+
T
|
51 |
+
U
|
52 |
+
V
|
53 |
+
W
|
54 |
+
X
|
55 |
+
Y
|
56 |
+
Z
|
57 |
+
[
|
58 |
+
]
|
59 |
+
_
|
60 |
+
a
|
61 |
+
b
|
62 |
+
c
|
63 |
+
d
|
64 |
+
e
|
65 |
+
f
|
66 |
+
g
|
67 |
+
h
|
68 |
+
i
|
69 |
+
j
|
70 |
+
k
|
71 |
+
l
|
72 |
+
m
|
73 |
+
n
|
74 |
+
o
|
75 |
+
p
|
76 |
+
q
|
77 |
+
r
|
78 |
+
s
|
79 |
+
t
|
80 |
+
u
|
81 |
+
v
|
82 |
+
w
|
83 |
+
x
|
84 |
+
y
|
85 |
+
z
|
86 |
+
{
|
87 |
+
|
|
88 |
+
}
|
89 |
+
§
|
90 |
+
«
|
91 |
+
°
|
92 |
+
²
|
93 |
+
º
|
94 |
+
»
|
95 |
+
À
|
96 |
+
É
|
97 |
+
Ü
|
98 |
+
à
|
99 |
+
â
|
100 |
+
ç
|
101 |
+
è
|
102 |
+
é
|
103 |
+
ê
|
104 |
+
ë
|
105 |
+
î
|
106 |
+
ï
|
107 |
+
ô
|
108 |
+
ù
|
109 |
+
û
|
110 |
+
ü
|
111 |
+
ÿ
|
112 |
+
Œ
|
113 |
+
œ
|
114 |
+
’
|
115 |
+
…
|
116 |
+
<unk>
|
117 |
+
<space>
|
weights.ckpt
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:e73ff66f52effd625d4063b5549755bfa0b69ed9fc28edf6c6d30af111f723cd
|
3 |
+
size 42819548
|