Upload model files

README.md

@@ -1,3 +1,57 @@
 ---
+library_name: PyLaia
 license: mit
+tags:
+- PyLaia
+- PyTorch
+- Handwritten text recognition
+metrics:
+- CER
+- WER
+language:
+- 'no'
+datasets:
+- Teklia/NorHand_v2
 ---
+
+# NorHand v2 handwritten text recognition
+
+This model performs Handwritten Text Recognition in Norwegian on historical documents.
+
+## Model description
+
+The model was trained using the PyLaia library on the [NorHand v2 dataset](https://zenodo.org/records/10555698).
+
+For training, text-lines were resized with a fixed height of 128 pixels, keeping the original aspect ratio. Vertical lines are discarded.
+
+| split | N lines | N horizontal lines |
+| ----- | ------: | -----------------: |
+| train | 146,693 |            145,061 |
+| val   |  15,119 |             14,980 |
+| test  |   1,830 |              1,793 |
+
+An external 6-gram character language model can be used to improve recognition. The language model is trained on the text from the NorHand v3 training set.
+
+## Evaluation results
+
+The model achieves the following results:
+
+| set   | Language model | CER (%)    | WER (%) | N lines   |
+|:------|:---------------|:----------:|:-------:|----------:|
+| test  | no             | 3.82       |   12.00 |     1,573 |
+| test  | yes            | 3.13       |    9.29 |     1,573 |
+
+## How to use
+
+Please refer to the [documentation](https://atr.pages.teklia.com/pylaia/).
+
+## Cite us
+
+```bibtex
+@inproceedings{pylaia-lib,
+    author = "Tarride, Solène and Schneider, Yoann and Generali, Marie and Boillet, Melodie and Abadie, Bastien and Kermorvant, Christopher",
+    title = "Improving Automatic Text Recognition with Language Models in the PyLaia Open-Source Library",
+    booktitle = "Submitted at ICDAR2024",
+    year = "2024"
+}
+```

language_model.arpa.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec4a7ec5732090516c96968af3e3bfb5c797bfb81f5d07a03877618d4e5b701f
+size 43250739

lexicon.txt

@@ -0,0 +1,149 @@
+<space> <space>
+! !
+" "
+# #
+$ $
+% %
+& &
+' '
+( (
+) )
+* *
++ +
+, ,
+- -
+. .
+/ /
+0 0
+1 1
+2 2
+3 3
+4 4
+5 5
+6 6
+7 7
+8 8
+9 9
+: :
+; ;
+< <
+= =
+> >
+? ?
+A A
+B B
+C C
+D D
+E E
+F F
+G G
+H H
+I I
+J J
+K K
+L L
+M M
+N N
+O O
+P P
+Q Q
+R R
+S S
+T T
+U U
+V V
+W W
+X X
+Y Y
+Z Z
+[ [
+\ \
+] ]
+a a
+b b
+c c
+d d
+e e
+f f
+g g
+h h
+i i
+j j
+k k
+l l
+m m
+n n
+o o
+p p
+q q
+r r
+s s
+t t
+u u
+v v
+w w
+x x
+y y
+z z
+{ {
+| |
+} }
+£ £
+§ §
+« «
+¬ ¬
+° °
+´ ´
+º º
+» »
+¼ ¼
+½ ½
+Ä Ä
+Å Å
+Æ Æ
+É É
+Ö Ö
+Ø Ø
+Ü Ü
+Þ Þ
+ß ß
+à à
+á á
+â â
+ä ä
+å å
+æ æ
+ç ç
+è è
+é é
+ê ê
+ë ë
+í í
+ï ï
+ð ð
+ó ó
+ô ô
+ö ö
+÷ ÷
+ø ø
+ù ù
+ú ú
+û û
+ü ü
+ý ý
+þ þ
+œ œ
+˚ ˚
+Ֆ Ֆ
+– –
+— —
+‘ ‘
+’ ’
+“ “
+” ”
+„ „
+… …
+⁰ ⁰
+⅓ ⅓
+ 
+<ctc> <ctc>

syms.txt

@@ -0,0 +1,150 @@
+<ctc> 0
+! 1
+" 2
+# 3
+$ 4
+% 5
+& 6
+' 7
+( 8
+) 9
+* 10
++ 11
+, 12
+- 13
+. 14
+/ 15
+0 16
+1 17
+2 18
+3 19
+4 20
+5 21
+6 22
+7 23
+8 24
+9 25
+: 26
+; 27
+< 28
+= 29
+> 30
+? 31
+A 32
+B 33
+C 34
+D 35
+E 36
+F 37
+G 38
+H 39
+I 40
+J 41
+K 42
+L 43
+M 44
+N 45
+O 46
+P 47
+Q 48
+R 49
+S 50
+T 51
+U 52
+V 53
+W 54
+X 55
+Y 56
+Z 57
+[ 58
+\ 59
+] 60
+a 61
+b 62
+c 63
+d 64
+e 65
+f 66
+g 67
+h 68
+i 69
+j 70
+k 71
+l 72
+m 73
+n 74
+o 75
+p 76
+q 77
+r 78
+s 79
+t 80
+u 81
+v 82
+w 83
+x 84
+y 85
+z 86
+{ 87
+| 88
+} 89
+£ 90
+§ 91
+« 92
+¬ 93
+° 94
+´ 95
+º 96
+» 97
+¼ 98
+½ 99
+Ä 100
+Å 101
+Æ 102
+É 103
+Ö 104
+Ø 105
+Ü 106
+Þ 107
+ß 108
+à 109
+á 110
+â 111
+ä 112
+å 113
+æ 114
+ç 115
+è 116
+é 117
+ê 118
+ë 119
+í 120
+ï 121
+ð 122
+ó 123
+ô 124
+ö 125
+÷ 126
+ø 127
+ù 128
+ú 129
+û 130
+ü 131
+ý 132
+þ 133
+œ 134
+˚ 135
+Ֆ 136
+– 137
+— 138
+‘ 139
+’ 140
+“ 141
+” 142
+„ 143
+… 144
+⁰ 145
+⅓ 146
+ 147
+<unk> 148
+<space> 149

tokens.txt

@@ -0,0 +1,150 @@
+<ctc>
+!
+"
+#
+$
+%
+&
+'
+(
+)
+*
++
+,
+-
+.
+/
+0
+1
+2
+3
+4
+5
+6
+7
+8
+9
+:
+;
+<
+=
+>
+?
+A
+B
+C
+D
+E
+F
+G
+H
+I
+J
+K
+L
+M
+N
+O
+P
+Q
+R
+S
+T
+U
+V
+W
+X
+Y
+Z
+[
+\
+]
+a
+b
+c
+d
+e
+f
+g
+h
+i
+j
+k
+l
+m
+n
+o
+p
+q
+r
+s
+t
+u
+v
+w
+x
+y
+z
+{
+|
+}
+£
+§
+«
+¬
+°
+´
+º
+»
+¼
+½
+Ä
+Å
+Æ
+É
+Ö
+Ø
+Ü
+Þ
+ß
+à
+á
+â
+ä
+å
+æ
+ç
+è
+é
+ê
+ë
+í
+ï
+ð
+ó
+ô
+ö
+÷
+ø
+ù
+ú
+û
+ü
+ý
+þ
+œ
+˚
+Ֆ
+–
+—
+‘
+’
+“
+”
+„
+…
+⁰
+⅓
+
+<unk>
+<space>

weights.ckpt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdd12e585cd86144586c0bdcbebf06e3a59532a88de886c5cc4164f0344a71b1
+size 42954972