botryan96 commited on
Commit
2d30b75
1 Parent(s): 2e92e68

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -20
README.md CHANGED
@@ -11,25 +11,19 @@ probably proofread and complete it, then remove this comment. -->
11
 
12
  # GeoBERT
13
 
14
- This model was trained from scratch on an unknown dataset.
15
- It achieves the following results on the evaluation set:
16
- - Train Loss: 0.0660
17
- - Validation Loss: 0.0470
18
- - Epoch: 0
19
 
20
- ## Model description
21
 
22
- More information needed
23
 
24
- ## Intended uses & limitations
25
 
26
- More information needed
 
 
 
27
 
28
- ## Training and evaluation data
29
-
30
- More information needed
31
-
32
- ## Training procedure
33
 
34
  ### Training hyperparameters
35
 
@@ -37,12 +31,6 @@ The following hyperparameters were used during training:
37
  - optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 14000, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
38
  - training_precision: mixed_float16
39
 
40
- ### Training results
41
-
42
- | Train Loss | Validation Loss | Epoch |
43
- |:----------:|:---------------:|:-----:|
44
- | 0.0660 | 0.0470 | 0 |
45
-
46
 
47
  ### Framework versions
48
 
@@ -50,3 +38,30 @@ The following hyperparameters were used during training:
50
  - TensorFlow 2.10.0
51
  - Datasets 2.4.0
52
  - Tokenizers 0.12.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
  # GeoBERT
13
 
14
+ GeoBERT is a NER model that was fine-tuned from SciBERT on the Geoscientific Corpus dataset.
15
+ The model was trained on the Labeled Geoscientific Corpus dataset (~1 million sentences).
 
 
 
16
 
 
17
 
18
+ ## Intended uses
19
 
20
+ The NER product in this model has a goal to identify four main semantic types or categories related to Geosciences.
21
 
22
+ 1. GeoPetro for any entities that belong to all terms in Geosciences
23
+ 2. GeoMeth for all tools or methods associated with Geosciences
24
+ 3. GeoLoc to identify geological locations
25
+ 4. GeoTime for identifying the geological time scale entities
26
 
 
 
 
 
 
27
 
28
  ### Training hyperparameters
29
 
 
31
  - optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 14000, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
32
  - training_precision: mixed_float16
33
 
 
 
 
 
 
 
34
 
35
  ### Framework versions
36
 
 
38
  - TensorFlow 2.10.0
39
  - Datasets 2.4.0
40
  - Tokenizers 0.12.1
41
+
42
+ ## Model performances (metric: seqeval)
43
+
44
+ Overall
45
+
46
+ precision|recall|f1
47
+ -|-|-
48
+ 0.8859|0.8971|0.8914
49
+
50
+ By entity
51
+
52
+ entity|precision|recall|f1
53
+ -|-|-|-
54
+ GeoLoc |0.9727|0.9591|0.9658
55
+ GeoMeth |0.9433|0.9447|0.9445
56
+ GeoPetro|0.9767|0.9745|0.9756
57
+ GeoTime |0.9695|0.9666|0.9680
58
+
59
+ ## How to use GeoBERT with HuggingFace
60
+
61
+ ##### Load GeoBERT and its sub-word tokenizer :
62
+
63
+ ```python
64
+ from transformers import AutoTokenizer, AutoModelForTokenClassification
65
+
66
+ tokenizer = AutoTokenizer.from_pretrained("botryan96/GeoBERT")
67
+ model = AutoModelForTokenClassification.from_pretrained("botryan96/GeoBERT")