Model save

README.md

@@ -6,13 +6,6 @@ tags:
 model-index:
 - name: deberta-v3-base_finetuned_ai4privacy_v2
   results: []
-datasets:
-- ai4privacy/pii-masking-200k
-language:
-- en
-metrics:
-- seqeval
-pipeline_tag: token-classification
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -20,20 +13,73 @@ should probably proofread and complete it, then remove this comment. -->
 
 # deberta-v3-base_finetuned_ai4privacy_v2
 
-This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on the [ai4privacy/pii-masking-200k](https://huggingface.co/ai4privacy/pii-masking-200k) dataset.
-
-## Useage
-GitHub Implementation: [Ai4Privacy](https://github.com/Sripaad/ai4privacy)
+This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0693
+- Overall Precision: 0.9664
+- Overall Recall: 0.9732
+- Overall F1: 0.9698
+- Overall Accuracy: 0.9728
+- Accountname F1: 1.0
+- Accountnumber F1: 1.0
+- Age F1: 0.9760
+- Amount F1: 0.9897
+- Bic F1: 0.9978
+- Bitcoinaddress F1: 0.9907
+- Buildingnumber F1: 0.9906
+- City F1: 0.9930
+- Companyname F1: 0.9994
+- County F1: 0.9939
+- Creditcardcvv F1: 1.0
+- Creditcardissuer F1: 0.9891
+- Creditcardnumber F1: 0.9590
+- Currency F1: 0.9052
+- Currencycode F1: 0.9875
+- Currencyname F1: 0.7022
+- Currencysymbol F1: 0.9892
+- Date F1: 0.9126
+- Dob F1: 0.7438
+- Email F1: 1.0
+- Ethereumaddress F1: 1.0
+- Eyecolor F1: 1.0
+- Firstname F1: 0.9934
+- Gender F1: 0.9991
+- Height F1: 1.0
+- Iban F1: 1.0
+- Ip F1: 0.1551
+- Ipv4 F1: 0.8393
+- Ipv6 F1: 0.8034
+- Jobarea F1: 0.9942
+- Jobtitle F1: 0.9993
+- Jobtype F1: 0.9928
+- Lastname F1: 0.9877
+- Litecoinaddress F1: 0.9770
+- Mac F1: 1.0
+- Maskednumber F1: 0.9451
+- Middlename F1: 0.9773
+- Nearbygpscoordinate F1: 1.0
+- Ordinaldirection F1: 0.9924
+- Password F1: 1.0
+- Phoneimei F1: 1.0
+- Phonenumber F1: 1.0
+- Pin F1: 0.9929
+- Prefix F1: 0.9722
+- Secondaryaddress F1: 0.9974
+- Sex F1: 0.9949
+- Ssn F1: 0.9970
+- State F1: 0.9941
+- Street F1: 0.9972
+- Time F1: 0.9967
+- Url F1: 1.0
+- Useragent F1: 1.0
+- Username F1: 0.9991
+- Vehiclevin F1: 1.0
+- Vehiclevrm F1: 1.0
+- Zipcode F1: 0.9890
 
 ## Model description
 
-This model has been finetuned on the World's largest open source privacy dataset.
-
-The purpose of the trained models is to remove personally identifiable information (PII) from text, especially in the context of AI assistants and LLMs.
-
-The example texts have 54 PII classes (types of sensitive data), targeting 229 discussion subjects / use cases split across business, education, psychology and legal fields, and 5 interactions styles (e.g. casual conversation, formal document, emails etc...).
-
-Take a look at the Github implementation for specific reasearch.
+More information needed
 
 ## Intended uses & limitations
 
@@ -43,104 +89,36 @@ More information needed
 
 More information needed
 
-## Training hyperparameters
+## Training procedure
+
+### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 6e-04
-- train_batch_size: 16
-- eval_batch_size: 16
+- learning_rate: 5e-05
+- train_batch_size: 4
+- eval_batch_size: 4
 - seed: 42
-- optimizer: Adam with betas=(0.96,0.996) and epsilon=1e-08
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine_with_restarts
 - lr_scheduler_warmup_ratio: 0.2
-- num_epochs: 10
-- mixed_precision_training: Native AMP
-
-## Class wise metrics
-It achieves the following results on the evaluation set:
-
-- Loss: 0.0211
-- Overall Precision: 0.9722
-- Overall Recall: 0.9792
-- Overall F1: 0.9757
-- Overall Accuracy: 0.9915
-
-- Accountname F1: 0.9993
-- Accountnumber F1: 0.9986
-- Age F1: 0.9884
-- Amount F1: 0.9984
-- Bic F1: 0.9942
-- Bitcoinaddress F1: 0.9974
-- Buildingnumber F1: 0.9898
-- City F1: 1.0
-- Companyname F1: 1.0
-- County F1: 0.9976
-- Creditcardcvv F1: 0.9541
-- Creditcardissuer F1: 0.9970
-- Creditcardnumber F1: 0.9754
-- Currency F1: 0.8966
-- Currencycode F1: 0.9946
-- Currencyname F1: 0.7697
-- Currencysymbol F1: 0.9958
-- Date F1: 0.9778
-- Dob F1: 0.9546
-- Email F1: 1.0
-- Ethereumaddress F1: 1.0
-- Eyecolor F1: 0.9925
-- Firstname F1: 0.9947
-- Gender F1: 1.0
-- Height F1: 1.0
-- Iban F1: 0.9978
-- Ip F1: 0.5404
-- Ipv4 F1: 0.8455
-- Ipv6 F1: 0.8855
-- Jobarea F1: 0.9091
-- Jobtitle F1: 1.0
-- Jobtype F1: 0.9672
-- Lastname F1: 0.9855
-- Litecoinaddress F1: 0.9949
-- Mac F1: 0.9965
-- Maskednumber F1: 0.9836
-- Middlename F1: 0.7385
-- Nearbygpscoordinate F1: 1.0
-- Ordinaldirection F1: 1.0
-- Password F1: 1.0
-- Phoneimei F1: 0.9978
-- Phonenumber F1: 0.9975
-- Pin F1: 0.9820
-- Prefix F1: 0.9872
-- Secondaryaddress F1: 1.0
-- Sex F1: 0.9916
-- Ssn F1: 0.9960
-- State F1: 0.9967
-- Street F1: 0.9991
-- Time F1: 1.0
-- Url F1: 1.0
-- Useragent F1: 0.9981
-- Username F1: 1.0
-- Vehiclevin F1: 0.9950
-- Vehiclevrm F1: 0.9870
-- Zipcode F1: 0.9966
+- num_epochs: 7
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | Accountname F1 | Accountnumber F1 | Age F1 | Amount F1 | Bic F1 | Bitcoinaddress F1 | Buildingnumber F1 | City F1 | Companyname F1 | County F1 | Creditcardcvv F1 | Creditcardissuer F1 | Creditcardnumber F1 | Currency F1 | Currencycode F1 | Currencyname F1 | Currencysymbol F1 | Date F1 | Dob F1 | Email F1 | Ethereumaddress F1 | Eyecolor F1 | Firstname F1 | Gender F1 | Height F1 | Iban F1 | Ip F1  | Ipv4 F1 | Ipv6 F1 | Jobarea F1 | Jobtitle F1 | Jobtype F1 | Lastname F1 | Litecoinaddress F1 | Mac F1 | Maskednumber F1 | Middlename F1 | Nearbygpscoordinate F1 | Ordinaldirection F1 | Password F1 | Phoneimei F1 | Phonenumber F1 | Pin F1 | Prefix F1 | Secondaryaddress F1 | Sex F1 | Ssn F1 | State F1 | Street F1 | Time F1 | Url F1 | Useragent F1 | Username F1 | Vehiclevin F1 | Vehiclevrm F1 | Zipcode F1 |
-|:-------------:|:-----:|:----:|:---------------:|:-----------------:|:--------------:|:----------:|:----------------:|:--------------:|:----------------:|:------:|:---------:|:------:|:-----------------:|:-----------------:|:-------:|:--------------:|:---------:|:----------------:|:-------------------:|:-------------------:|:-----------:|:---------------:|:---------------:|:-----------------:|:-------:|:------:|:--------:|:------------------:|:-----------:|:------------:|:---------:|:---------:|:-------:|:------:|:-------:|:-------:|:----------:|:-----------:|:----------:|:-----------:|:------------------:|:------:|:---------------:|:-------------:|:----------------------:|:-------------------:|:-----------:|:------------:|:--------------:|:------:|:---------:|:-------------------:|:------:|:------:|:--------:|:---------:|:-------:|:------:|:------------:|:-----------:|:-------------:|:-------------:|:----------:|
-| 1.5081        | 1.0   | 669  | 0.1618          | 0.7172            | 0.7690         | 0.7422     | 0.9464           | 0.9134         | 0.7806           | 0.7878 | 0.3447    | 0.6647 | 0.8155            | 0.7413            | 0.6679  | 0.8354         | 0.7719    | 0.0              | 0.0                 | 0.3622              | 0.4985      | 0.0             | 0.0             | 0.0166            | 0.8014  | 0.3108 | 0.9665   | 0.9878             | 0.0         | 0.7814       | 0.7444    | 0.9244    | 0.9059  | 0.0    | 0.8207  | 0.7064  | 0.0        | 0.8649      | 0.0        | 0.2521      | 0.0                | 0.9755 | 0.5147          | 0.0           | 0.9918                 | 0.0                 | 0.9443      | 0.9839       | 0.9700         | 0.1919 | 0.6695    | 0.8664              | 0.0    | 0.8882 | 0.4056   | 0.6466    | 0.9538  | 0.9870 | 0.9519       | 0.7703      | 0.9380        | 0.7398        | 0.6931     |
-| 0.1884        | 2.0   | 1338 | 0.1022          | 0.8709            | 0.8877         | 0.8792     | 0.9648           | 0.9565         | 0.9263           | 0.8838 | 0.9202    | 0.8272 | 0.8935            | 0.8713            | 0.9326  | 0.9596         | 0.9403    | 0.0              | 0.9699              | 0.8038              | 0.6842      | 0.4724          | 0.3972          | 0.7884            | 0.8431  | 0.5134 | 0.9913   | 0.9830             | 0.8794      | 0.9270       | 0.9272    | 0.9718    | 0.9341  | 0.0    | 0.8180  | 0.8136  | 0.0        | 0.9206      | 0.0882     | 0.5360      | 0.7641             | 0.9789 | 0.6638          | 0.0           | 0.9930                 | 0.0                 | 0.9487      | 0.9646       | 0.9624         | 0.8082 | 0.9381    | 0.9679              | 0.9333 | 0.9392 | 0.8885   | 0.9371    | 0.9681  | 0.9883 | 0.9481       | 0.9683      | 0.7937        | 0.7858        | 0.8649     |
-| 0.0915        | 3.0   | 2007 | 0.0748          | 0.8934            | 0.9163         | 0.9047     | 0.9702           | 0.9783         | 0.9562           | 0.8920 | 0.9582    | 0.9474 | 0.9315            | 0.9056            | 0.9587  | 0.9785         | 0.9757    | 0.6190           | 0.9699              | 0.8209              | 0.7763      | 0.8342          | 0.0359          | 0.9223            | 0.7257  | 0.6549 | 0.9954   | 0.9818             | 0.8056      | 0.9518       | 0.9519    | 0.9887    | 0.9806  | 0.0039 | 0.8328  | 0.7923  | 0.0        | 0.9657      | 0.7634     | 0.6688      | 0.8525             | 0.9790 | 0.7631          | 0.0           | 0.9883                 | 0.0                 | 0.9646      | 0.9957       | 0.9829         | 0.9054 | 0.9413    | 0.9776              | 0.9421 | 0.9663 | 0.9659   | 0.9498    | 0.9682  | 0.9925 | 0.9944       | 0.9913      | 0.9497        | 0.9762        | 0.9148     |
-| 0.0782        | 4.0   | 2676 | 0.0632          | 0.9004            | 0.9286         | 0.9143     | 0.9622           | 0.9958         | 0.9832           | 0.9433 | 0.9646    | 0.9855 | 0.9655            | 0.9527            | 0.9808  | 0.9904         | 0.9707    | 0.5823           | 0.9670              | 0.8964              | 0.7717      | 0.8830          | 0.3387          | 0.9780            | 0.8780  | 0.6913 | 0.9776   | 0.9951             | 0.8857      | 0.9675       | 0.9865    | 0.9859    | 0.9742  | 0.2961 | 0.8395  | 0.2833  | 0.6471     | 0.9952      | 0.9355     | 0.8277      | 0.8808             | 0.9756 | 0.8859          | 0.0           | 0.9883                 | 0.0                 | 0.9838      | 0.9946       | 0.9853         | 0.9066 | 0.9351    | 0.9942              | 0.9134 | 0.9911 | 0.9493   | 0.9869    | 0.9768  | 0.9979 | 0.9944       | 0.9953      | 0.9572        | 0.9848        | 0.9489     |
-| 0.067         | 5.0   | 3345 | 0.0517          | 0.9389            | 0.9496         | 0.9442     | 0.9767           | 0.9936         | 0.9867           | 0.9579 | 0.9584    | 0.9884 | 0.9746            | 0.9542            | 0.9958  | 0.9887         | 0.9903    | 0.8952           | 0.9970              | 0.9397              | 0.8107      | 0.9140          | 0.6029          | 0.9848            | 0.9160  | 0.8132 | 0.9969   | 0.9951             | 0.9412      | 0.9731       | 0.9942    | 0.9915    | 0.9870  | 0.2084 | 0.7981  | 0.8064  | 0.7568     | 0.9827      | 0.9134     | 0.8692      | 0.8289             | 0.9877 | 0.9392          | 0.0513        | 0.9883                 | 0.6667              | 0.9722      | 0.9904       | 0.9943         | 0.9591 | 0.9676    | 0.9903              | 0.9748 | 0.9921 | 0.9790   | 0.9904    | 0.9787  | 0.9952 | 0.9926       | 0.9893      | 0.9749        | 0.9891        | 0.9653     |
-| 0.0568        | 6.0   | 4014 | 0.0472          | 0.9539            | 0.9594         | 0.9567     | 0.9797           | 0.9986         | 0.9986           | 0.9508 | 0.9703    | 0.9884 | 0.9722            | 0.9755            | 0.9975  | 0.9965         | 0.9904    | 0.84             | 0.9670              | 0.95                | 0.8498      | 0.9565          | 0.6278          | 0.9944            | 0.9165  | 0.8625 | 0.9985   | 0.9927             | 0.9630      | 0.9894       | 0.9962    | 0.9944    | 0.9913  | 0.1921 | 0.8345  | 0.8222  | 0.8293     | 0.9979      | 0.9516     | 0.9465      | 0.9340             | 0.9772 | 0.9563          | 0.6667        | 0.9977                 | 0.6667              | 0.9985      | 0.9978       | 0.9975         | 0.9704 | 0.9686    | 0.9971              | 0.9748 | 0.9931 | 0.9886   | 0.9948    | 0.9892  | 1.0    | 0.9963       | 0.9980      | 0.9749        | 0.9935        | 0.9830     |
-| 0.0482        | 7.0   | 4683 | 0.0426          | 0.9632            | 0.9678         | 0.9655     | 0.9826           | 0.9993         | 0.9972           | 0.9707 | 0.9926    | 0.9942 | 0.9602            | 0.9847            | 0.9992  | 0.9983         | 0.9952    | 0.9444           | 0.9970              | 0.9627              | 0.8634      | 0.9730          | 0.72            | 0.9958            | 0.9586  | 0.9102 | 1.0      | 0.9951             | 0.9925      | 0.9917       | 0.9943    | 1.0       | 0.9935  | 0.2256 | 0.8430  | 0.8488  | 0.8571     | 1.0         | 0.9516     | 0.9499      | 0.9198             | 0.9930 | 0.9704          | 0.6866        | 0.9977                 | 0.9091              | 1.0         | 0.9978       | 0.9975         | 0.9880 | 0.9797    | 0.9971              | 0.9915 | 0.9940 | 0.9951   | 1.0       | 0.9873  | 1.0    | 0.9963       | 0.9993      | 0.9950        | 0.9870        | 0.9863     |
-| 0.042         | 8.0   | 5352 | 0.0288          | 0.9697            | 0.9749         | 0.9723     | 0.9874           | 0.9993         | 0.9986           | 0.9808 | 0.9959    | 0.9942 | 0.9932            | 0.9911            | 1.0     | 1.0            | 0.9976    | 0.9444           | 0.9970              | 0.9716              | 0.8907      | 0.9730          | 0.7561          | 0.9944            | 0.9725  | 0.9429 | 1.0      | 0.9976             | 0.9925      | 0.9927       | 0.9971    | 1.0       | 0.9957  | 0.4056 | 0.8498  | 0.8565  | 0.9091     | 1.0         | 0.9672     | 0.9757      | 0.9848             | 0.9965 | 0.9808          | 0.7164        | 1.0                    | 1.0                 | 1.0         | 0.9978       | 0.9975         | 0.9880 | 0.9872    | 0.9971              | 0.9831 | 0.9960 | 0.9967   | 0.9991    | 0.9941  | 1.0    | 0.9981       | 1.0         | 0.9950        | 0.9892        | 0.9966     |
-| 0.0327        | 9.0   | 6021 | 0.0237          | 0.9688            | 0.9778         | 0.9733     | 0.9899           | 0.9993         | 0.9986           | 0.9872 | 0.9984    | 0.9942 | 0.9957            | 0.9898            | 1.0     | 1.0            | 1.0       | 0.9444           | 0.9970              | 0.9746              | 0.8958      | 0.9946          | 0.7761          | 0.9958            | 0.9787  | 0.9557 | 1.0      | 1.0                | 0.9925      | 0.9947       | 1.0       | 1.0       | 0.9957  | 0.5173 | 0.8455  | 0.8317  | 0.9091     | 1.0         | 0.9672     | 0.9871      | 0.9949             | 0.9965 | 0.9818          | 0.7576        | 1.0                    | 1.0                 | 1.0         | 0.9978       | 0.9975         | 0.9820 | 0.9879    | 0.9990              | 0.9916 | 0.9960 | 0.9967   | 0.9991    | 0.9971  | 1.0    | 0.9981       | 1.0         | 0.9950        | 0.9913        | 0.9966     |
-| 0.0267        | 10.0  | 6690 | 0.0211          | 0.9722            | 0.9792         | 0.9757     | 0.9915           | 0.9993         | 0.9986           | 0.9884 | 0.9984    | 0.9942 | 0.9974            | 0.9898            | 1.0     | 1.0            | 0.9976    | 0.9541           | 0.9970              | 0.9754              | 0.8966      | 0.9946          | 0.7697          | 0.9958            | 0.9778  | 0.9546 | 1.0      | 1.0                | 0.9925      | 0.9947       | 1.0       | 1.0       | 0.9978  | 0.5404 | 0.8455  | 0.8855  | 0.9091     | 1.0         | 0.9672     | 0.9855      | 0.9949             | 0.9965 | 0.9836          | 0.7385        | 1.0                    | 1.0                 | 1.0         | 0.9978       | 0.9975         | 0.9820 | 0.9872    | 1.0                 | 0.9916 | 0.9960 | 0.9967   | 0.9991    | 1.0     | 1.0    | 0.9981       | 1.0         | 0.9950        | 0.9870        | 0.9966     |
+| Training Loss | Epoch | Step  | Validation Loss | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | Accountname F1 | Accountnumber F1 | Age F1 | Amount F1 | Bic F1 | Bitcoinaddress F1 | Buildingnumber F1 | City F1 | Companyname F1 | County F1 | Creditcardcvv F1 | Creditcardissuer F1 | Creditcardnumber F1 | Currency F1 | Currencycode F1 | Currencyname F1 | Currencysymbol F1 | Date F1 | Dob F1 | Email F1 | Ethereumaddress F1 | Eyecolor F1 | Firstname F1 | Gender F1 | Height F1 | Iban F1 | Ip F1  | Ipv4 F1 | Ipv6 F1 | Jobarea F1 | Jobtitle F1 | Jobtype F1 | Lastname F1 | Litecoinaddress F1 | Mac F1 | Maskednumber F1 | Middlename F1 | Nearbygpscoordinate F1 | Ordinaldirection F1 | Password F1 | Phoneimei F1 | Phonenumber F1 | Pin F1 | Prefix F1 | Secondaryaddress F1 | Sex F1 | Ssn F1 | State F1 | Street F1 | Time F1 | Url F1 | Useragent F1 | Username F1 | Vehiclevin F1 | Vehiclevrm F1 | Zipcode F1 |
+|:-------------:|:-----:|:-----:|:---------------:|:-----------------:|:--------------:|:----------:|:----------------:|:--------------:|:----------------:|:------:|:---------:|:------:|:-----------------:|:-----------------:|:-------:|:--------------:|:---------:|:----------------:|:-------------------:|:-------------------:|:-----------:|:---------------:|:---------------:|:-----------------:|:-------:|:------:|:--------:|:------------------:|:-----------:|:------------:|:---------:|:---------:|:-------:|:------:|:-------:|:-------:|:----------:|:-----------:|:----------:|:-----------:|:------------------:|:------:|:---------------:|:-------------:|:----------------------:|:-------------------:|:-----------:|:------------:|:--------------:|:------:|:---------:|:-------------------:|:------:|:------:|:--------:|:---------:|:-------:|:------:|:------------:|:-----------:|:-------------:|:-------------:|:----------:|
+| 0.3984        | 1.0   | 2393  | 0.5120          | 0.7268            | 0.7819         | 0.7533     | 0.8741           | 0.9265         | 0.9819           | 0.8237 | 0.5053    | 0.2315 | 0.8197            | 0.7840            | 0.4886  | 0.8657         | 0.6338    | 0.8775           | 0.8575              | 0.7152              | 0.4533      | 0.0959          | 0.0             | 0.6480            | 0.7621  | 0.1884 | 0.9840   | 1.0                | 0.6194      | 0.8740       | 0.6610    | 0.9642    | 0.9039  | 0.0    | 0.8500  | 0.0220  | 0.6325     | 0.7840      | 0.6899     | 0.7667      | 0.0                | 0.2966 | 0.0             | 0.3682        | 0.9986                 | 0.9387              | 0.8558      | 0.9879       | 0.9687         | 0.7455 | 0.9252    | 0.9661              | 0.9110 | 0.9771 | 0.5282   | 0.7988    | 0.8453  | 0.9648 | 0.9804       | 0.9356      | 0.7741        | 0.6780        | 0.7915     |
+| 0.2097        | 2.0   | 4786  | 0.1406          | 0.8392            | 0.8913         | 0.8645     | 0.9509           | 0.9760         | 0.9114           | 0.9227 | 0.7647    | 0.9190 | 0.9554            | 0.8975            | 0.8881  | 0.9535         | 0.8414    | 0.9114           | 0.9820              | 0.8503              | 0.7525      | 0.6171          | 0.0077          | 0.8787            | 0.3161  | 0.2847 | 0.9924   | 0.9918             | 0.9495      | 0.9076       | 0.9625    | 0.9890    | 0.9870  | 0.0    | 0.8484  | 0.8007  | 0.8651     | 0.9660      | 0.9164     | 0.8695      | 0.8756             | 0.9685 | 0.7768          | 0.6697        | 0.9956                 | 0.9754              | 0.9652      | 0.9976       | 0.9849         | 0.7977 | 0.9373    | 0.9923              | 0.9815 | 0.9828 | 0.8093   | 0.9445    | 0.9735  | 0.9933 | 0.9651       | 0.9854      | 0.9843        | 0.975         | 0.8123     |
+| 0.1271        | 3.0   | 7179  | 0.1049          | 0.9218            | 0.9312         | 0.9265     | 0.9618           | 0.9950         | 0.9880           | 0.9172 | 0.9309    | 0.9652 | 0.8222            | 0.9160            | 0.9364  | 0.9749         | 0.9556    | 0.9211           | 0.9856              | 0.8939              | 0.8237      | 0.76            | 0.0080          | 0.9360            | 0.8735  | 0.5567 | 0.9993   | 0.9973             | 0.9872      | 0.9547       | 0.9773    | 0.9574    | 0.9694  | 0.0    | 0.8510  | 0.8032  | 0.9404     | 0.9844      | 0.9522     | 0.9294      | 0.8584             | 1.0    | 0.8603          | 0.8908        | 1.0                    | 0.9829              | 0.9513      | 1.0          | 0.9792         | 0.8579 | 0.9413    | 0.9968              | 0.9513 | 0.9929 | 0.9278   | 0.9484    | 0.9862  | 0.9940 | 0.8884       | 0.9943      | 0.9616        | 0.9648        | 0.9395     |
+| 0.1345        | 4.0   | 9572  | 0.0941          | 0.9463            | 0.9580         | 0.9521     | 0.9659           | 0.9975         | 0.9979           | 0.9356 | 0.9597    | 0.9084 | 0.9569            | 0.9827            | 0.9734  | 0.9835         | 0.9780    | 0.9634           | 0.9904              | 0.9393              | 0.8542      | 0.8915          | 0.4069          | 0.9636            | 0.8873  | 0.6572 | 0.9993   | 1.0                | 0.9923      | 0.9796       | 0.9983    | 0.9917    | 0.9972  | 0.0    | 0.8515  | 0.8027  | 0.9689     | 0.9943      | 0.9685     | 0.9668      | 0.8162             | 0.9912 | 0.9110          | 0.9364        | 1.0                    | 0.9848              | 0.9734      | 0.9976       | 0.9949         | 0.9739 | 0.9609    | 0.9968              | 0.9906 | 0.9899 | 0.9772   | 0.9875    | 0.9855  | 0.9978 | 1.0          | 0.9972      | 0.9867        | 0.9817        | 0.9780     |
+| 0.1067        | 5.0   | 11965 | 0.0724          | 0.9556            | 0.9659         | 0.9607     | 0.9699           | 0.9967         | 0.9965           | 0.9705 | 0.9742    | 0.9892 | 0.9736            | 0.9891            | 0.9794  | 0.9951         | 0.9860    | 0.9897           | 0.9892              | 0.9517              | 0.8386      | 0.9770          | 0.4186          | 0.9822            | 0.8869  | 0.7016 | 1.0      | 1.0                | 0.9949      | 0.9859       | 0.9983    | 1.0       | 0.9954  | 0.0075 | 0.8569  | 0.8012  | 0.9819     | 0.9979      | 0.9856     | 0.9843      | 0.9383             | 1.0    | 0.9318          | 0.9461        | 1.0                    | 0.9905              | 1.0         | 1.0          | 0.9978         | 0.9906 | 0.9646    | 0.9981              | 0.9924 | 0.9970 | 0.9862   | 0.9966    | 0.9951  | 0.9970 | 1.0          | 0.9981      | 0.9933        | 1.0           | 0.9913     |
+| 0.0808        | 6.0   | 14358 | 0.0693          | 0.9664            | 0.9732         | 0.9698     | 0.9728           | 1.0            | 1.0              | 0.9760 | 0.9897    | 0.9978 | 0.9907            | 0.9906            | 0.9930  | 0.9994         | 0.9939    | 1.0              | 0.9891              | 0.9590              | 0.9052      | 0.9875          | 0.7022          | 0.9892            | 0.9126  | 0.7438 | 1.0      | 1.0                | 1.0         | 0.9934       | 0.9991    | 1.0       | 1.0     | 0.1551 | 0.8393  | 0.8034  | 0.9942     | 0.9993      | 0.9928     | 0.9877      | 0.9770             | 1.0    | 0.9451          | 0.9773        | 1.0                    | 0.9924              | 1.0         | 1.0          | 1.0            | 0.9929 | 0.9722    | 0.9974              | 0.9949 | 0.9970 | 0.9941   | 0.9972    | 0.9967  | 1.0    | 1.0          | 0.9991      | 1.0           | 1.0           | 0.9890     |
+| 0.0779        | 7.0   | 16751 | 0.0697          | 0.9698            | 0.9756         | 0.9727     | 0.9739           | 0.9983         | 1.0              | 0.9815 | 0.9904    | 1.0    | 0.9938            | 0.9935            | 0.9930  | 0.9994         | 0.9935    | 1.0              | 0.9903              | 0.9584              | 0.9206      | 0.9917          | 0.7753          | 0.9914            | 0.9315  | 0.8305 | 1.0      | 1.0                | 1.0         | 0.9939       | 1.0       | 1.0       | 1.0     | 0.1404 | 0.8382  | 0.8029  | 0.9958     | 1.0         | 0.9944     | 0.9910      | 0.9875             | 1.0    | 0.9480          | 0.9788        | 1.0                    | 0.9924              | 1.0         | 1.0          | 1.0            | 0.9929 | 0.9747    | 0.9961              | 0.9949 | 0.9970 | 0.9925   | 0.9983    | 0.9967  | 1.0    | 1.0          | 0.9991      | 1.0           | 1.0           | 0.9953     |
 
 
 ### Framework versions
 
 - Transformers 4.35.2
-- Pytorch 2.1.0+cu118
+- Pytorch 2.1.0+cu121
 - Datasets 2.15.0
-- Tokenizers 0.15.0
+- Tokenizers 0.15.0

all_results.json

@@ -1,74 +1,74 @@
 {
-    "epoch": 10.0,
-    "eval_ACCOUNTNAME_f1": 0.9992912827781716,
-    "eval_ACCOUNTNUMBER_f1": 0.9985974754558204,
-    "eval_AGE_f1": 0.988446726572529,
-    "eval_AMOUNT_f1": 0.9983552631578947,
-    "eval_BIC_f1": 0.9942196531791907,
-    "eval_BITCOINADDRESS_f1": 0.9974293059125964,
-    "eval_BUILDINGNUMBER_f1": 0.989821882951654,
-    "eval_CITY_f1": 1.0,
-    "eval_COMPANYNAME_f1": 1.0,
-    "eval_COUNTY_f1": 0.9975903614457832,
-    "eval_CREDITCARDCVV_f1": 0.9541284403669724,
-    "eval_CREDITCARDISSUER_f1": 0.9970326409495549,
-    "eval_CREDITCARDNUMBER_f1": 0.9753846153846153,
-    "eval_CURRENCYCODE_f1": 0.9945945945945946,
-    "eval_CURRENCYNAME_f1": 0.7696969696969697,
-    "eval_CURRENCYSYMBOL_f1": 0.9958275382475661,
-    "eval_CURRENCY_f1": 0.8966442953020135,
-    "eval_DATE_f1": 0.9778380823156944,
-    "eval_DOB_f1": 0.9546247818499127,
+    "epoch": 7.0,
+    "eval_ACCOUNTNAME_f1": 1.0,
+    "eval_ACCOUNTNUMBER_f1": 1.0,
+    "eval_AGE_f1": 0.9759704251386321,
+    "eval_AMOUNT_f1": 0.9896755162241888,
+    "eval_BIC_f1": 0.9978213507625272,
+    "eval_BITCOINADDRESS_f1": 0.9906976744186046,
+    "eval_BUILDINGNUMBER_f1": 0.9906137184115523,
+    "eval_CITY_f1": 0.9929988331388565,
+    "eval_COMPANYNAME_f1": 0.9993853718500306,
+    "eval_COUNTY_f1": 0.9939393939393938,
+    "eval_CREDITCARDCVV_f1": 1.0,
+    "eval_CREDITCARDISSUER_f1": 0.9891435464414957,
+    "eval_CREDITCARDNUMBER_f1": 0.9589778076664425,
+    "eval_CURRENCYCODE_f1": 0.9874999999999999,
+    "eval_CURRENCYNAME_f1": 0.7022222222222221,
+    "eval_CURRENCYSYMBOL_f1": 0.9892163910855499,
+    "eval_CURRENCY_f1": 0.9052197802197802,
+    "eval_DATE_f1": 0.9125964010282777,
+    "eval_DOB_f1": 0.7437641723356009,
     "eval_EMAIL_f1": 1.0,
     "eval_ETHEREUMADDRESS_f1": 1.0,
-    "eval_EYECOLOR_f1": 0.9924812030075187,
-    "eval_FIRSTNAME_f1": 0.9947054930509596,
-    "eval_GENDER_f1": 1.0,
+    "eval_EYECOLOR_f1": 1.0,
+    "eval_FIRSTNAME_f1": 0.9934411893310013,
+    "eval_GENDER_f1": 0.999137187230371,
     "eval_HEIGHT_f1": 1.0,
-    "eval_IBAN_f1": 0.9978354978354979,
-    "eval_IPV4_f1": 0.8455497382198952,
-    "eval_IPV6_f1": 0.8854625550660794,
-    "eval_IP_f1": 0.5403508771929824,
-    "eval_JOBAREA_f1": 0.909090909090909,
-    "eval_JOBTITLE_f1": 1.0,
-    "eval_JOBTYPE_f1": 0.9672131147540983,
-    "eval_LASTNAME_f1": 0.9854604200323102,
-    "eval_LITECOINADDRESS_f1": 0.9948979591836735,
-    "eval_MAC_f1": 0.9964912280701754,
-    "eval_MASKEDNUMBER_f1": 0.9835766423357665,
-    "eval_MIDDLENAME_f1": 0.7384615384615385,
+    "eval_IBAN_f1": 1.0,
+    "eval_IPV4_f1": 0.8393393393393394,
+    "eval_IPV6_f1": 0.8034083992696287,
+    "eval_IP_f1": 0.1550632911392405,
+    "eval_JOBAREA_f1": 0.9941812136325853,
+    "eval_JOBTITLE_f1": 0.9992841803865427,
+    "eval_JOBTYPE_f1": 0.9927594529364441,
+    "eval_LASTNAME_f1": 0.9876789876789875,
+    "eval_LITECOINADDRESS_f1": 0.977035490605428,
+    "eval_MAC_f1": 1.0,
+    "eval_MASKEDNUMBER_f1": 0.9451162790697675,
+    "eval_MIDDLENAME_f1": 0.9773175542406312,
     "eval_NEARBYGPSCOORDINATE_f1": 1.0,
-    "eval_ORDINALDIRECTION_f1": 1.0,
+    "eval_ORDINALDIRECTION_f1": 0.9923664122137404,
     "eval_PASSWORD_f1": 1.0,
-    "eval_PHONEIMEI_f1": 0.9978448275862069,
-    "eval_PHONENUMBER_f1": 0.9975429975429976,
-    "eval_PIN_f1": 0.9820359281437125,
-    "eval_PREFIX_f1": 0.9872286079182631,
-    "eval_SECONDARYADDRESS_f1": 1.0,
-    "eval_SEX_f1": 0.9915966386554621,
-    "eval_SSN_f1": 0.996031746031746,
-    "eval_STATE_f1": 0.9967320261437909,
-    "eval_STREET_f1": 0.9991266375545852,
-    "eval_TIME_f1": 1.0,
+    "eval_PHONEIMEI_f1": 1.0,
+    "eval_PHONENUMBER_f1": 1.0,
+    "eval_PIN_f1": 0.9929078014184397,
+    "eval_PREFIX_f1": 0.9721608484312859,
+    "eval_SECONDARYADDRESS_f1": 0.9974226804123711,
+    "eval_SEX_f1": 0.9948892674616695,
+    "eval_SSN_f1": 0.9969604863221885,
+    "eval_STATE_f1": 0.994127068873465,
+    "eval_STREET_f1": 0.9971542401821286,
+    "eval_TIME_f1": 0.9967426710097721,
     "eval_URL_f1": 1.0,
-    "eval_USERAGENT_f1": 0.9981481481481481,
-    "eval_USERNAME_f1": 1.0,
-    "eval_VEHICLEVIN_f1": 0.9949748743718593,
-    "eval_VEHICLEVRM_f1": 0.987012987012987,
-    "eval_ZIPCODE_f1": 0.9965694682675815,
-    "eval_loss": 0.021080104634165764,
-    "eval_overall_accuracy": 0.9914984644710367,
-    "eval_overall_f1": 0.9756949897911104,
-    "eval_overall_precision": 0.9721830985915493,
-    "eval_overall_recall": 0.9792323455233292,
-    "eval_runtime": 40.5088,
-    "eval_samples": 10697,
-    "eval_samples_per_second": 264.066,
-    "eval_steps_per_second": 16.515,
-    "total_flos": 4787420476179840.0,
-    "train_loss": 0.1772846938248707,
-    "train_runtime": 1590.4288,
-    "train_samples": 32088,
-    "train_samples_per_second": 67.259,
-    "train_steps_per_second": 4.206
+    "eval_USERAGENT_f1": 1.0,
+    "eval_USERNAME_f1": 0.9990521327014218,
+    "eval_VEHICLEVIN_f1": 1.0,
+    "eval_VEHICLEVRM_f1": 1.0,
+    "eval_ZIPCODE_f1": 0.9889937106918238,
+    "eval_loss": 0.06927423179149628,
+    "eval_overall_accuracy": 0.972837236759815,
+    "eval_overall_f1": 0.9697938644196623,
+    "eval_overall_precision": 0.9664124529878814,
+    "eval_overall_recall": 0.97319902159333,
+    "eval_runtime": 107.6205,
+    "eval_samples": 9571,
+    "eval_samples_per_second": 88.933,
+    "eval_steps_per_second": 22.236,
+    "total_flos": 3973095621417330.0,
+    "train_loss": 0.2593412553578745,
+    "train_runtime": 2714.4275,
+    "train_samples": 33930,
+    "train_samples_per_second": 24.682,
+    "train_steps_per_second": 6.171
 }

eval_results.json

@@ -1,68 +1,68 @@
 {
-    "epoch": 10.0,
-    "eval_ACCOUNTNAME_f1": 0.9992912827781716,
-    "eval_ACCOUNTNUMBER_f1": 0.9985974754558204,
-    "eval_AGE_f1": 0.988446726572529,
-    "eval_AMOUNT_f1": 0.9983552631578947,
-    "eval_BIC_f1": 0.9942196531791907,
-    "eval_BITCOINADDRESS_f1": 0.9974293059125964,
-    "eval_BUILDINGNUMBER_f1": 0.989821882951654,
-    "eval_CITY_f1": 1.0,
-    "eval_COMPANYNAME_f1": 1.0,
-    "eval_COUNTY_f1": 0.9975903614457832,
-    "eval_CREDITCARDCVV_f1": 0.9541284403669724,
-    "eval_CREDITCARDISSUER_f1": 0.9970326409495549,
-    "eval_CREDITCARDNUMBER_f1": 0.9753846153846153,
-    "eval_CURRENCYCODE_f1": 0.9945945945945946,
-    "eval_CURRENCYNAME_f1": 0.7696969696969697,
-    "eval_CURRENCYSYMBOL_f1": 0.9958275382475661,
-    "eval_CURRENCY_f1": 0.8966442953020135,
-    "eval_DATE_f1": 0.9778380823156944,
-    "eval_DOB_f1": 0.9546247818499127,
+    "epoch": 7.0,
+    "eval_ACCOUNTNAME_f1": 1.0,
+    "eval_ACCOUNTNUMBER_f1": 1.0,
+    "eval_AGE_f1": 0.9759704251386321,
+    "eval_AMOUNT_f1": 0.9896755162241888,
+    "eval_BIC_f1": 0.9978213507625272,
+    "eval_BITCOINADDRESS_f1": 0.9906976744186046,
+    "eval_BUILDINGNUMBER_f1": 0.9906137184115523,
+    "eval_CITY_f1": 0.9929988331388565,
+    "eval_COMPANYNAME_f1": 0.9993853718500306,
+    "eval_COUNTY_f1": 0.9939393939393938,
+    "eval_CREDITCARDCVV_f1": 1.0,
+    "eval_CREDITCARDISSUER_f1": 0.9891435464414957,
+    "eval_CREDITCARDNUMBER_f1": 0.9589778076664425,
+    "eval_CURRENCYCODE_f1": 0.9874999999999999,
+    "eval_CURRENCYNAME_f1": 0.7022222222222221,
+    "eval_CURRENCYSYMBOL_f1": 0.9892163910855499,
+    "eval_CURRENCY_f1": 0.9052197802197802,
+    "eval_DATE_f1": 0.9125964010282777,
+    "eval_DOB_f1": 0.7437641723356009,
     "eval_EMAIL_f1": 1.0,
     "eval_ETHEREUMADDRESS_f1": 1.0,
-    "eval_EYECOLOR_f1": 0.9924812030075187,
-    "eval_FIRSTNAME_f1": 0.9947054930509596,
-    "eval_GENDER_f1": 1.0,
+    "eval_EYECOLOR_f1": 1.0,
+    "eval_FIRSTNAME_f1": 0.9934411893310013,
+    "eval_GENDER_f1": 0.999137187230371,
     "eval_HEIGHT_f1": 1.0,
-    "eval_IBAN_f1": 0.9978354978354979,
-    "eval_IPV4_f1": 0.8455497382198952,
-    "eval_IPV6_f1": 0.8854625550660794,
-    "eval_IP_f1": 0.5403508771929824,
-    "eval_JOBAREA_f1": 0.909090909090909,
-    "eval_JOBTITLE_f1": 1.0,
-    "eval_JOBTYPE_f1": 0.9672131147540983,
-    "eval_LASTNAME_f1": 0.9854604200323102,
-    "eval_LITECOINADDRESS_f1": 0.9948979591836735,
-    "eval_MAC_f1": 0.9964912280701754,
-    "eval_MASKEDNUMBER_f1": 0.9835766423357665,
-    "eval_MIDDLENAME_f1": 0.7384615384615385,
+    "eval_IBAN_f1": 1.0,
+    "eval_IPV4_f1": 0.8393393393393394,
+    "eval_IPV6_f1": 0.8034083992696287,
+    "eval_IP_f1": 0.1550632911392405,
+    "eval_JOBAREA_f1": 0.9941812136325853,
+    "eval_JOBTITLE_f1": 0.9992841803865427,
+    "eval_JOBTYPE_f1": 0.9927594529364441,
+    "eval_LASTNAME_f1": 0.9876789876789875,
+    "eval_LITECOINADDRESS_f1": 0.977035490605428,
+    "eval_MAC_f1": 1.0,
+    "eval_MASKEDNUMBER_f1": 0.9451162790697675,
+    "eval_MIDDLENAME_f1": 0.9773175542406312,
     "eval_NEARBYGPSCOORDINATE_f1": 1.0,
-    "eval_ORDINALDIRECTION_f1": 1.0,
+    "eval_ORDINALDIRECTION_f1": 0.9923664122137404,
     "eval_PASSWORD_f1": 1.0,
-    "eval_PHONEIMEI_f1": 0.9978448275862069,
-    "eval_PHONENUMBER_f1": 0.9975429975429976,
-    "eval_PIN_f1": 0.9820359281437125,
-    "eval_PREFIX_f1": 0.9872286079182631,
-    "eval_SECONDARYADDRESS_f1": 1.0,
-    "eval_SEX_f1": 0.9915966386554621,
-    "eval_SSN_f1": 0.996031746031746,
-    "eval_STATE_f1": 0.9967320261437909,
-    "eval_STREET_f1": 0.9991266375545852,
-    "eval_TIME_f1": 1.0,
+    "eval_PHONEIMEI_f1": 1.0,
+    "eval_PHONENUMBER_f1": 1.0,
+    "eval_PIN_f1": 0.9929078014184397,
+    "eval_PREFIX_f1": 0.9721608484312859,
+    "eval_SECONDARYADDRESS_f1": 0.9974226804123711,
+    "eval_SEX_f1": 0.9948892674616695,
+    "eval_SSN_f1": 0.9969604863221885,
+    "eval_STATE_f1": 0.994127068873465,
+    "eval_STREET_f1": 0.9971542401821286,
+    "eval_TIME_f1": 0.9967426710097721,
     "eval_URL_f1": 1.0,
-    "eval_USERAGENT_f1": 0.9981481481481481,
-    "eval_USERNAME_f1": 1.0,
-    "eval_VEHICLEVIN_f1": 0.9949748743718593,
-    "eval_VEHICLEVRM_f1": 0.987012987012987,
-    "eval_ZIPCODE_f1": 0.9965694682675815,
-    "eval_loss": 0.021080104634165764,
-    "eval_overall_accuracy": 0.9914984644710367,
-    "eval_overall_f1": 0.9756949897911104,
-    "eval_overall_precision": 0.9721830985915493,
-    "eval_overall_recall": 0.9792323455233292,
-    "eval_runtime": 40.5088,
-    "eval_samples": 10697,
-    "eval_samples_per_second": 264.066,
-    "eval_steps_per_second": 16.515
+    "eval_USERAGENT_f1": 1.0,
+    "eval_USERNAME_f1": 0.9990521327014218,
+    "eval_VEHICLEVIN_f1": 1.0,
+    "eval_VEHICLEVRM_f1": 1.0,
+    "eval_ZIPCODE_f1": 0.9889937106918238,
+    "eval_loss": 0.06927423179149628,
+    "eval_overall_accuracy": 0.972837236759815,
+    "eval_overall_f1": 0.9697938644196623,
+    "eval_overall_precision": 0.9664124529878814,
+    "eval_overall_recall": 0.97319902159333,
+    "eval_runtime": 107.6205,
+    "eval_samples": 9571,
+    "eval_samples_per_second": 88.933,
+    "eval_steps_per_second": 22.236
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f2563c2af9e9acb1c0e23d47930c2f0162e6ccc3c2ac9fce988d3d895d11871e
+oid sha256:179e55224bbc3ff4521c89e7c2e7cad1e7074c73b37eedd1cfde0c79848f2862
 size 735692028

train_results.json

@@ -1,9 +1,9 @@
 {
-    "epoch": 10.0,
-    "total_flos": 4787420476179840.0,
-    "train_loss": 0.1772846938248707,
-    "train_runtime": 1590.4288,
-    "train_samples": 32088,
-    "train_samples_per_second": 67.259,
-    "train_steps_per_second": 4.206
+    "epoch": 7.0,
+    "total_flos": 3973095621417330.0,
+    "train_loss": 0.2593412553578745,
+    "train_runtime": 2714.4275,
+    "train_samples": 33930,
+    "train_samples_per_second": 24.682,
+    "train_steps_per_second": 6.171
 }

trainer_state.json

@@ -1,854 +1,770 @@
 {
-  "best_metric": 0.021080104634165764,
-  "best_model_checkpoint": "deberta-v3-base_finetuned_ai4privacy_v2/checkpoint-6690",
-  "epoch": 10.0,
+  "best_metric": 0.06927423179149628,
+  "best_model_checkpoint": "deberta-v3-base_finetuned_ai4privacy_v2/checkpoint-14358",
+  "epoch": 7.0,
   "eval_steps": 500,
-  "global_step": 6690,
+  "global_step": 16751,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
-      "epoch": 0.75,
-      "learning_rate": 1.853512705530643e-05,
-      "loss": 1.5081,
+      "epoch": 0.21,
+      "learning_rate": 7.460459564309162e-06,
+      "loss": 2.7753,
       "step": 500
     },
     {
-      "epoch": 1.0,
-      "eval_ACCOUNTNAME_f1": 0.9134287661895023,
-      "eval_ACCOUNTNUMBER_f1": 0.7806031933767001,
-      "eval_AGE_f1": 0.7878048780487804,
-      "eval_AMOUNT_f1": 0.34473507712944335,
-      "eval_BIC_f1": 0.6646525679758308,
-      "eval_BITCOINADDRESS_f1": 0.8154631655725747,
-      "eval_BUILDINGNUMBER_f1": 0.7412882787750792,
-      "eval_CITY_f1": 0.6679245283018868,
-      "eval_COMPANYNAME_f1": 0.8354430379746836,
-      "eval_COUNTY_f1": 0.7718865598027127,
-      "eval_CREDITCARDCVV_f1": 0.0,
-      "eval_CREDITCARDISSUER_f1": 0.0,
-      "eval_CREDITCARDNUMBER_f1": 0.36220472440944884,
-      "eval_CURRENCYCODE_f1": 0.0,
-      "eval_CURRENCYNAME_f1": 0.0,
-      "eval_CURRENCYSYMBOL_f1": 0.016620498614958446,
-      "eval_CURRENCY_f1": 0.49849246231155775,
-      "eval_DATE_f1": 0.8013569543912552,
-      "eval_DOB_f1": 0.310828025477707,
-      "eval_EMAIL_f1": 0.9664974619289339,
-      "eval_ETHEREUMADDRESS_f1": 0.9878345498783455,
-      "eval_EYECOLOR_f1": 0.0,
-      "eval_FIRSTNAME_f1": 0.7813765182186234,
-      "eval_GENDER_f1": 0.7444444444444444,
-      "eval_HEIGHT_f1": 0.9243697478991596,
-      "eval_IBAN_f1": 0.9059080962800875,
-      "eval_IPV4_f1": 0.820737039350406,
-      "eval_IPV6_f1": 0.7063989962358844,
-      "eval_IP_f1": 0.0,
-      "eval_JOBAREA_f1": 0.0,
-      "eval_JOBTITLE_f1": 0.8649386084583902,
-      "eval_JOBTYPE_f1": 0.0,
-      "eval_LASTNAME_f1": 0.25206611570247933,
-      "eval_LITECOINADDRESS_f1": 0.0,
-      "eval_MAC_f1": 0.9755244755244754,
-      "eval_MASKEDNUMBER_f1": 0.5146618791143028,
-      "eval_MIDDLENAME_f1": 0.0,
-      "eval_NEARBYGPSCOORDINATE_f1": 0.991812865497076,
-      "eval_ORDINALDIRECTION_f1": 0.0,
-      "eval_PASSWORD_f1": 0.9443207126948775,
-      "eval_PHONEIMEI_f1": 0.9838536060279871,
-      "eval_PHONENUMBER_f1": 0.9700404858299595,
-      "eval_PIN_f1": 0.19191919191919193,
-      "eval_PREFIX_f1": 0.6694966646452395,
-      "eval_SECONDARYADDRESS_f1": 0.8663594470046083,
-      "eval_SEX_f1": 0.0,
-      "eval_SSN_f1": 0.8882051282051281,
-      "eval_STATE_f1": 0.4056224899598393,
-      "eval_STREET_f1": 0.6465798045602607,
-      "eval_TIME_f1": 0.9538461538461538,
-      "eval_URL_f1": 0.9869773817683345,
-      "eval_USERAGENT_f1": 0.9519408502772644,
-      "eval_USERNAME_f1": 0.7702627166014534,
-      "eval_VEHICLEVIN_f1": 0.9379652605459057,
-      "eval_VEHICLEVRM_f1": 0.7397769516728624,
-      "eval_ZIPCODE_f1": 0.6930501930501931,
-      "eval_loss": 0.16179943084716797,
-      "eval_overall_accuracy": 0.9463517949804088,
-      "eval_overall_f1": 0.742188837120852,
-      "eval_overall_precision": 0.7171891653497005,
-      "eval_overall_recall": 0.7689943253467844,
-      "eval_runtime": 44.1568,
-      "eval_samples_per_second": 242.251,
-      "eval_steps_per_second": 15.151,
-      "step": 669
-    },
-    {
-      "epoch": 1.49,
-      "learning_rate": 3.721973094170404e-05,
-      "loss": 0.1884,
+      "epoch": 0.42,
+      "learning_rate": 1.4920919128618324e-05,
+      "loss": 1.0147,
       "step": 1000
     },
     {
-      "epoch": 2.0,
-      "eval_ACCOUNTNAME_f1": 0.9564606741573034,
-      "eval_ACCOUNTNUMBER_f1": 0.9263456090651557,
-      "eval_AGE_f1": 0.8838383838383839,
-      "eval_AMOUNT_f1": 0.9201646090534981,
-      "eval_BIC_f1": 0.8271954674220964,
-      "eval_BITCOINADDRESS_f1": 0.8934707903780068,
-      "eval_BUILDINGNUMBER_f1": 0.8713105076741441,
-      "eval_CITY_f1": 0.9325751421608448,
-      "eval_COMPANYNAME_f1": 0.9595872742906276,
-      "eval_COUNTY_f1": 0.9402985074626865,
-      "eval_CREDITCARDCVV_f1": 0.0,
-      "eval_CREDITCARDISSUER_f1": 0.9698795180722892,
-      "eval_CREDITCARDNUMBER_f1": 0.8037634408602151,
-      "eval_CURRENCYCODE_f1": 0.47244094488188976,
-      "eval_CURRENCYNAME_f1": 0.3972125435540069,
-      "eval_CURRENCYSYMBOL_f1": 0.7883817427385892,
-      "eval_CURRENCY_f1": 0.6842105263157895,
-      "eval_DATE_f1": 0.8431372549019608,
-      "eval_DOB_f1": 0.5133819951338199,
-      "eval_EMAIL_f1": 0.9913309535951045,
-      "eval_ETHEREUMADDRESS_f1": 0.9830097087378641,
-      "eval_EYECOLOR_f1": 0.8794326241134752,
-      "eval_FIRSTNAME_f1": 0.9269537480063795,
-      "eval_GENDER_f1": 0.9272030651340996,
-      "eval_HEIGHT_f1": 0.9717514124293786,
-      "eval_IBAN_f1": 0.9340540540540541,
-      "eval_IPV4_f1": 0.8179551122194514,
-      "eval_IPV6_f1": 0.8136272545090181,
-      "eval_IP_f1": 0.0,
-      "eval_JOBAREA_f1": 0.0,
-      "eval_JOBTITLE_f1": 0.9206137424949967,
-      "eval_JOBTYPE_f1": 0.08823529411764705,
-      "eval_LASTNAME_f1": 0.5360230547550433,
-      "eval_LITECOINADDRESS_f1": 0.7641025641025642,
-      "eval_MAC_f1": 0.9789473684210527,
-      "eval_MASKEDNUMBER_f1": 0.6637649619151251,
-      "eval_MIDDLENAME_f1": 0.0,
-      "eval_NEARBYGPSCOORDINATE_f1": 0.9929577464788732,
-      "eval_ORDINALDIRECTION_f1": 0.0,
-      "eval_PASSWORD_f1": 0.9486988847583644,
-      "eval_PHONEIMEI_f1": 0.9645542427497316,
-      "eval_PHONENUMBER_f1": 0.9623567921440261,
-      "eval_PIN_f1": 0.8081841432225064,
-      "eval_PREFIX_f1": 0.9380530973451328,
-      "eval_SECONDARYADDRESS_f1": 0.9678675754625121,
-      "eval_SEX_f1": 0.9333333333333333,
-      "eval_SSN_f1": 0.9391634980988594,
-      "eval_STATE_f1": 0.8885400313971742,
-      "eval_STREET_f1": 0.9371231696813092,
-      "eval_TIME_f1": 0.9681159420289854,
-      "eval_URL_f1": 0.9883481836874571,
-      "eval_USERAGENT_f1": 0.9481481481481482,
-      "eval_USERNAME_f1": 0.9682959048877147,
-      "eval_VEHICLEVIN_f1": 0.7937336814621411,
-      "eval_VEHICLEVRM_f1": 0.7858407079646017,
-      "eval_ZIPCODE_f1": 0.8648648648648649,
-      "eval_loss": 0.10224327445030212,
-      "eval_overall_accuracy": 0.9647527268876417,
-      "eval_overall_f1": 0.8792349726775955,
-      "eval_overall_precision": 0.8709403031240334,
-      "eval_overall_recall": 0.8876891551071879,
-      "eval_runtime": 44.4987,
-      "eval_samples_per_second": 240.389,
-      "eval_steps_per_second": 15.034,
-      "step": 1338
-    },
-    {
-      "epoch": 2.24,
-      "learning_rate": 4.9892556438331585e-05,
-      "loss": 0.1196,
+      "epoch": 0.63,
+      "learning_rate": 2.2381378692927485e-05,
+      "loss": 0.6168,
       "step": 1500
     },
     {
-      "epoch": 2.99,
-      "learning_rate": 4.815828046188957e-05,
-      "loss": 0.0915,
+      "epoch": 0.84,
+      "learning_rate": 2.984183825723665e-05,
+      "loss": 0.3984,
       "step": 2000
     },
     {
-      "epoch": 3.0,
-      "eval_ACCOUNTNAME_f1": 0.9783368273934312,
-      "eval_ACCOUNTNUMBER_f1": 0.9562243502051984,
-      "eval_AGE_f1": 0.891959798994975,
-      "eval_AMOUNT_f1": 0.9581624282198523,
-      "eval_BIC_f1": 0.9473684210526315,
-      "eval_BITCOINADDRESS_f1": 0.9314616019818333,
-      "eval_BUILDINGNUMBER_f1": 0.9055627425614489,
-      "eval_CITY_f1": 0.9587194608256108,
-      "eval_COMPANYNAME_f1": 0.9784668389319552,
-      "eval_COUNTY_f1": 0.975728155339806,
-      "eval_CREDITCARDCVV_f1": 0.6190476190476191,
-      "eval_CREDITCARDISSUER_f1": 0.9698795180722892,
-      "eval_CREDITCARDNUMBER_f1": 0.8209255533199196,
-      "eval_CURRENCYCODE_f1": 0.8342245989304814,
-      "eval_CURRENCYNAME_f1": 0.03592814371257485,
-      "eval_CURRENCYSYMBOL_f1": 0.922266139657444,
-      "eval_CURRENCY_f1": 0.7763157894736842,
-      "eval_DATE_f1": 0.7257085020242915,
-      "eval_DOB_f1": 0.6549019607843137,
-      "eval_EMAIL_f1": 0.9953964194373401,
-      "eval_ETHEREUMADDRESS_f1": 0.9818181818181817,
-      "eval_EYECOLOR_f1": 0.8055555555555556,
-      "eval_FIRSTNAME_f1": 0.9517725966145002,
-      "eval_GENDER_f1": 0.9519321394910463,
-      "eval_HEIGHT_f1": 0.9887005649717514,
-      "eval_IBAN_f1": 0.980603448275862,
-      "eval_IPV4_f1": 0.8328154133001864,
-      "eval_IPV6_f1": 0.7922511690046761,
-      "eval_IP_f1": 0.0038535645472061657,
-      "eval_JOBAREA_f1": 0.0,
-      "eval_JOBTITLE_f1": 0.9656593406593407,
-      "eval_JOBTYPE_f1": 0.7633587786259541,
-      "eval_LASTNAME_f1": 0.6688,
-      "eval_LITECOINADDRESS_f1": 0.8524590163934426,
-      "eval_MAC_f1": 0.979020979020979,
-      "eval_MASKEDNUMBER_f1": 0.7631296891747053,
-      "eval_MIDDLENAME_f1": 0.0,
-      "eval_NEARBYGPSCOORDINATE_f1": 0.9882629107981221,
-      "eval_ORDINALDIRECTION_f1": 0.0,
-      "eval_PASSWORD_f1": 0.9645817633760362,
-      "eval_PHONEIMEI_f1": 0.9956896551724138,
-      "eval_PHONENUMBER_f1": 0.982940698619009,
-      "eval_PIN_f1": 0.9054441260744985,
-      "eval_PREFIX_f1": 0.9413218035824583,
-      "eval_SECONDARYADDRESS_f1": 0.977648202137998,
-      "eval_SEX_f1": 0.9421487603305785,
-      "eval_SSN_f1": 0.9663366336633663,
-      "eval_STATE_f1": 0.9658536585365853,
-      "eval_STREET_f1": 0.9497872340425533,
-      "eval_TIME_f1": 0.9682386910490857,
-      "eval_URL_f1": 0.9924605894448253,
-      "eval_USERAGENT_f1": 0.9944444444444445,
-      "eval_USERNAME_f1": 0.9913275517011341,
-      "eval_VEHICLEVIN_f1": 0.949748743718593,
-      "eval_VEHICLEVRM_f1": 0.9762419006479481,
-      "eval_ZIPCODE_f1": 0.9148211243611585,
-      "eval_loss": 0.07479778677225113,
-      "eval_overall_accuracy": 0.9701535528963253,
-      "eval_overall_f1": 0.9047155863356937,
-      "eval_overall_precision": 0.8933840479483633,
-      "eval_overall_recall": 0.9163382723833543,
-      "eval_runtime": 42.8584,
-      "eval_samples_per_second": 249.589,
-      "eval_steps_per_second": 15.61,
-      "step": 2007
-    },
-    {
-      "epoch": 3.74,
-      "learning_rate": 4.444341676079644e-05,
-      "loss": 0.0782,
+      "epoch": 1.0,
+      "eval_ACCOUNTNAME_f1": 0.9264822134387352,
+      "eval_ACCOUNTNUMBER_f1": 0.9819193324061196,
+      "eval_AGE_f1": 0.8237378210806023,
+      "eval_AMOUNT_f1": 0.505338078291815,
+      "eval_BIC_f1": 0.23153692614770457,
+      "eval_BITCOINADDRESS_f1": 0.8197146562905318,
+      "eval_BUILDINGNUMBER_f1": 0.7840236686390533,
+      "eval_CITY_f1": 0.48855905998763144,
+      "eval_COMPANYNAME_f1": 0.865654862282181,
+      "eval_COUNTY_f1": 0.633784373635967,
+      "eval_CREDITCARDCVV_f1": 0.8775055679287305,
+      "eval_CREDITCARDISSUER_f1": 0.8574537540805224,
+      "eval_CREDITCARDNUMBER_f1": 0.7152119700748131,
+      "eval_CURRENCYCODE_f1": 0.0958904109589041,
+      "eval_CURRENCYNAME_f1": 0.0,
+      "eval_CURRENCYSYMBOL_f1": 0.6479750778816199,
+      "eval_CURRENCY_f1": 0.4532891100055279,
+      "eval_DATE_f1": 0.7620984139894267,
+      "eval_DOB_f1": 0.1883720930232558,
+      "eval_EMAIL_f1": 0.9839645172296144,
+      "eval_ETHEREUMADDRESS_f1": 1.0,
+      "eval_EYECOLOR_f1": 0.6193548387096774,
+      "eval_FIRSTNAME_f1": 0.8739598132737975,
+      "eval_GENDER_f1": 0.6610328638497652,
+      "eval_HEIGHT_f1": 0.9641873278236913,
+      "eval_IBAN_f1": 0.9038634321653188,
+      "eval_IPV4_f1": 0.8500000000000001,
+      "eval_IPV6_f1": 0.02197802197802198,
+      "eval_IP_f1": 0.0,
+      "eval_JOBAREA_f1": 0.6324786324786326,
+      "eval_JOBTITLE_f1": 0.7839905826957033,
+      "eval_JOBTYPE_f1": 0.6899166034874906,
+      "eval_LASTNAME_f1": 0.7667121418826739,
+      "eval_LITECOINADDRESS_f1": 0.0,
+      "eval_MAC_f1": 0.2966292134831461,
+      "eval_MASKEDNUMBER_f1": 0.0,
+      "eval_MIDDLENAME_f1": 0.36821705426356593,
+      "eval_NEARBYGPSCOORDINATE_f1": 0.9985528219971056,
+      "eval_ORDINALDIRECTION_f1": 0.9386973180076628,
+      "eval_PASSWORD_f1": 0.8558322411533421,
+      "eval_PHONEIMEI_f1": 0.9878934624697336,
+      "eval_PHONENUMBER_f1": 0.968705547652916,
+      "eval_PIN_f1": 0.7454909819639278,
+      "eval_PREFIX_f1": 0.9252418645558488,
+      "eval_SECONDARYADDRESS_f1": 0.9660804020100502,
+      "eval_SEX_f1": 0.9109863672814755,
+      "eval_SSN_f1": 0.9770687936191427,
+      "eval_STATE_f1": 0.5281933256616801,
+      "eval_STREET_f1": 0.7987957852483694,
+      "eval_TIME_f1": 0.8453267162944583,
+      "eval_URL_f1": 0.964809384164223,
+      "eval_USERAGENT_f1": 0.9804325437693101,
+      "eval_USERNAME_f1": 0.9355870260392873,
+      "eval_VEHICLEVIN_f1": 0.7740585774058577,
+      "eval_VEHICLEVRM_f1": 0.678030303030303,
+      "eval_ZIPCODE_f1": 0.7915451895043731,
+      "eval_loss": 0.5120039582252502,
+      "eval_overall_accuracy": 0.8741298388558002,
+      "eval_overall_f1": 0.7533481152993349,
+      "eval_overall_precision": 0.7267993350283591,
+      "eval_overall_recall": 0.781909997106862,
+      "eval_runtime": 232.5447,
+      "eval_samples_per_second": 41.158,
+      "eval_steps_per_second": 10.29,
+      "step": 2393
+    },
+    {
+      "epoch": 1.04,
+      "learning_rate": 3.730229782154581e-05,
+      "loss": 0.3099,
       "step": 2500
     },
     {
-      "epoch": 4.0,
-      "eval_ACCOUNTNAME_f1": 0.9957507082152975,
-      "eval_ACCOUNTNUMBER_f1": 0.9832167832167832,
-      "eval_AGE_f1": 0.9432534678436318,
-      "eval_AMOUNT_f1": 0.9645507007419621,
-      "eval_BIC_f1": 0.9855072463768114,
-      "eval_BITCOINADDRESS_f1": 0.9654591406908171,
-      "eval_BUILDINGNUMBER_f1": 0.9527458492975734,
-      "eval_CITY_f1": 0.980849292256453,
-      "eval_COMPANYNAME_f1": 0.990426457789382,
-      "eval_COUNTY_f1": 0.970691676436108,
-      "eval_CREDITCARDCVV_f1": 0.5822784810126582,
-      "eval_CREDITCARDISSUER_f1": 0.9669669669669669,
-      "eval_CREDITCARDNUMBER_f1": 0.8963503649635036,
-      "eval_CURRENCYCODE_f1": 0.8829787234042553,
-      "eval_CURRENCYNAME_f1": 0.3387096774193549,
-      "eval_CURRENCYSYMBOL_f1": 0.9779614325068869,
-      "eval_CURRENCY_f1": 0.7717265353418309,
-      "eval_DATE_f1": 0.8780082987551867,
-      "eval_DOB_f1": 0.6913086913086914,
-      "eval_EMAIL_f1": 0.9776422764227641,
-      "eval_ETHEREUMADDRESS_f1": 0.9951338199513382,
-      "eval_EYECOLOR_f1": 0.8857142857142857,
-      "eval_FIRSTNAME_f1": 0.9674902470741222,
-      "eval_GENDER_f1": 0.9865125240847784,
-      "eval_HEIGHT_f1": 0.9859154929577464,
-      "eval_IBAN_f1": 0.9741935483870969,
-      "eval_IPV4_f1": 0.8394753279200501,
-      "eval_IPV6_f1": 0.2833333333333333,
-      "eval_IP_f1": 0.29608540925266896,
-      "eval_JOBAREA_f1": 0.6470588235294118,
-      "eval_JOBTITLE_f1": 0.9951557093425606,
-      "eval_JOBTYPE_f1": 0.9354838709677421,
-      "eval_LASTNAME_f1": 0.8277027027027027,
-      "eval_LITECOINADDRESS_f1": 0.8808290155440415,
-      "eval_MAC_f1": 0.975609756097561,
-      "eval_MASKEDNUMBER_f1": 0.8859060402684563,
-      "eval_MIDDLENAME_f1": 0.0,
-      "eval_NEARBYGPSCOORDINATE_f1": 0.9882629107981221,
-      "eval_ORDINALDIRECTION_f1": 0.0,
-      "eval_PASSWORD_f1": 0.9838337182448037,
-      "eval_PHONEIMEI_f1": 0.9946178686759957,
-      "eval_PHONENUMBER_f1": 0.9853420195439738,
-      "eval_PIN_f1": 0.9065934065934067,
-      "eval_PREFIX_f1": 0.9351121892055791,
-      "eval_SECONDARYADDRESS_f1": 0.9941520467836258,
-      "eval_SEX_f1": 0.9133858267716536,
-      "eval_SSN_f1": 0.9910802775024777,
-      "eval_STATE_f1": 0.9493243243243243,
-      "eval_STREET_f1": 0.986922406277245,
-      "eval_TIME_f1": 0.97678916827853,
-      "eval_URL_f1": 0.9979437971213161,
-      "eval_USERAGENT_f1": 0.9944444444444445,
-      "eval_USERNAME_f1": 0.9953239812959251,
-      "eval_VEHICLEVIN_f1": 0.9571788413098236,
-      "eval_VEHICLEVRM_f1": 0.9848156182212582,
-      "eval_ZIPCODE_f1": 0.9489177489177489,
-      "eval_loss": 0.06322777271270752,
-      "eval_overall_accuracy": 0.9622048077941332,
-      "eval_overall_f1": 0.9143112111277087,
-      "eval_overall_precision": 0.9004241335829736,
-      "eval_overall_recall": 0.9286333543505675,
-      "eval_runtime": 43.2116,
-      "eval_samples_per_second": 247.55,
-      "eval_steps_per_second": 15.482,
-      "step": 2676
-    },
-    {
-      "epoch": 4.48,
-      "learning_rate": 3.906567516921906e-05,
-      "loss": 0.067,
+      "epoch": 1.25,
+      "learning_rate": 4.476275738585497e-05,
+      "loss": 0.2843,
       "step": 3000
     },
     {
-      "epoch": 5.0,
-      "eval_ACCOUNTNAME_f1": 0.9936305732484078,
-      "eval_ACCOUNTNUMBER_f1": 0.9866853538892781,
-      "eval_AGE_f1": 0.9578544061302682,
-      "eval_AMOUNT_f1": 0.9583673469387756,
-      "eval_BIC_f1": 0.9884393063583815,
-      "eval_BITCOINADDRESS_f1": 0.9746192893401014,
-      "eval_BUILDINGNUMBER_f1": 0.954248366013072,
-      "eval_CITY_f1": 0.9958368026644463,
-      "eval_COMPANYNAME_f1": 0.9887250650477016,
-      "eval_COUNTY_f1": 0.9903147699757869,
-      "eval_CREDITCARDCVV_f1": 0.8952380952380952,
-      "eval_CREDITCARDISSUER_f1": 0.9970326409495549,
-      "eval_CREDITCARDNUMBER_f1": 0.9396681749622926,
-      "eval_CURRENCYCODE_f1": 0.9139784946236559,
-      "eval_CURRENCYNAME_f1": 0.6028985507246378,
-      "eval_CURRENCYSYMBOL_f1": 0.9848275862068966,
-      "eval_CURRENCY_f1": 0.8107382550335571,
-      "eval_DATE_f1": 0.9160439560439562,
-      "eval_DOB_f1": 0.8131672597864769,
-      "eval_EMAIL_f1": 0.9969325153374233,
-      "eval_ETHEREUMADDRESS_f1": 0.9951338199513382,
-      "eval_EYECOLOR_f1": 0.9411764705882353,
-      "eval_FIRSTNAME_f1": 0.9731499672560576,
-      "eval_GENDER_f1": 0.9942307692307691,
-      "eval_HEIGHT_f1": 0.9915014164305949,
-      "eval_IBAN_f1": 0.9870410367170628,
-      "eval_IPV4_f1": 0.7980835044490076,
-      "eval_IPV6_f1": 0.8063660477453581,
-      "eval_IP_f1": 0.20839363241678727,
-      "eval_JOBAREA_f1": 0.7567567567567568,
-      "eval_JOBTITLE_f1": 0.9827228749136143,
-      "eval_JOBTYPE_f1": 0.9133858267716536,
-      "eval_LASTNAME_f1": 0.869158878504673,
-      "eval_LITECOINADDRESS_f1": 0.8288770053475937,
-      "eval_MAC_f1": 0.9877408056042032,
-      "eval_MASKEDNUMBER_f1": 0.9391955098222637,
-      "eval_MIDDLENAME_f1": 0.05128205128205127,
-      "eval_NEARBYGPSCOORDINATE_f1": 0.9882629107981221,
-      "eval_ORDINALDIRECTION_f1": 0.6666666666666666,
-      "eval_PASSWORD_f1": 0.9722430607651913,
-      "eval_PHONEIMEI_f1": 0.990353697749196,
-      "eval_PHONENUMBER_f1": 0.9942669942669943,
-      "eval_PIN_f1": 0.9590643274853801,
-      "eval_PREFIX_f1": 0.9676190476190477,
-      "eval_SECONDARYADDRESS_f1": 0.9902534113060428,
-      "eval_SEX_f1": 0.9747899159663865,
-      "eval_SSN_f1": 0.9920792079207922,
-      "eval_STATE_f1": 0.9789983844911146,
-      "eval_STREET_f1": 0.990409764603313,
-      "eval_TIME_f1": 0.9786821705426356,
-      "eval_URL_f1": 0.9952087611225188,
-      "eval_USERAGENT_f1": 0.9925925925925926,
-      "eval_USERNAME_f1": 0.9893190921228303,
-      "eval_VEHICLEVIN_f1": 0.9748743718592965,
-      "eval_VEHICLEVRM_f1": 0.9891067538126361,
-      "eval_ZIPCODE_f1": 0.9653423499577346,
-      "eval_loss": 0.05170274153351784,
-      "eval_overall_accuracy": 0.9766853754103568,
-      "eval_overall_f1": 0.9442191265502852,
-      "eval_overall_precision": 0.9389393289950513,
-      "eval_overall_recall": 0.9495586380832283,
-      "eval_runtime": 44.92,
-      "eval_samples_per_second": 238.134,
-      "eval_steps_per_second": 14.893,
-      "step": 3345
-    },
-    {
-      "epoch": 5.23,
-      "learning_rate": 3.248498141437852e-05,
-      "loss": 0.0656,
+      "epoch": 1.46,
+      "learning_rate": 4.9984747938973194e-05,
+      "loss": 0.2633,
       "step": 3500
     },
     {
-      "epoch": 5.98,
-      "learning_rate": 2.5264142449579226e-05,
-      "loss": 0.0568,
+      "epoch": 1.67,
+      "learning_rate": 4.971116387357723e-05,
+      "loss": 0.2219,
       "step": 4000
     },
     {
-      "epoch": 6.0,
-      "eval_ACCOUNTNAME_f1": 0.9985835694050991,
-      "eval_ACCOUNTNUMBER_f1": 0.9985974754558204,
-      "eval_AGE_f1": 0.9508196721311476,
-      "eval_AMOUNT_f1": 0.9703459637561779,
-      "eval_BIC_f1": 0.9884393063583815,
-      "eval_BITCOINADDRESS_f1": 0.9721988205560236,
-      "eval_BUILDINGNUMBER_f1": 0.9754838709677419,
-      "eval_CITY_f1": 0.9974937343358397,
-      "eval_COMPANYNAME_f1": 0.9965156794425087,
-      "eval_COUNTY_f1": 0.9903846153846153,
-      "eval_CREDITCARDCVV_f1": 0.84,
-      "eval_CREDITCARDISSUER_f1": 0.9669669669669669,
-      "eval_CREDITCARDNUMBER_f1": 0.95,
-      "eval_CURRENCYCODE_f1": 0.9565217391304348,
-      "eval_CURRENCYNAME_f1": 0.6278317152103561,
-      "eval_CURRENCYSYMBOL_f1": 0.9944289693593316,
-      "eval_CURRENCY_f1": 0.8498074454428755,
-      "eval_DATE_f1": 0.9164743885556068,
-      "eval_DOB_f1": 0.8625204582651392,
-      "eval_EMAIL_f1": 0.9984670413898825,
-      "eval_ETHEREUMADDRESS_f1": 0.9927007299270073,
-      "eval_EYECOLOR_f1": 0.962962962962963,
-      "eval_FIRSTNAME_f1": 0.9894109861019192,
-      "eval_GENDER_f1": 0.9961612284069098,
-      "eval_HEIGHT_f1": 0.9943502824858756,
-      "eval_IBAN_f1": 0.9913419913419913,
-      "eval_IPV4_f1": 0.8345138441725691,
-      "eval_IPV6_f1": 0.8221614227086183,
-      "eval_IP_f1": 0.1920529801324503,
-      "eval_JOBAREA_f1": 0.8292682926829269,
-      "eval_JOBTITLE_f1": 0.9979181124219292,
-      "eval_JOBTYPE_f1": 0.9516129032258065,
-      "eval_LASTNAME_f1": 0.9465153970826581,
-      "eval_LITECOINADDRESS_f1": 0.9340369393139841,
-      "eval_MAC_f1": 0.9772329246935202,
-      "eval_MASKEDNUMBER_f1": 0.9562790697674419,
-      "eval_MIDDLENAME_f1": 0.6666666666666667,
-      "eval_NEARBYGPSCOORDINATE_f1": 0.9976525821596244,
-      "eval_ORDINALDIRECTION_f1": 0.6666666666666666,
-      "eval_PASSWORD_f1": 0.9984662576687117,
-      "eval_PHONEIMEI_f1": 0.9978448275862069,
-      "eval_PHONENUMBER_f1": 0.9975429975429976,
-      "eval_PIN_f1": 0.9704142011834319,
-      "eval_PREFIX_f1": 0.9686323713927227,
-      "eval_SECONDARYADDRESS_f1": 0.9970788704965919,
-      "eval_SEX_f1": 0.9747899159663865,
-      "eval_SSN_f1": 0.9930624380574826,
-      "eval_STATE_f1": 0.9885807504078303,
-      "eval_STREET_f1": 0.9947643979057592,
-      "eval_TIME_f1": 0.989247311827957,
-      "eval_URL_f1": 1.0,
-      "eval_USERAGENT_f1": 0.9962962962962963,
-      "eval_USERNAME_f1": 0.9979933110367893,
-      "eval_VEHICLEVIN_f1": 0.9748743718592965,
-      "eval_VEHICLEVRM_f1": 0.9934640522875817,
-      "eval_ZIPCODE_f1": 0.9829931972789114,
-      "eval_loss": 0.047248952090740204,
-      "eval_overall_accuracy": 0.9796844223234141,
-      "eval_overall_f1": 0.9566601178781926,
-      "eval_overall_precision": 0.953886538160163,
-      "eval_overall_recall": 0.9594498738965952,
-      "eval_runtime": 44.1595,
-      "eval_samples_per_second": 242.235,
-      "eval_steps_per_second": 15.15,
-      "step": 4014
-    },
-    {
-      "epoch": 6.73,
-      "learning_rate": 1.8020712977939116e-05,
-      "loss": 0.0482,
+      "epoch": 1.88,
+      "learning_rate": 4.909840312863836e-05,
+      "loss": 0.2097,
       "step": 4500
     },
     {
-      "epoch": 7.0,
-      "eval_ACCOUNTNAME_f1": 0.9992912827781716,
-      "eval_ACCOUNTNUMBER_f1": 0.9971949509116409,
-      "eval_AGE_f1": 0.9707006369426752,
-      "eval_AMOUNT_f1": 0.9926168990976211,
-      "eval_BIC_f1": 0.9942196531791907,
-      "eval_BITCOINADDRESS_f1": 0.9602076124567474,
-      "eval_BUILDINGNUMBER_f1": 0.9846938775510203,
-      "eval_CITY_f1": 0.9991645781119466,
-      "eval_COMPANYNAME_f1": 0.9982578397212544,
-      "eval_COUNTY_f1": 0.9951923076923077,
-      "eval_CREDITCARDCVV_f1": 0.9444444444444444,
-      "eval_CREDITCARDISSUER_f1": 0.9970326409495549,
-      "eval_CREDITCARDNUMBER_f1": 0.9627376425855514,
-      "eval_CURRENCYCODE_f1": 0.972972972972973,
-      "eval_CURRENCYNAME_f1": 0.72,
-      "eval_CURRENCYSYMBOL_f1": 0.9958275382475661,
-      "eval_CURRENCY_f1": 0.8633879781420765,
-      "eval_DATE_f1": 0.9586482881280569,
-      "eval_DOB_f1": 0.910233393177738,
-      "eval_EMAIL_f1": 1.0,
-      "eval_ETHEREUMADDRESS_f1": 0.9951338199513382,
-      "eval_EYECOLOR_f1": 0.9924812030075187,
-      "eval_FIRSTNAME_f1": 0.9917245945051308,
-      "eval_GENDER_f1": 0.9942528735632183,
-      "eval_HEIGHT_f1": 1.0,
-      "eval_IBAN_f1": 0.9935205183585313,
-      "eval_IPV4_f1": 0.8430268918073796,
-      "eval_IPV6_f1": 0.848780487804878,
-      "eval_IP_f1": 0.22564102564102562,
-      "eval_JOBAREA_f1": 0.8571428571428571,
-      "eval_JOBTITLE_f1": 1.0,
-      "eval_JOBTYPE_f1": 0.9516129032258065,
-      "eval_LASTNAME_f1": 0.9499192245557351,
-      "eval_LITECOINADDRESS_f1": 0.9198113207547169,
-      "eval_MAC_f1": 0.9929824561403509,
-      "eval_MASKEDNUMBER_f1": 0.9704251386321626,
-      "eval_MIDDLENAME_f1": 0.6865671641791046,
-      "eval_NEARBYGPSCOORDINATE_f1": 0.9976525821596244,
-      "eval_ORDINALDIRECTION_f1": 0.9090909090909091,
-      "eval_PASSWORD_f1": 1.0,
-      "eval_PHONEIMEI_f1": 0.9978448275862069,
-      "eval_PHONENUMBER_f1": 0.9975429975429976,
-      "eval_PIN_f1": 0.9879518072289156,
-      "eval_PREFIX_f1": 0.979721166032953,
-      "eval_SECONDARYADDRESS_f1": 0.9970731707317074,
-      "eval_SEX_f1": 0.9914529914529915,
-      "eval_SSN_f1": 0.9940476190476191,
-      "eval_STATE_f1": 0.9950900163666122,
-      "eval_STREET_f1": 1.0,
-      "eval_TIME_f1": 0.9872922776148583,
-      "eval_URL_f1": 1.0,
-      "eval_USERAGENT_f1": 0.9962962962962963,
-      "eval_USERNAME_f1": 0.9993311036789299,
-      "eval_VEHICLEVIN_f1": 0.9949748743718593,
-      "eval_VEHICLEVRM_f1": 0.9869565217391305,
-      "eval_ZIPCODE_f1": 0.9863247863247864,
-      "eval_loss": 0.04255259409546852,
-      "eval_overall_accuracy": 0.982552155035476,
-      "eval_overall_f1": 0.965500973049478,
-      "eval_overall_precision": 0.9632476956265934,
-      "eval_overall_recall": 0.967764817150063,
-      "eval_runtime": 46.2398,
-      "eval_samples_per_second": 231.338,
-      "eval_steps_per_second": 14.468,
-      "step": 4683
-    },
-    {
-      "epoch": 7.47,
-      "learning_rate": 1.1374179732055429e-05,
-      "loss": 0.042,
+      "epoch": 2.0,
+      "eval_ACCOUNTNAME_f1": 0.9760132340777501,
+      "eval_ACCOUNTNUMBER_f1": 0.9114103472714385,
+      "eval_AGE_f1": 0.9226519337016574,
+      "eval_AMOUNT_f1": 0.7646662171274444,
+      "eval_BIC_f1": 0.9190371991247264,
+      "eval_BITCOINADDRESS_f1": 0.9554043839758124,
+      "eval_BUILDINGNUMBER_f1": 0.8974535443909154,
+      "eval_CITY_f1": 0.8880643166357453,
+      "eval_COMPANYNAME_f1": 0.95350278983261,
+      "eval_COUNTY_f1": 0.8414096916299558,
+      "eval_CREDITCARDCVV_f1": 0.9114470842332614,
+      "eval_CREDITCARDISSUER_f1": 0.9819927971188475,
+      "eval_CREDITCARDNUMBER_f1": 0.8503075871496923,
+      "eval_CURRENCYCODE_f1": 0.6170678336980308,
+      "eval_CURRENCYNAME_f1": 0.007662835249042145,
+      "eval_CURRENCYSYMBOL_f1": 0.8787465940054496,
+      "eval_CURRENCY_f1": 0.7525103366804489,
+      "eval_DATE_f1": 0.31613165379926855,
+      "eval_DOB_f1": 0.28471818710052293,
+      "eval_EMAIL_f1": 0.9923875432525952,
+      "eval_ETHEREUMADDRESS_f1": 0.9918256130790191,
+      "eval_EYECOLOR_f1": 0.9494949494949495,
+      "eval_FIRSTNAME_f1": 0.907639923387955,
+      "eval_GENDER_f1": 0.962510897994769,
+      "eval_HEIGHT_f1": 0.989010989010989,
+      "eval_IBAN_f1": 0.9870129870129869,
+      "eval_IPV4_f1": 0.8483985765124555,
+      "eval_IPV6_f1": 0.8007268322228952,
+      "eval_IP_f1": 0.0,
+      "eval_JOBAREA_f1": 0.8650674662668666,
+      "eval_JOBTITLE_f1": 0.9659574468085106,
+      "eval_JOBTYPE_f1": 0.9163526752072344,
+      "eval_LASTNAME_f1": 0.8694558037118592,
+      "eval_LITECOINADDRESS_f1": 0.8755555555555555,
+      "eval_MAC_f1": 0.9685314685314685,
+      "eval_MASKEDNUMBER_f1": 0.7768014059753954,
+      "eval_MIDDLENAME_f1": 0.6696606786427146,
+      "eval_NEARBYGPSCOORDINATE_f1": 0.9956458635703919,
+      "eval_ORDINALDIRECTION_f1": 0.9754253308128544,
+      "eval_PASSWORD_f1": 0.9652173913043479,
+      "eval_PHONEIMEI_f1": 0.9975786924939467,
+      "eval_PHONENUMBER_f1": 0.9848812095032397,
+      "eval_PIN_f1": 0.7976653696498055,
+      "eval_PREFIX_f1": 0.9372599231754162,
+      "eval_SECONDARYADDRESS_f1": 0.9922779922779922,
+      "eval_SEX_f1": 0.9814814814814814,
+      "eval_SSN_f1": 0.9828109201213346,
+      "eval_STATE_f1": 0.8092592592592593,
+      "eval_STREET_f1": 0.944475602916433,
+      "eval_TIME_f1": 0.9735364875701684,
+      "eval_URL_f1": 0.993288590604027,
+      "eval_USERAGENT_f1": 0.9650924024640657,
+      "eval_USERNAME_f1": 0.9853566367501181,
+      "eval_VEHICLEVIN_f1": 0.9843400447427293,
+      "eval_VEHICLEVRM_f1": 0.975,
+      "eval_ZIPCODE_f1": 0.8123393316195372,
+      "eval_loss": 0.1406150460243225,
+      "eval_overall_accuracy": 0.9509451221294181,
+      "eval_overall_f1": 0.8644601747337542,
+      "eval_overall_precision": 0.8391689778129953,
+      "eval_overall_recall": 0.8913232161174088,
+      "eval_runtime": 108.1218,
+      "eval_samples_per_second": 88.521,
+      "eval_steps_per_second": 22.132,
+      "step": 4786
+    },
+    {
+      "epoch": 2.09,
+      "learning_rate": 4.815487624102509e-05,
+      "loss": 0.1793,
       "step": 5000
     },
     {
-      "epoch": 8.0,
-      "eval_ACCOUNTNAME_f1": 0.9992912827781716,
-      "eval_ACCOUNTNUMBER_f1": 0.9985974754558204,
-      "eval_AGE_f1": 0.9807938540332907,
-      "eval_AMOUNT_f1": 0.9958915365653246,
-      "eval_BIC_f1": 0.9942196531791907,
-      "eval_BITCOINADDRESS_f1": 0.9931506849315068,
-      "eval_BUILDINGNUMBER_f1": 0.9911280101394169,
-      "eval_CITY_f1": 1.0,
-      "eval_COMPANYNAME_f1": 1.0,
-      "eval_COUNTY_f1": 0.9975903614457832,
-      "eval_CREDITCARDCVV_f1": 0.9444444444444444,
-      "eval_CREDITCARDISSUER_f1": 0.9970326409495549,
-      "eval_CREDITCARDNUMBER_f1": 0.9715603382013834,
-      "eval_CURRENCYCODE_f1": 0.972972972972973,
-      "eval_CURRENCYNAME_f1": 0.7560975609756099,
-      "eval_CURRENCYSYMBOL_f1": 0.9944289693593316,
-      "eval_CURRENCY_f1": 0.8906666666666667,
-      "eval_DATE_f1": 0.9724853405502931,
-      "eval_DOB_f1": 0.9429323968393327,
+      "epoch": 2.3,
+      "learning_rate": 4.6893533726481556e-05,
+      "loss": 0.1465,
+      "step": 5500
+    },
+    {
+      "epoch": 2.51,
+      "learning_rate": 4.533168832543704e-05,
+      "loss": 0.1729,
+      "step": 6000
+    },
+    {
+      "epoch": 2.72,
+      "learning_rate": 4.349077737446525e-05,
+      "loss": 0.1537,
+      "step": 6500
+    },
+    {
+      "epoch": 2.93,
+      "learning_rate": 4.139606856499824e-05,
+      "loss": 0.1271,
+      "step": 7000
+    },
+    {
+      "epoch": 3.0,
+      "eval_ACCOUNTNAME_f1": 0.9950166112956811,
+      "eval_ACCOUNTNUMBER_f1": 0.9880197322057787,
+      "eval_AGE_f1": 0.9172093023255814,
+      "eval_AMOUNT_f1": 0.9309090909090908,
+      "eval_BIC_f1": 0.9652173913043479,
+      "eval_BITCOINADDRESS_f1": 0.822174226061915,
+      "eval_BUILDINGNUMBER_f1": 0.9159779614325069,
+      "eval_CITY_f1": 0.9364021804966686,
+      "eval_COMPANYNAME_f1": 0.9748928352725045,
+      "eval_COUNTY_f1": 0.9556109725685785,
+      "eval_CREDITCARDCVV_f1": 0.9211087420042643,
+      "eval_CREDITCARDISSUER_f1": 0.9856115107913669,
+      "eval_CREDITCARDNUMBER_f1": 0.8938992042440319,
+      "eval_CURRENCYCODE_f1": 0.76,
+      "eval_CURRENCYNAME_f1": 0.007999999999999998,
+      "eval_CURRENCYSYMBOL_f1": 0.9360222531293463,
+      "eval_CURRENCY_f1": 0.8237476808905382,
+      "eval_DATE_f1": 0.8735161686451085,
+      "eval_DOB_f1": 0.5567282321899736,
+      "eval_EMAIL_f1": 0.9993093922651933,
+      "eval_ETHEREUMADDRESS_f1": 0.9972527472527473,
+      "eval_EYECOLOR_f1": 0.9872122762148339,
+      "eval_FIRSTNAME_f1": 0.9546780345523895,
+      "eval_GENDER_f1": 0.9773123909249564,
+      "eval_HEIGHT_f1": 0.9574468085106383,
+      "eval_IBAN_f1": 0.9693593314763231,
+      "eval_IPV4_f1": 0.851033499643621,
+      "eval_IPV6_f1": 0.8031687995124923,
+      "eval_IP_f1": 0.0,
+      "eval_JOBAREA_f1": 0.9403973509933775,
+      "eval_JOBTITLE_f1": 0.9844192634560907,
+      "eval_JOBTYPE_f1": 0.9522240527182867,
+      "eval_LASTNAME_f1": 0.9294419499679282,
+      "eval_LITECOINADDRESS_f1": 0.8583690987124464,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.8603351955307262,
+      "eval_MIDDLENAME_f1": 0.8907963757749167,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.98292220113852,
+      "eval_PASSWORD_f1": 0.9513274336283185,
+      "eval_PHONEIMEI_f1": 1.0,
+      "eval_PHONENUMBER_f1": 0.9792114695340501,
+      "eval_PIN_f1": 0.85785536159601,
+      "eval_PREFIX_f1": 0.9412773253321903,
+      "eval_SECONDARYADDRESS_f1": 0.996776273372018,
+      "eval_SEX_f1": 0.9512843224092117,
+      "eval_SSN_f1": 0.9928934010152284,
+      "eval_STATE_f1": 0.9278033794162827,
+      "eval_STREET_f1": 0.9483516483516483,
+      "eval_TIME_f1": 0.9861900893582454,
+      "eval_URL_f1": 0.9940298507462686,
+      "eval_USERAGENT_f1": 0.8884339815762539,
+      "eval_USERNAME_f1": 0.9943074003795066,
+      "eval_VEHICLEVIN_f1": 0.9616252821670428,
+      "eval_VEHICLEVRM_f1": 0.9648033126293997,
+      "eval_ZIPCODE_f1": 0.9394904458598726,
+      "eval_loss": 0.10487008839845657,
+      "eval_overall_accuracy": 0.9617801261331822,
+      "eval_overall_f1": 0.9265209996074839,
+      "eval_overall_precision": 0.9218412351271837,
+      "eval_overall_recall": 0.9312485205544304,
+      "eval_runtime": 110.1516,
+      "eval_samples_per_second": 86.889,
+      "eval_steps_per_second": 21.725,
+      "step": 7179
+    },
+    {
+      "epoch": 3.13,
+      "learning_rate": 3.907631312794684e-05,
+      "loss": 0.1286,
+      "step": 7500
+    },
+    {
+      "epoch": 3.34,
+      "learning_rate": 3.656335120449241e-05,
+      "loss": 0.114,
+      "step": 8000
+    },
+    {
+      "epoch": 3.55,
+      "learning_rate": 3.389167481959079e-05,
+      "loss": 0.113,
+      "step": 8500
+    },
+    {
+      "epoch": 3.76,
+      "learning_rate": 3.1097954456659104e-05,
+      "loss": 0.1151,
+      "step": 9000
+    },
+    {
+      "epoch": 3.97,
+      "learning_rate": 2.8220535731513697e-05,
+      "loss": 0.1345,
+      "step": 9500
+    },
+    {
+      "epoch": 4.0,
+      "eval_ACCOUNTNAME_f1": 0.9975020815986678,
+      "eval_ACCOUNTNUMBER_f1": 0.9978947368421052,
+      "eval_AGE_f1": 0.9355742296918768,
+      "eval_AMOUNT_f1": 0.9597069597069597,
+      "eval_BIC_f1": 0.9083665338645419,
+      "eval_BITCOINADDRESS_f1": 0.9569303054032889,
+      "eval_BUILDINGNUMBER_f1": 0.9826589595375722,
+      "eval_CITY_f1": 0.9733570159857904,
+      "eval_COMPANYNAME_f1": 0.9835064141722665,
+      "eval_COUNTY_f1": 0.9779559118236473,
+      "eval_CREDITCARDCVV_f1": 0.9634146341463414,
+      "eval_CREDITCARDISSUER_f1": 0.9903614457831326,
+      "eval_CREDITCARDNUMBER_f1": 0.9392928619079386,
+      "eval_CURRENCYCODE_f1": 0.8915187376725838,
+      "eval_CURRENCYNAME_f1": 0.4068767908309455,
+      "eval_CURRENCYSYMBOL_f1": 0.9636493228795439,
+      "eval_CURRENCY_f1": 0.8541801685029164,
+      "eval_DATE_f1": 0.8872881355932203,
+      "eval_DOB_f1": 0.6571767497034402,
+      "eval_EMAIL_f1": 0.9993093922651933,
+      "eval_ETHEREUMADDRESS_f1": 1.0,
+      "eval_EYECOLOR_f1": 0.9922879177377892,
+      "eval_FIRSTNAME_f1": 0.9795516675779113,
+      "eval_GENDER_f1": 0.9982788296041308,
+      "eval_HEIGHT_f1": 0.9916897506925207,
+      "eval_IBAN_f1": 0.9972093023255814,
+      "eval_IPV4_f1": 0.8514570007107322,
+      "eval_IPV6_f1": 0.8026715239829992,
+      "eval_IP_f1": 0.0,
+      "eval_JOBAREA_f1": 0.9689034369885434,
+      "eval_JOBTITLE_f1": 0.9942938659058489,
+      "eval_JOBTYPE_f1": 0.9684542586750788,
+      "eval_LASTNAME_f1": 0.966765383349786,
+      "eval_LITECOINADDRESS_f1": 0.8162475822050289,
+      "eval_MAC_f1": 0.9911816578483246,
+      "eval_MASKEDNUMBER_f1": 0.9109653233364573,
+      "eval_MIDDLENAME_f1": 0.9363817097415508,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.9848484848484849,
+      "eval_PASSWORD_f1": 0.9733727810650887,
+      "eval_PHONEIMEI_f1": 0.9975845410628019,
+      "eval_PHONENUMBER_f1": 0.994909090909091,
+      "eval_PIN_f1": 0.9738717339667459,
+      "eval_PREFIX_f1": 0.960879120879121,
+      "eval_SECONDARYADDRESS_f1": 0.9967804249839021,
+      "eval_SEX_f1": 0.9906063193851409,
+      "eval_SSN_f1": 0.9898785425101214,
+      "eval_STATE_f1": 0.9772366331392272,
+      "eval_STREET_f1": 0.9875,
+      "eval_TIME_f1": 0.9854838709677419,
+      "eval_URL_f1": 0.9977661950856292,
+      "eval_USERAGENT_f1": 1.0,
+      "eval_USERNAME_f1": 0.9971590909090909,
+      "eval_VEHICLEVIN_f1": 0.9866666666666667,
+      "eval_VEHICLEVRM_f1": 0.9817444219066938,
+      "eval_ZIPCODE_f1": 0.9779874213836477,
+      "eval_loss": 0.0940849557518959,
+      "eval_overall_accuracy": 0.9658744863784346,
+      "eval_overall_f1": 0.9521270339149187,
+      "eval_overall_precision": 0.9462773419234166,
+      "eval_overall_recall": 0.9580494989611005,
+      "eval_runtime": 107.6668,
+      "eval_samples_per_second": 88.895,
+      "eval_steps_per_second": 22.226,
+      "step": 9572
+    },
+    {
+      "epoch": 4.18,
+      "learning_rate": 2.5298913074034208e-05,
+      "loss": 0.096,
+      "step": 10000
+    },
+    {
+      "epoch": 4.39,
+      "learning_rate": 2.237318764161293e-05,
+      "loss": 0.0922,
+      "step": 10500
+    },
+    {
+      "epoch": 4.6,
+      "learning_rate": 1.9483516904877303e-05,
+      "loss": 0.0866,
+      "step": 11000
+    },
+    {
+      "epoch": 4.81,
+      "learning_rate": 1.666956346047685e-05,
+      "loss": 0.1067,
+      "step": 11500
+    },
+    {
+      "epoch": 5.0,
+      "eval_ACCOUNTNAME_f1": 0.9966722129783694,
+      "eval_ACCOUNTNUMBER_f1": 0.9965010496850945,
+      "eval_AGE_f1": 0.970479704797048,
+      "eval_AMOUNT_f1": 0.9742078113485629,
+      "eval_BIC_f1": 0.9892008639308855,
+      "eval_BITCOINADDRESS_f1": 0.9736434108527132,
+      "eval_BUILDINGNUMBER_f1": 0.9891225525743292,
+      "eval_CITY_f1": 0.9793753682969948,
+      "eval_COMPANYNAME_f1": 0.9950920245398772,
+      "eval_COUNTY_f1": 0.985985985985986,
+      "eval_CREDITCARDCVV_f1": 0.9896907216494846,
+      "eval_CREDITCARDISSUER_f1": 0.9891696750902527,
+      "eval_CREDITCARDNUMBER_f1": 0.9517426273458446,
+      "eval_CURRENCYCODE_f1": 0.977035490605428,
+      "eval_CURRENCYNAME_f1": 0.4186046511627907,
+      "eval_CURRENCYSYMBOL_f1": 0.9821556031406138,
+      "eval_CURRENCY_f1": 0.8385852090032155,
+      "eval_DATE_f1": 0.8868817204301075,
+      "eval_DOB_f1": 0.7016393442622952,
       "eval_EMAIL_f1": 1.0,
-      "eval_ETHEREUMADDRESS_f1": 0.9975669099756691,
-      "eval_EYECOLOR_f1": 0.9924812030075187,
-      "eval_FIRSTNAME_f1": 0.9927248677248677,
-      "eval_GENDER_f1": 0.99712368168744,
+      "eval_ETHEREUMADDRESS_f1": 1.0,
+      "eval_EYECOLOR_f1": 0.9948717948717949,
+      "eval_FIRSTNAME_f1": 0.9859247855729053,
+      "eval_GENDER_f1": 0.9982758620689656,
       "eval_HEIGHT_f1": 1.0,
-      "eval_IBAN_f1": 0.9956803455723542,
-      "eval_IPV4_f1": 0.8498074454428755,
-      "eval_IPV6_f1": 0.8565371024734982,
-      "eval_IP_f1": 0.4055555555555556,
-      "eval_JOBAREA_f1": 0.909090909090909,
-      "eval_JOBTITLE_f1": 1.0,
-      "eval_JOBTYPE_f1": 0.9672131147540983,
-      "eval_LASTNAME_f1": 0.9756888168557536,
-      "eval_LITECOINADDRESS_f1": 0.9847715736040609,
-      "eval_MAC_f1": 0.9964912280701754,
-      "eval_MASKEDNUMBER_f1": 0.9808219178082193,
-      "eval_MIDDLENAME_f1": 0.716417910447761,
+      "eval_IBAN_f1": 0.9953574744661094,
+      "eval_IPV4_f1": 0.8569395017793595,
+      "eval_IPV6_f1": 0.8012195121951219,
+      "eval_IP_f1": 0.0075046904315197015,
+      "eval_JOBAREA_f1": 0.981907894736842,
+      "eval_JOBTITLE_f1": 0.9978525411596277,
+      "eval_JOBTYPE_f1": 0.9856230031948882,
+      "eval_LASTNAME_f1": 0.9843071786310518,
+      "eval_LITECOINADDRESS_f1": 0.9382716049382716,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.9318394024276376,
+      "eval_MIDDLENAME_f1": 0.9461134954697187,
       "eval_NEARBYGPSCOORDINATE_f1": 1.0,
-      "eval_ORDINALDIRECTION_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.9904761904761905,
       "eval_PASSWORD_f1": 1.0,
-      "eval_PHONEIMEI_f1": 0.9978448275862069,
-      "eval_PHONENUMBER_f1": 0.9975429975429976,
-      "eval_PIN_f1": 0.9879518072289156,
-      "eval_PREFIX_f1": 0.9871794871794872,
-      "eval_SECONDARYADDRESS_f1": 0.9970731707317074,
-      "eval_SEX_f1": 0.9830508474576272,
-      "eval_SSN_f1": 0.996031746031746,
-      "eval_STATE_f1": 0.9967320261437909,
-      "eval_STREET_f1": 0.9991266375545852,
-      "eval_TIME_f1": 0.9940944881889763,
-      "eval_URL_f1": 1.0,
-      "eval_USERAGENT_f1": 0.9981481481481481,
-      "eval_USERNAME_f1": 1.0,
-      "eval_VEHICLEVIN_f1": 0.9949748743718593,
-      "eval_VEHICLEVRM_f1": 0.9891540130151845,
-      "eval_ZIPCODE_f1": 0.9965694682675815,
-      "eval_loss": 0.02879495732486248,
-      "eval_overall_accuracy": 0.9873557132267288,
-      "eval_overall_f1": 0.9722718965551124,
-      "eval_overall_precision": 0.9696993453804241,
-      "eval_overall_recall": 0.974858133669609,
-      "eval_runtime": 45.3987,
-      "eval_samples_per_second": 235.623,
-      "eval_steps_per_second": 14.736,
-      "step": 5352
-    },
-    {
-      "epoch": 8.22,
-      "learning_rate": 5.892980518869237e-06,
-      "loss": 0.0376,
-      "step": 5500
+      "eval_PHONEIMEI_f1": 1.0,
+      "eval_PHONENUMBER_f1": 0.9978150036416606,
+      "eval_PIN_f1": 0.9905660377358492,
+      "eval_PREFIX_f1": 0.964613368283093,
+      "eval_SECONDARYADDRESS_f1": 0.9980657640232109,
+      "eval_SEX_f1": 0.9923534409515719,
+      "eval_SSN_f1": 0.9969604863221885,
+      "eval_STATE_f1": 0.986228813559322,
+      "eval_STREET_f1": 0.9965870307167235,
+      "eval_TIME_f1": 0.9951219512195122,
+      "eval_URL_f1": 0.9970238095238095,
+      "eval_USERAGENT_f1": 1.0,
+      "eval_USERNAME_f1": 0.9981042654028436,
+      "eval_VEHICLEVIN_f1": 0.9932885906040269,
+      "eval_VEHICLEVRM_f1": 1.0,
+      "eval_ZIPCODE_f1": 0.9913317572892041,
+      "eval_loss": 0.07239188998937607,
+      "eval_overall_accuracy": 0.9698826341845472,
+      "eval_overall_f1": 0.9607325049051668,
+      "eval_overall_precision": 0.9556324650654454,
+      "eval_overall_recall": 0.9658872728229136,
+      "eval_runtime": 110.9479,
+      "eval_samples_per_second": 86.266,
+      "eval_steps_per_second": 21.569,
+      "step": 11965
     },
     {
-      "epoch": 8.97,
-      "learning_rate": 2.0458891705253473e-06,
-      "loss": 0.0327,
-      "step": 6000
+      "epoch": 5.01,
+      "learning_rate": 1.3969950636335067e-05,
+      "loss": 0.1009,
+      "step": 12000
+    },
+    {
+      "epoch": 5.22,
+      "learning_rate": 1.1421732361535509e-05,
+      "loss": 0.0828,
+      "step": 12500
+    },
+    {
+      "epoch": 5.43,
+      "learning_rate": 9.059884577220362e-06,
+      "loss": 0.0752,
+      "step": 13000
+    },
+    {
+      "epoch": 5.64,
+      "learning_rate": 6.916825169215169e-06,
+      "loss": 0.087,
+      "step": 13500
+    },
+    {
+      "epoch": 5.85,
+      "learning_rate": 5.021969011614444e-06,
+      "loss": 0.0808,
+      "step": 14000
     },
     {
-      "epoch": 9.0,
-      "eval_ACCOUNTNAME_f1": 0.9992912827781716,
-      "eval_ACCOUNTNUMBER_f1": 0.9985974754558204,
-      "eval_AGE_f1": 0.9871794871794872,
-      "eval_AMOUNT_f1": 0.9983552631578947,
-      "eval_BIC_f1": 0.9942196531791907,
-      "eval_BITCOINADDRESS_f1": 0.9957155098543274,
-      "eval_BUILDINGNUMBER_f1": 0.989821882951654,
-      "eval_CITY_f1": 1.0,
-      "eval_COMPANYNAME_f1": 1.0,
-      "eval_COUNTY_f1": 1.0,
-      "eval_CREDITCARDCVV_f1": 0.9444444444444444,
-      "eval_CREDITCARDISSUER_f1": 0.9970326409495549,
-      "eval_CREDITCARDNUMBER_f1": 0.9746348962336665,
-      "eval_CURRENCYCODE_f1": 0.9945945945945946,
-      "eval_CURRENCYNAME_f1": 0.7761194029850748,
-      "eval_CURRENCYSYMBOL_f1": 0.9958275382475661,
-      "eval_CURRENCY_f1": 0.8958051420838972,
-      "eval_DATE_f1": 0.9787041232442228,
-      "eval_DOB_f1": 0.9556907037358819,
+      "epoch": 6.0,
+      "eval_ACCOUNTNAME_f1": 1.0,
+      "eval_ACCOUNTNUMBER_f1": 1.0,
+      "eval_AGE_f1": 0.9759704251386321,
+      "eval_AMOUNT_f1": 0.9896755162241888,
+      "eval_BIC_f1": 0.9978213507625272,
+      "eval_BITCOINADDRESS_f1": 0.9906976744186046,
+      "eval_BUILDINGNUMBER_f1": 0.9906137184115523,
+      "eval_CITY_f1": 0.9929988331388565,
+      "eval_COMPANYNAME_f1": 0.9993853718500306,
+      "eval_COUNTY_f1": 0.9939393939393938,
+      "eval_CREDITCARDCVV_f1": 1.0,
+      "eval_CREDITCARDISSUER_f1": 0.9891435464414957,
+      "eval_CREDITCARDNUMBER_f1": 0.9589778076664425,
+      "eval_CURRENCYCODE_f1": 0.9874999999999999,
+      "eval_CURRENCYNAME_f1": 0.7022222222222221,
+      "eval_CURRENCYSYMBOL_f1": 0.9892163910855499,
+      "eval_CURRENCY_f1": 0.9052197802197802,
+      "eval_DATE_f1": 0.9125964010282777,
+      "eval_DOB_f1": 0.7437641723356009,
       "eval_EMAIL_f1": 1.0,
       "eval_ETHEREUMADDRESS_f1": 1.0,
-      "eval_EYECOLOR_f1": 0.9924812030075187,
-      "eval_FIRSTNAME_f1": 0.9947054930509596,
-      "eval_GENDER_f1": 1.0,
+      "eval_EYECOLOR_f1": 1.0,
+      "eval_FIRSTNAME_f1": 0.9934411893310013,
+      "eval_GENDER_f1": 0.999137187230371,
       "eval_HEIGHT_f1": 1.0,
-      "eval_IBAN_f1": 0.9956709956709957,
-      "eval_IPV4_f1": 0.8455074337427277,
-      "eval_IPV6_f1": 0.8316981132075472,
-      "eval_IP_f1": 0.5173160173160173,
-      "eval_JOBAREA_f1": 0.909090909090909,
-      "eval_JOBTITLE_f1": 1.0,
-      "eval_JOBTYPE_f1": 0.9672131147540983,
-      "eval_LASTNAME_f1": 0.9870550161812298,
-      "eval_LITECOINADDRESS_f1": 0.9948979591836735,
-      "eval_MAC_f1": 0.9964912280701754,
-      "eval_MASKEDNUMBER_f1": 0.9817518248175182,
-      "eval_MIDDLENAME_f1": 0.7575757575757577,
+      "eval_IBAN_f1": 1.0,
+      "eval_IPV4_f1": 0.8393393393393394,
+      "eval_IPV6_f1": 0.8034083992696287,
+      "eval_IP_f1": 0.1550632911392405,
+      "eval_JOBAREA_f1": 0.9941812136325853,
+      "eval_JOBTITLE_f1": 0.9992841803865427,
+      "eval_JOBTYPE_f1": 0.9927594529364441,
+      "eval_LASTNAME_f1": 0.9876789876789875,
+      "eval_LITECOINADDRESS_f1": 0.977035490605428,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.9451162790697675,
+      "eval_MIDDLENAME_f1": 0.9773175542406312,
       "eval_NEARBYGPSCOORDINATE_f1": 1.0,
-      "eval_ORDINALDIRECTION_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.9923664122137404,
       "eval_PASSWORD_f1": 1.0,
-      "eval_PHONEIMEI_f1": 0.9978448275862069,
-      "eval_PHONENUMBER_f1": 0.9975429975429976,
-      "eval_PIN_f1": 0.9820359281437125,
-      "eval_PREFIX_f1": 0.9878594249201278,
-      "eval_SECONDARYADDRESS_f1": 0.9990243902439024,
-      "eval_SEX_f1": 0.9915966386554621,
-      "eval_SSN_f1": 0.996031746031746,
-      "eval_STATE_f1": 0.9967320261437909,
-      "eval_STREET_f1": 0.9991266375545852,
-      "eval_TIME_f1": 0.9970559371933267,
+      "eval_PHONEIMEI_f1": 1.0,
+      "eval_PHONENUMBER_f1": 1.0,
+      "eval_PIN_f1": 0.9929078014184397,
+      "eval_PREFIX_f1": 0.9721608484312859,
+      "eval_SECONDARYADDRESS_f1": 0.9974226804123711,
+      "eval_SEX_f1": 0.9948892674616695,
+      "eval_SSN_f1": 0.9969604863221885,
+      "eval_STATE_f1": 0.994127068873465,
+      "eval_STREET_f1": 0.9971542401821286,
+      "eval_TIME_f1": 0.9967426710097721,
       "eval_URL_f1": 1.0,
-      "eval_USERAGENT_f1": 0.9981481481481481,
-      "eval_USERNAME_f1": 1.0,
-      "eval_VEHICLEVIN_f1": 0.9949748743718593,
-      "eval_VEHICLEVRM_f1": 0.9913419913419913,
-      "eval_ZIPCODE_f1": 0.9965694682675815,
-      "eval_loss": 0.023680225014686584,
-      "eval_overall_accuracy": 0.9899036323202373,
-      "eval_overall_f1": 0.9732687442389534,
-      "eval_overall_precision": 0.9687658610861671,
-      "eval_overall_recall": 0.9778136822194199,
-      "eval_runtime": 42.4615,
-      "eval_samples_per_second": 251.922,
-      "eval_steps_per_second": 15.755,
-      "step": 6021
-    },
-    {
-      "epoch": 9.72,
-      "learning_rate": 1.619241524784959e-07,
-      "loss": 0.0267,
-      "step": 6500
+      "eval_USERAGENT_f1": 1.0,
+      "eval_USERNAME_f1": 0.9990521327014218,
+      "eval_VEHICLEVIN_f1": 1.0,
+      "eval_VEHICLEVRM_f1": 1.0,
+      "eval_ZIPCODE_f1": 0.9889937106918238,
+      "eval_loss": 0.06927423179149628,
+      "eval_overall_accuracy": 0.972837236759815,
+      "eval_overall_f1": 0.9697938644196623,
+      "eval_overall_precision": 0.9664124529878814,
+      "eval_overall_recall": 0.97319902159333,
+      "eval_runtime": 110.465,
+      "eval_samples_per_second": 86.643,
+      "eval_steps_per_second": 21.663,
+      "step": 14358
+    },
+    {
+      "epoch": 6.06,
+      "learning_rate": 3.4013242286423108e-06,
+      "loss": 0.0728,
+      "step": 14500
+    },
+    {
+      "epoch": 6.27,
+      "learning_rate": 2.077135216354695e-06,
+      "loss": 0.0667,
+      "step": 15000
+    },
+    {
+      "epoch": 6.48,
+      "learning_rate": 1.0675773239408465e-06,
+      "loss": 0.0848,
+      "step": 15500
     },
     {
-      "epoch": 10.0,
-      "eval_ACCOUNTNAME_f1": 0.9992912827781716,
-      "eval_ACCOUNTNUMBER_f1": 0.9985974754558204,
-      "eval_AGE_f1": 0.988446726572529,
-      "eval_AMOUNT_f1": 0.9983552631578947,
-      "eval_BIC_f1": 0.9942196531791907,
-      "eval_BITCOINADDRESS_f1": 0.9974293059125964,
-      "eval_BUILDINGNUMBER_f1": 0.989821882951654,
-      "eval_CITY_f1": 1.0,
-      "eval_COMPANYNAME_f1": 1.0,
-      "eval_COUNTY_f1": 0.9975903614457832,
-      "eval_CREDITCARDCVV_f1": 0.9541284403669724,
-      "eval_CREDITCARDISSUER_f1": 0.9970326409495549,
-      "eval_CREDITCARDNUMBER_f1": 0.9753846153846153,
-      "eval_CURRENCYCODE_f1": 0.9945945945945946,
-      "eval_CURRENCYNAME_f1": 0.7696969696969697,
-      "eval_CURRENCYSYMBOL_f1": 0.9958275382475661,
-      "eval_CURRENCY_f1": 0.8966442953020135,
-      "eval_DATE_f1": 0.9778380823156944,
-      "eval_DOB_f1": 0.9546247818499127,
+      "epoch": 6.69,
+      "learning_rate": 3.8650738532032825e-07,
+      "loss": 0.0619,
+      "step": 16000
+    },
+    {
+      "epoch": 6.9,
+      "learning_rate": 4.3273525150466786e-08,
+      "loss": 0.0779,
+      "step": 16500
+    },
+    {
+      "epoch": 7.0,
+      "eval_ACCOUNTNAME_f1": 0.9983333333333333,
+      "eval_ACCOUNTNUMBER_f1": 1.0,
+      "eval_AGE_f1": 0.9815157116451017,
+      "eval_AMOUNT_f1": 0.9904059040590405,
+      "eval_BIC_f1": 1.0,
+      "eval_BITCOINADDRESS_f1": 0.9937791601866253,
+      "eval_BUILDINGNUMBER_f1": 0.9934829833454019,
+      "eval_CITY_f1": 0.9929988331388565,
+      "eval_COMPANYNAME_f1": 0.9993853718500306,
+      "eval_COUNTY_f1": 0.9934508816120907,
+      "eval_CREDITCARDCVV_f1": 1.0,
+      "eval_CREDITCARDISSUER_f1": 0.9903381642512077,
+      "eval_CREDITCARDNUMBER_f1": 0.9583892617449664,
+      "eval_CURRENCYCODE_f1": 0.9916666666666667,
+      "eval_CURRENCYNAME_f1": 0.7753303964757708,
+      "eval_CURRENCYSYMBOL_f1": 0.9913544668587896,
+      "eval_CURRENCY_f1": 0.9206128133704735,
+      "eval_DATE_f1": 0.9315068493150684,
+      "eval_DOB_f1": 0.8305084745762712,
       "eval_EMAIL_f1": 1.0,
       "eval_ETHEREUMADDRESS_f1": 1.0,
-      "eval_EYECOLOR_f1": 0.9924812030075187,
-      "eval_FIRSTNAME_f1": 0.9947054930509596,
+      "eval_EYECOLOR_f1": 1.0,
+      "eval_FIRSTNAME_f1": 0.9938771047452438,
       "eval_GENDER_f1": 1.0,
       "eval_HEIGHT_f1": 1.0,
-      "eval_IBAN_f1": 0.9978354978354979,
-      "eval_IPV4_f1": 0.8455497382198952,
-      "eval_IPV6_f1": 0.8854625550660794,
-      "eval_IP_f1": 0.5403508771929824,
-      "eval_JOBAREA_f1": 0.909090909090909,
+      "eval_IBAN_f1": 1.0,
+      "eval_IPV4_f1": 0.8381804623415362,
+      "eval_IPV6_f1": 0.8029197080291971,
+      "eval_IP_f1": 0.14035087719298248,
+      "eval_JOBAREA_f1": 0.9958368026644463,
       "eval_JOBTITLE_f1": 1.0,
-      "eval_JOBTYPE_f1": 0.9672131147540983,
-      "eval_LASTNAME_f1": 0.9854604200323102,
-      "eval_LITECOINADDRESS_f1": 0.9948979591836735,
-      "eval_MAC_f1": 0.9964912280701754,
-      "eval_MASKEDNUMBER_f1": 0.9835766423357665,
-      "eval_MIDDLENAME_f1": 0.7384615384615385,
+      "eval_JOBTYPE_f1": 0.9943593875906527,
+      "eval_LASTNAME_f1": 0.9910029990003333,
+      "eval_LITECOINADDRESS_f1": 0.9875,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.9479553903345725,
+      "eval_MIDDLENAME_f1": 0.9788281634662728,
       "eval_NEARBYGPSCOORDINATE_f1": 1.0,
-      "eval_ORDINALDIRECTION_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.9923664122137404,
       "eval_PASSWORD_f1": 1.0,
-      "eval_PHONEIMEI_f1": 0.9978448275862069,
-      "eval_PHONENUMBER_f1": 0.9975429975429976,
-      "eval_PIN_f1": 0.9820359281437125,
-      "eval_PREFIX_f1": 0.9872286079182631,
-      "eval_SECONDARYADDRESS_f1": 1.0,
-      "eval_SEX_f1": 0.9915966386554621,
-      "eval_SSN_f1": 0.996031746031746,
-      "eval_STATE_f1": 0.9967320261437909,
-      "eval_STREET_f1": 0.9991266375545852,
-      "eval_TIME_f1": 1.0,
+      "eval_PHONEIMEI_f1": 1.0,
+      "eval_PHONENUMBER_f1": 1.0,
+      "eval_PIN_f1": 0.9929078014184397,
+      "eval_PREFIX_f1": 0.9747003994673767,
+      "eval_SECONDARYADDRESS_f1": 0.9961340206185567,
+      "eval_SEX_f1": 0.9948892674616695,
+      "eval_SSN_f1": 0.9969604863221885,
+      "eval_STATE_f1": 0.9925133689839571,
+      "eval_STREET_f1": 0.9982925441092771,
+      "eval_TIME_f1": 0.9967426710097721,
       "eval_URL_f1": 1.0,
-      "eval_USERAGENT_f1": 0.9981481481481481,
-      "eval_USERNAME_f1": 1.0,
-      "eval_VEHICLEVIN_f1": 0.9949748743718593,
-      "eval_VEHICLEVRM_f1": 0.987012987012987,
-      "eval_ZIPCODE_f1": 0.9965694682675815,
-      "eval_loss": 0.021080104634165764,
-      "eval_overall_accuracy": 0.9914984644710367,
-      "eval_overall_f1": 0.9756949897911104,
-      "eval_overall_precision": 0.9721830985915493,
-      "eval_overall_recall": 0.9792323455233292,
-      "eval_runtime": 45.9584,
-      "eval_samples_per_second": 232.754,
-      "eval_steps_per_second": 14.557,
-      "step": 6690
-    },
-    {
-      "epoch": 10.0,
-      "step": 6690,
-      "total_flos": 4787420476179840.0,
-      "train_loss": 0.1772846938248707,
-      "train_runtime": 1590.4288,
-      "train_samples_per_second": 67.259,
-      "train_steps_per_second": 4.206
-    },
-    {
-      "epoch": 10.0,
-      "eval_ACCOUNTNAME_f1": 0.9992912827781716,
-      "eval_ACCOUNTNUMBER_f1": 0.9985974754558204,
-      "eval_AGE_f1": 0.988446726572529,
-      "eval_AMOUNT_f1": 0.9983552631578947,
-      "eval_BIC_f1": 0.9942196531791907,
-      "eval_BITCOINADDRESS_f1": 0.9974293059125964,
-      "eval_BUILDINGNUMBER_f1": 0.989821882951654,
-      "eval_CITY_f1": 1.0,
-      "eval_COMPANYNAME_f1": 1.0,
-      "eval_COUNTY_f1": 0.9975903614457832,
-      "eval_CREDITCARDCVV_f1": 0.9541284403669724,
-      "eval_CREDITCARDISSUER_f1": 0.9970326409495549,
-      "eval_CREDITCARDNUMBER_f1": 0.9753846153846153,
-      "eval_CURRENCYCODE_f1": 0.9945945945945946,
-      "eval_CURRENCYNAME_f1": 0.7696969696969697,
-      "eval_CURRENCYSYMBOL_f1": 0.9958275382475661,
-      "eval_CURRENCY_f1": 0.8966442953020135,
-      "eval_DATE_f1": 0.9778380823156944,
-      "eval_DOB_f1": 0.9546247818499127,
+      "eval_USERAGENT_f1": 1.0,
+      "eval_USERNAME_f1": 0.9990521327014218,
+      "eval_VEHICLEVIN_f1": 1.0,
+      "eval_VEHICLEVRM_f1": 1.0,
+      "eval_ZIPCODE_f1": 0.9952755905511811,
+      "eval_loss": 0.0696588084101677,
+      "eval_overall_accuracy": 0.9738600962072371,
+      "eval_overall_f1": 0.9726880564333198,
+      "eval_overall_precision": 0.9698269100036605,
+      "eval_overall_recall": 0.9755661345046158,
+      "eval_runtime": 106.6576,
+      "eval_samples_per_second": 89.736,
+      "eval_steps_per_second": 22.436,
+      "step": 16751
+    },
+    {
+      "epoch": 7.0,
+      "step": 16751,
+      "total_flos": 3973095621417330.0,
+      "train_loss": 0.2593412553578745,
+      "train_runtime": 2714.4275,
+      "train_samples_per_second": 24.682,
+      "train_steps_per_second": 6.171
+    },
+    {
+      "epoch": 7.0,
+      "eval_ACCOUNTNAME_f1": 1.0,
+      "eval_ACCOUNTNUMBER_f1": 1.0,
+      "eval_AGE_f1": 0.9759704251386321,
+      "eval_AMOUNT_f1": 0.9896755162241888,
+      "eval_BIC_f1": 0.9978213507625272,
+      "eval_BITCOINADDRESS_f1": 0.9906976744186046,
+      "eval_BUILDINGNUMBER_f1": 0.9906137184115523,
+      "eval_CITY_f1": 0.9929988331388565,
+      "eval_COMPANYNAME_f1": 0.9993853718500306,
+      "eval_COUNTY_f1": 0.9939393939393938,
+      "eval_CREDITCARDCVV_f1": 1.0,
+      "eval_CREDITCARDISSUER_f1": 0.9891435464414957,
+      "eval_CREDITCARDNUMBER_f1": 0.9589778076664425,
+      "eval_CURRENCYCODE_f1": 0.9874999999999999,
+      "eval_CURRENCYNAME_f1": 0.7022222222222221,
+      "eval_CURRENCYSYMBOL_f1": 0.9892163910855499,
+      "eval_CURRENCY_f1": 0.9052197802197802,
+      "eval_DATE_f1": 0.9125964010282777,
+      "eval_DOB_f1": 0.7437641723356009,
       "eval_EMAIL_f1": 1.0,
       "eval_ETHEREUMADDRESS_f1": 1.0,
-      "eval_EYECOLOR_f1": 0.9924812030075187,
-      "eval_FIRSTNAME_f1": 0.9947054930509596,
-      "eval_GENDER_f1": 1.0,
+      "eval_EYECOLOR_f1": 1.0,
+      "eval_FIRSTNAME_f1": 0.9934411893310013,
+      "eval_GENDER_f1": 0.999137187230371,
       "eval_HEIGHT_f1": 1.0,
-      "eval_IBAN_f1": 0.9978354978354979,
-      "eval_IPV4_f1": 0.8455497382198952,
-      "eval_IPV6_f1": 0.8854625550660794,
-      "eval_IP_f1": 0.5403508771929824,
-      "eval_JOBAREA_f1": 0.909090909090909,
-      "eval_JOBTITLE_f1": 1.0,
-      "eval_JOBTYPE_f1": 0.9672131147540983,
-      "eval_LASTNAME_f1": 0.9854604200323102,
-      "eval_LITECOINADDRESS_f1": 0.9948979591836735,
-      "eval_MAC_f1": 0.9964912280701754,
-      "eval_MASKEDNUMBER_f1": 0.9835766423357665,
-      "eval_MIDDLENAME_f1": 0.7384615384615385,
+      "eval_IBAN_f1": 1.0,
+      "eval_IPV4_f1": 0.8393393393393394,
+      "eval_IPV6_f1": 0.8034083992696287,
+      "eval_IP_f1": 0.1550632911392405,
+      "eval_JOBAREA_f1": 0.9941812136325853,
+      "eval_JOBTITLE_f1": 0.9992841803865427,
+      "eval_JOBTYPE_f1": 0.9927594529364441,
+      "eval_LASTNAME_f1": 0.9876789876789875,
+      "eval_LITECOINADDRESS_f1": 0.977035490605428,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.9451162790697675,
+      "eval_MIDDLENAME_f1": 0.9773175542406312,
       "eval_NEARBYGPSCOORDINATE_f1": 1.0,
-      "eval_ORDINALDIRECTION_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.9923664122137404,
       "eval_PASSWORD_f1": 1.0,
-      "eval_PHONEIMEI_f1": 0.9978448275862069,
-      "eval_PHONENUMBER_f1": 0.9975429975429976,
-      "eval_PIN_f1": 0.9820359281437125,
-      "eval_PREFIX_f1": 0.9872286079182631,
-      "eval_SECONDARYADDRESS_f1": 1.0,
-      "eval_SEX_f1": 0.9915966386554621,
-      "eval_SSN_f1": 0.996031746031746,
-      "eval_STATE_f1": 0.9967320261437909,
-      "eval_STREET_f1": 0.9991266375545852,
-      "eval_TIME_f1": 1.0,
+      "eval_PHONEIMEI_f1": 1.0,
+      "eval_PHONENUMBER_f1": 1.0,
+      "eval_PIN_f1": 0.9929078014184397,
+      "eval_PREFIX_f1": 0.9721608484312859,
+      "eval_SECONDARYADDRESS_f1": 0.9974226804123711,
+      "eval_SEX_f1": 0.9948892674616695,
+      "eval_SSN_f1": 0.9969604863221885,
+      "eval_STATE_f1": 0.994127068873465,
+      "eval_STREET_f1": 0.9971542401821286,
+      "eval_TIME_f1": 0.9967426710097721,
       "eval_URL_f1": 1.0,
-      "eval_USERAGENT_f1": 0.9981481481481481,
-      "eval_USERNAME_f1": 1.0,
-      "eval_VEHICLEVIN_f1": 0.9949748743718593,
-      "eval_VEHICLEVRM_f1": 0.987012987012987,
-      "eval_ZIPCODE_f1": 0.9965694682675815,
-      "eval_loss": 0.021080104634165764,
-      "eval_overall_accuracy": 0.9914984644710367,
-      "eval_overall_f1": 0.9756949897911104,
-      "eval_overall_precision": 0.9721830985915493,
-      "eval_overall_recall": 0.9792323455233292,
-      "eval_runtime": 40.5088,
-      "eval_samples_per_second": 264.066,
-      "eval_steps_per_second": 16.515,
-      "step": 6690
+      "eval_USERAGENT_f1": 1.0,
+      "eval_USERNAME_f1": 0.9990521327014218,
+      "eval_VEHICLEVIN_f1": 1.0,
+      "eval_VEHICLEVRM_f1": 1.0,
+      "eval_ZIPCODE_f1": 0.9889937106918238,
+      "eval_loss": 0.06927423179149628,
+      "eval_overall_accuracy": 0.972837236759815,
+      "eval_overall_f1": 0.9697938644196623,
+      "eval_overall_precision": 0.9664124529878814,
+      "eval_overall_recall": 0.97319902159333,
+      "eval_runtime": 107.6205,
+      "eval_samples_per_second": 88.933,
+      "eval_steps_per_second": 22.236,
+      "step": 16751
     }
   ],
   "logging_steps": 500,
-  "max_steps": 6690,
-  "num_train_epochs": 10,
+  "max_steps": 16751,
+  "num_train_epochs": 7,
   "save_steps": 500,
-  "total_flos": 4787420476179840.0,
+  "total_flos": 3973095621417330.0,
   "trial_name": null,
   "trial_params": null
 }