microsoft

/

layoutlm-base-uncased

like 33

Hello,

In the version 4.38.1 of transformers, I have this issue while loading the model :
Some weights of LayoutLMForSequenceClassification were not initialized from the model checkpoint at microsoft/layoutlm-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight', 'layoutlm.embeddings.word_embeddings.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

Model structure :

LayoutLMEmbeddings(
  (word_embeddings): Embedding(30522, 768, padding_idx=0)
  (position_embeddings): Embedding(512, 768)
  (x_position_embeddings): Embedding(1024, 768)
  (y_position_embeddings): Embedding(1024, 768)
  (h_position_embeddings): Embedding(1024, 768)
  (w_position_embeddings): Embedding(1024, 768)
  (token_type_embeddings): Embedding(2, 768)
  (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
  (dropout): Dropout(p=0.1, inplace=False)
)

Code to reduce issue :

from transformers import LayoutLMForSequenceClassification

model = LayoutLMForSequenceClassification.from_pretrained("microsoft/layoutlm-base-uncased")