Is the LayoutLM working now?
Author: shaonanqinghuaizongshishi
Pull request: no

Getting endless erros when trying to use the LayoutLMForTokenClassification from transformers for NER task, is just me doing wrong or the function still on work?
Really appreciate if anyone can give some information.

Comments:
Hi @shaonanqinghuaizongshishi 

Could you please post the code snippet, stack trace and your env info so that we can take a look ?

I am working on:
ubuntu 16.04
torch 1.5.0
transformers 3.4.0


```
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
tokenizer = LayoutLMTokenizer.from_pretrained(model_path)
model = LayoutLMForTokenClassification.from_pretrained(model_path, num_labels=config.num_labels).to(device)
outputs = model(b_input_ids, bbox=b_boxes, token_type_ids=None,
                            attention_mask=b_input_mask, labels=b_labels)
```


Then I run CUDA_LAUNCH_BLOCKING=1 python layoutLM.py, and got the following error:

```
Traceback (most recent call last):
  File "layoutLM.py", line 275, in <module>
    train(train_dataloader, validation_dataloader)
  File "layoutLM.py", line 162, in train
    attention_mask=b_input_mask, labels=b_labels)
  File "/Classification/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Classification/lib/python3.6/site-packages/transformers/modeling_layoutlm.py", line 864, in forward
    return_dict=return_dict,
  File "/Classification/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Classification/lib/python3.6/site-packages/transformers/modeling_layoutlm.py", line 701, in forward
    inputs_embeds=inputs_embeds,
  File "/Classification/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Classification/lib/python3.6/site-packages/transformers/modeling_layoutlm.py", line 118, in forward
    + token_type_embeddings
RuntimeError: CUDA error: an illegal memory access was encountered
```



Hi there!

I have been investigating the model by making [integration tests](https://github.com/NielsRogge/transformers/blob/e5431da34ab2d03d6114303f18fd70192c880913/tests/test_modeling_layoutlm.py#L318), and turns out it outputs the same tensors as the original repository on the same input data, so there are no issues (tested this both for the base model - `LayoutLMModel` as well as the models with heads on top - `LayoutLMForTokenClassification` and `LayoutLMForSequenceClassification`).

However, the model is poorly documented in my opinion, I needed to first look at the original repository to understand everything. I made a demo notebook that showcases how to fine-tune HuggingFace's `LayoutLMForTokenClassification` on the FUNSD dataset (a sequence labeling task): https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LayoutLM/Fine_tuning_LayoutLMForTokenClassification_on_FUNSD.ipynb

Let me know if this helps you!


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.