Repository: iree-org/iree

Issue #9716: Numeric issue in LayoutLm on CPU
Created: 2022-07-06T04:21:39Z
What happened?
The output from IREE CPU and golden values mismatch.
Steps to reproduce your issue
Download the artifacts from https://storage.googleapis.com/iree-shared-files/nod-perf/hanchung/layoutlm_repro.zip
$ iree-compile --iree-mlir-to-vm-bytecode-module --iree-input-type=mhlo --iree-hal-target-backends=dylib-llvm-aot --iree-llvm-target-triple=x86_64-linux-gnu --iree-llvm-target-cpu-features=host core-input.mlir -o /tmp/a.vmfb
$ iree-run-module --module_file=/tmp/a.vmfb --device=local-sync --entry_function=forward --function_input=@input1.npy --function_input=@input2.npy --print_max_element_count=1073741824

What component(s) does this issue relate to?
Compiler
Version information
No response
Additional context
No response

Duplicate issue #9536: numerics issue with bert on cpu 
Created: 2022-06-16T23:35:16Z
What happened?
When running bert-based-uncased we seemed to be hitting a numerics issue:
https://github.com/nod-ai/SHARK/blob/main/tank/tf/hf_masked_lm/bert-base-uncased_test.py
IREE_SAVE_TEMPS  is at iree-shared-files/nod-perf/anush/iree_save_temps_bert_cpu
https://storage.googleapis.com/iree-shared-files/nod-perf/anush/iree_save_temps_bert_cpu/core-input.mlir
https://storage.googleapis.com/iree-shared-files/nod-perf/anush/iree_save_temps_bert_cpu/core-command-line.txt
E       assert True == False                                                                                                          E        +  where False = compare_tensors_tf(<tf.Tensor: shape=(1, 16, 30522), dtype=float32, numpy=\narray([[[ -7.290994 ,  -7.257680
4,  -7.2415266, ...,  -6.464171 ,\n          -6.3598876,  -4.309055 ],\n        [ -8.907964 ,  -8.83476  ,  -8.829174 , ...,  -8.76890
6 ,\n          -8.046408 ,  -6.274063 ],\n        [-14.354682 , -14.289836 , -14.320234 , ..., -13.399221 ,\n         -10.088572 , -11
.495969 ],\n        ...,\n        [ -8.209971 ,  -8.197388 ,  -8.366645 , ...,  -7.885937 ,\n          -8.4152775,  -3.0067217],\n        [ -7.3771143,  -7.7098002,  -7.6014137, ...,  -8.009481 ,\n          -7.0031247,  -5.9178457],\n        [ -5.960579 ,  -6.303993 ,  -6.0986414, ...,  -6.343738 ,\n          -5.770889 ,  -4.7784796]]], dtype=float32)>, array([[[ -7.291651 ,  -7.2583814,  -7.2421713
, ...,  -6.464848 ,\n          -6.3605504,  -4.309505 ],\n        [ -8.909623 ,  -8.8363695,  -8.830784 , ...,  -8.770344 ,\n         
 -8.047972 ,  -6.2748594],\n        [-14.355753 , -14.291211 , -14.321319 , ..., -13.400625 ,\n         -10.089375 , -11.49576  ],\n  
      ...,\n        [ -8.208423 ,  -8.195899 ,  -8.3648815, ...,  -7.8849854,\n          -8.413467 ,  -3.008876 ],\n        [ -7.37734
75,  -7.7098885,  -7.601365 , ...,  -8.009851 ,\n          -7.0026016,  -5.9188967],\n        [ -5.9612074,  -6.304814 ,  -6.09935  , 
...,  -6.3439813,\n          -5.772274 ,  -4.775791 ]]], dtype=float32))

Steps to reproduce your issue

Go to '...'
Click on '....'
Scroll down to '....'
See error

What component(s) does this issue relate to?
No response
Version information
No response
Additional context
No response

Closing comment (2022-07-12T06:37:20Z): One of the models still seems to have some correctness issue, that also probably needs to be tracked by a separate bug.

It's been tracked in #9716