I've also made a video on Bert what you do in Bert is you simply take that image and you cross a block out two of the pixels or many of the pixels and you simply ask your network to reconstruct those pixels okay and now you can see the attention flows in all direction birth the B stands actually for a bi-directional so this is the contrast to the autoregressive pre training framework now the these two things have been applied in text both the autoregressive is usually it's easier to actually make it produce something like