vector per token and he would simply concatenate the two from the from so from here is this leather stain on the left produces one disaster the right produces maybe here another one and you simply concatenate the two to get the the final embedding the final word vector for each token so the fundamental limitation here is that this is kind of you have information from the left end you have entered information from the right so other than here the original transformer you actually have you actually can condition on the left context and the