a corporate hybrid called a prefix L M which is a single well you could have untied weights but you think of it as like a partial and tying of the masking and a self attention matrix where you allow some part of the sequence to do bi-directional attention like in the past and then you switch over at some point to doing auto regressive like language modeling so you can get the benefits potentially of for past contexts doing bidirectional representations or in the limit if your downstream task is always just going to