never seen a word before like self attention you can't assign zero probability to that if you're trying to optimize for like log loss or something because you just get an infinite loss and you know if we just start going to longer and longer strings this count method explodes and the observances of every substring get rare and rare and this just kind of hits a wall so in the like 80s and 90s the way we kind of handle this is we kind of accepted that we couldn't handle the longer term dependencies here and we kind of use