Baby Lox generally takes a few in-game days to grow up. You can leave the area and they will grow, but be sure to leave food for them and their parents. There is no definitive time that it takes a single Lox to reach adulthood.