If you say it first, he'll think he's got it made with you and begin to take you for granted. So the best thing you could do is to hang in there, make him very happy when he's with you, and he'll be drawn to you. Men use the L word to cement the relationship. If you cement it first, he won't have to. I'd say go with the flow, especially if the relationship is improving all the time.
Relationships are like business deals.
They either get better and better and become good for both parties, or they spiral downward and fall apart. As long as yours is getting better, you're on the right track. After all, he has already told you he's open to someday marrying. Don't worry that he hasn't brought the subject up again. Men think if they talk about it, they have to do it.