You are not his boss and you don't get to say whether he keeps old love letters or not. Some things are meant to be private and that includes the sentimental souvenirs a person wants to keep. Old reminders are a person's history. Of course not. Of course not.
Memories are just that.
Your husband's past, including all the women he dated before you, is what molded him into the person you love. He's yours now. She's just a face on a picture and a signature on a letter that reminds him of a relationship he once had. Believe me, if that's the biggest upset you ever have, you'll be a lucky bride. I hope he has some letters from you too in his memory box.