You have a few options on how to approach this problem. The first solution is to give yourself more time with this man and his smallish penis. Even if you've had great sex with other people, it's likely that the first few times you try to have sex won't go that well. From your letter it sounds like you've only had sex once. Just because he came doesn't mean he had a great time.
Guys ejaculate in their sleep.
Have sex again and see how it goes. Generally, the more you get to know a person sexually, the more interesting the sex becomes. The second option is to deal with it and be thankful that you have a loving relationship. Great sex involves more than just a big penis. I understand that you'd like him to have a larger penis, but that won't always be the answer to your problems.