Well, no! In a hot air balloon, you don't have a direct link with the ground, it's not like when you are on a bridge or a ladder, and thanks to that you won't feel dizzy.
You may be a little apprehensive about the void, but this will dissipate very quickly (I promise!), as the wonderment quickly takes over.
Don't hesitate to tell your pilot about your apprehensions, he will be able to reassure you and make you feel confident.