Basically, I was already fit. Like you, I had to cut out good stuff and the weight started to come off. There is no real secret to weight loss. Burn more than you consume. You are right, caffeine can have both a positive and negative ergogenic effect on cyclists.
Dehydration is the most likely adverse effect.
Since you are racing in heat and humidity in Gainesville, I would be careful. If you use energy gels or soft drinks while you race, your caffeine consumption is probably already on the high side. Whatever you decide, always try it in training first. It is tough to answer your question without seeing you on your bike. I am inclined to think that your saddle is too far back, or your stem is too high.