Not at all. The modules and shortcodes do not make Divi slow. They barely add to the page weight. Besides, Divi is designed with the best coding structure. The drag and drop modules don't add visible weight to the page loading. However, if you compare it with themes like GeneratePress, it is surely heavier but that's a tradeoff for the functionality you get.