Prevent duplicate loading of plugins

@sersut Leaving some comments below with explanations/concerns