A modded server is just a server that has mods installed. All you need to do is open up your server’s FTP (File transfer protocol).  At the top create a file called “mods.” Now whenever you download a mod, extract the modinfo.xml file to this folder.  Remember however that some mods require your users to install them as well. Also, make sure to take down your server before installing mods.