You can run mods on your Terraria server, including TModLoader.   You have 5GB of free ram. If your server uses over that, it will shut down until started again. You can upgrade and downgrade your maximum ram at any time.