Yes! It is relatively simple to remove the Safari theme and revert your Firefox installation back to its default settings. First, go back to the White Sur directory that you extracted. From there, you then need to type the following command: ./tweaks.sh -f -r && ./install.sh -u.
Doing that will allow you to remove all the Safari-related files for Firefox as well as delete all theme files from your machine. With that done, the last thing that you need to do is to restart the browser to reload your default settings.