Yes, you can remove the added file or even the folder if you want by using PDF Merge Tool.