The files you remove from Quick Access will stay intact in their original location because quick access only shows the file shortcut.