It's up to you whether you want to keep this file in the repo, since we'll be going the web store route.  If you do, I suggest adding a notification here that it's generating a new .pem file, and modifying .gitignore to ignore the generated .crx and .pem files.