This is the best way to sure that the deployed site is working. Defect Tracking Everyone in the company should be involved in noticing problems and generating defect reports. Email is a good choice for originating bug notices because it does not require the installation of any new software that would be a barrier to use. A newsgroup, or any defect tracking software that allows full participation by everyone in the company can be considered if email is insufficient. For a sobering look at test automation see Test Automation Snake Oil by James Bach.
Automation is a great idea.
To make it a good investment, as well, the secret is to think about testing first and automation second. If testing is a means to the end of understanding the quality of the software, automation is just a means to a means. You wouldn't know it from the advertisements, but it's only one of many strategies that support effective software testing. Automate tests where experience running the tests indicates that automation would be worth the effort. Use automated testing tools like Linkbot where possible.