Furthermore, if implemented properly a robot can actually reduce traffic on the Web. The uses for robots in libraries are obvious. As the Internet continues to grow, demand for information from it will also grow. The structure of the Internet no longer lends itself to browsing. With so much information available a way to find it is necessary.
Robot generated databases are great tools for librarians.
However, It is important that they be aware that search engines are not all the same. Each is indexed differently and provides different information. Furthermore, they are not the only types of databases and search engines available on the Internet. Some are maintained by humans through submissions of URLs. Users with little experience searching will require the assistance of librarians in finding the information they require.