Dealers rely mostly on experience. Do they have a number of customers looking for that book, or have other dealers been looking for that book. How much does the condition of this copy vary from the other copies the dealer has seen. Dealers gain experience by actually pricing books and trying to sell them. Remember, you must compare condition as well as price.
Price guides can also help establish the value of a book.
Before using a price guide, read the introduction or preface which describes how the information was assembled. For example, some price guides are compilations of dealer's catalogs, so the prices are retail asking prices not necessarily prices from actual sales. Some price quides are compiled from auction results. Again you must be careful because the guide may include books which were not sold because the bids did not exceed the reserve price. There are both general stock and specialty price guides.