It is very easy to inject bias into market research. But the place where bias is injected in most law firm surveys is in the response vehicle. For example, many firms will ask clients or non-clients to rate the firm on a list of specific traits or characteristics. While this type of analysis is valid, and the results can be quantified, the list of characteristics typically is developed by the lawyers with no client input. So, in effect, the firm is being rated on factors that are important to the lawyers, not the clients.
Some reports have as few as four or five people included in the percentages.
Make up of the sample. If you develop the mailing list for client surveys from billing records, for example, you could have accounts payable personnel providing evaluations of the firm instead of the real decision-makers. Bias can even be injected in the visual reporting of the results. Disproportional graphs or charts or uneven axes can maximize or minimize the actual differences in time and numbers. A poorly written question can render the responses meaningless.