The logs and data streams are numerous, large, and sport integration challenges specific to the Web environment. For example, the clocks of all the servers may not always be fully synchronized. And, a single user in a single session will typically interact with many separate servers. It is essential to tie those separate interactions together in the right order to get a clear picture of the behavior of any one visitor. These are challenging integration problems, compounded by the immense size of the logs and the pressure in the world of e-business to do everything quickly.
Users are often anonymous.
Until users make the decision to register and begin visiting regularly under registered identities, they are anonymous. At the same time, the behavior patterns that result in converting anonymous users to voluntarily registered users are critically important to the successful Web site. Thus, it is important whenever possible to analyze data associated with anonymous as well as registered users and to make the most of that information. Dealing with the complexity and uncertainty of data about the large number of anonymous visitors to a major Web site adds an additional dimension to the analysis. Data must be mapped and aggregated from the level at which it is initially logged to a level at which it can be meaningfully interpreted.