This year, the call will be distributed by electronic means and by word of mouth. In subsequent years, the call for challenges will be integrated with regular call for papers. Winter of 1997 Submitted challenge papers are reviewed by members of the program committee. Reviewing may require examination of on-line sources such as files encoding problem instances or code used to generate instances, as well as web pages providing detailed evaluation criteria. Challenge papers may be featured in a special track or forum.
Work progresses with organization and feedback from the challenger.
At this point, the challenger may want to solicit papers that address aspects of the challenge that merit particular emphasis given the progress to date. It is perhaps impossible to state exactly what we are looking for in terms of technical challenges. The hope is that submissions that issue challenges will address a felt need, exploit or expose an opportunity, or rally the community to tackle a problem that might be difficult for an individual researcher or lab. As a concrete example, an individual or consortium might suggest that progress in learning Bayesian networks would profit from running competing algorithms on a suite of test networks and gathering statistics based on particular performance criteria. There might be an accompanying scientific rationale or hypothesis that algorithms will differ on two or more orthogonal measures that are sensitive to specific properties of the networks.