RevolutionAnalytics/RHadoop — Issue #27: kmeans not reproducible as test case
Author: piccolbo
Created: 2011-11-02 20:56:23+00:00
State: open
Labels: branch-dev, complexity-medium, importance-medium, pkg-rmr, type-improvement

To test the implementation of a backend it would be nice to see that different backends return the same results when running the same program on the same input. In fact I was planning to make that a requirement for new backends. The problem with randomized algorithms is that it isn't trivial to get reproducible results. The normal approach of re-seeding the RNG doesn't extend easily to a parallel framework where the degree of parallelism and the exact input to each of the sequential processes is not controlled by the programmer. So I am excluding kmeans from the cross-backend test cases for now. Ideas welcome.