jmdeldin/cross_validation — pull request #1 by ronandi (2013-11-27 22:25:48+00:00)
Title: Fix inconsistent k value and exclusion logic

Hi,

Thanks so much for writing this gem. I've been using it for one of my projects, and it has saved me a lot of time. The source is very well documented.

I think I may have found a couple errors:

- Percentages and folds lead to different k values, and regardless of which method is used, they are partitioned the same way. The partitioner subset method creates partitions of size k for ary, instead of creating k partitions. When using percentages, this is fine because you've included logic that calculates the correct size, and thus produces the right number of partitions. This is not the case if using folds. For example, in this case, instead of creating 10 folds (10 partitions), you create MANY partitions of size 10.

- Reading up on k-fold cross validation, I found that the partitions should be random, so I've shuffled the documents array before partitioning to give this effect. This addition causes a test to fail. I'm not sure if you'll agree with the randomization, so I haven't taken the time to fix the test. I'm a bit...lazy :D

-I think there is a small error in excluding a fold/partition. The exclude_index method was previously passed the documents array. This would mean that it simply dropped one document, instead of one partition.

Other than the test I mentioned, nothing else fails.

-Regards,
-Rohit