GSoC 2014: Standard Extreme Learning Machines

Finished implementing the standard extreme learning machines (ELMs). I am getting the following results with 550 hidden neurons against the digits datasets,

Training accuracy using the logistic activation function: 0.999444
Training accuracy using the tanh activation function: 1.000000

Fortunately, this algorithm is much easier to implement and debug than multi-layer perceptron :).
I will push a test file soon.

@ogrisel , @larsmans
I have a problem with saying things like "it is desirable to...".

Here, I guess the point is that you want the batch size to be as large as possible while being small enough to fit. But is this even accurate? Because if X doesn't fit in memory, you're already swapping by the time that mini batches are formed. I guess if the data really doesn't fit in memory, you'd have to use partial_fit and read it from disk in chunks. Am I misunderstanding?