sampler addded with tests cases and documentation.

Added support for kernels: 
 * Radial Basis Function Kernel (RBFSampler)
 * Additive Chi Squared Kernel (AdditiveChi2Sampler)
 * Skewed Chi Squared Kernel (SkewedChi2Sampler)
I add that csr_matrix for compatibility. For example, it will read in sparse matrix and if the option is to use dense. It should convert to dense from the sparse matrix.

If I didn’t add the csr_matrix, it will give me an error (numpy.ndarray' object has no attribute 'todense') [trying to conver a numpy array to dense]

Do you prefer that I search another way to solve this problem?

Thanks

From: Dan Blanchard [mailto:notifications@github.com]
Sent: Wednesday, July 02, 2014 3:25 PM
To: EducationalTestingService/skll
Cc: Murrugarra Llerena, Nils E
Subject: Re: [skll] sampler addded with tests cases and documentation. (#124)


In skll/learner.py:

> @@ -982,6 +1008,19 @@ def predict(self, examples, prediction_prefix=None, append=False,

>          # filter features based on those selected from training set

>          xtest = self.feat_selector.transform(xtest)

>

> +        # Sampler

> +        if self.sampler:

> +            logger = logging.getLogger(__name__)

> +            logger.warning('Sampler converts sparse matrix to dense')

> +            if isinstance(self.sampler, SkewedChi2Sampler):

> +                logger = logging.getLogger(__name__)

> +                logger.warning('SkewedChi2Sampler uses a dense matrix')

> +                xtest = self.sampler.fit_transform(xtest.todense())

> +                xtest = csr_matrix(xtest)

I like the idea behind trying to turn xtest and xtrain back into sparse matrices, but I'm not sure if it's necessary, because the main reason we want to use sparse is for cases where the data set won't fit into memory at all when it's dense, so if fit_transform didn't run out of memory, we're probably okay.

And because this is Python, it's possible/likely that the old dense version won't get garbage collected right away, so making a sparse copy of the matrix will actually probably use more memory in the long run.

Anyway, my point is, you can remove all of your calls to csr_matrix.

—
Reply to this email directly or view it on GitHub<https://github.com/EducationalTestingService/skll/pull/124/files#r14478012>.

________________________________

This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.


Thank you for your compliance.

________________________________