Search similar

My plan for conditional groups was just to apply the clustering algorithm
to the conditional distribution.

This would involve drawing conditional samples and scoring them against the
conditional conditional. In principle, I think this approach also works
against the original dataset if  we apply filters to ensure we get rows
which match our conditions, but we run into problems of limited data.

On Thu, Oct 30, 2014 at 10:33 AM, Fritz Obermeyer <notifications@github.com>
wrote:

> In loom/preql.py:
>
> >              external_id = self.rowid_map[row_id]
> >              writer.writerow((external_id, score))
> >
> > +    def cluster(
> > +            self,
> > +            rows_to_cluster=None,
> > +            seed_rows=None,
> > +            cluster_count=None,
> > +            nearest_neighbors=10):
> > +        if seed_rows is None:
> > +            seed_rows = self._query_server.sample(
>
> I am wary of generating samples from the model instead of subsampling the
> original dataset. Do you think there is some equivalence theorem? Note that
> since we have already shuffled the rows, it is cheap to subsample: simply
> grab the first n rows of samples/sample.0/shuffled.pbs.gz
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/priorknowledge/loom/pull/87/files#r19621437>.
>