I also wonder whether is some confusion underlying the use of this massive
amount of data

Keyvan and I are discussing that in a separate, private thread....

The training data for a single composition formula (like A precedes B, B
precedes C, therefore A precedes C) shouldn't need to contain more than
100^3 points or so.

Maybe the 2GB file is used to *create* the training data for each of the
composition formulas, though....  In that case it could be a useful
intermediate step.   But still the regression algorithm would only see a
million triples, for each composition formula it's trying to induce...





On Wed, Dec 11, 2013 at 6:35 AM, Linas Vepstas <notifications@github.com>wrote:

> In opencog/python/spatiotemporal/temporal_events/composition_formulas.py:
>
> > @@ -0,0 +1,73 @@
> > +from spatiotemporal.temporal_events.formulas import TemporalRelation
> > +from spatiotemporal.temporal_events.trapezium import generate_random_events
> > +import os
> > +import csv
> > +import numpy as np
> > +import ffx
> > +
> > +__author__ = 'keyvan'
> > +
> > +
> > +def create_sample_file(number_of_events=100):
>
> Yikes! Well, that's why ffx is slow. 2.4 GB is pretty outrageous. Looking
> at the code, it looks like you generate 100^3 == 1 million events, each of
> which is 2.4 KBytes. that seems wrong -- a single even should not use
> 2.4KB, since a single event is just four float pt numbers. And generating a
> million ... seems excessive. I don't understand what the point of this is,
> but its certainly rather far from a minimal solution.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/opencog/opencog/pull/428/files#r8250597>
> .
>



-- 
Ben Goertzel, PhD
http://goertzel.org

"In an insane world, the sane man must appear to be insane". -- Capt. James
T. Kirk