On Apr 25, 2013, at 2:07 PM, Barnaby Rowe <notifications@github.com> wrote:

> In devel/modules/test_comparison_basic.py:
> 
> > +
> > +    # Try an eight core run setting
> > +    res8 = galsim.utilities.compare_dft_vs_photon_config(
> > +        config, random_seed=rseed, size=imsize, pixel_scale=dx,
> > +        abs_tol_ellip=tol_ellip, abs_tol_size=tol_size, n_photons_per_trial=np,
> > +        wmult=wmult, nproc=8, logger=logger)
> > +
> > +    print res8
> > +    return
> > +
> > +
> > +if __name__ == "__main__":
> > +
> > +    for n_photons in n_photons_test:
> > +        test_comparison_config(n_photons)
> > +        test_comparison_object(n_photons)
> You don't get identical results, because the object version uses only one random generator sequence to generate all the photons, whereas the config version uses a number of differently seeded random number generators, one for each image.
> 
> So, over the last couple of days I've been talking with a colleague (Sree Balan, who wrote the G08/G10 code) about how we currently do the random number generation, and he has got me spooked about it once again. He's on github so I think I might open an issue to have the discussion there and get him involved: he's sent me a bit of material about what we might do instead of using lots of differently seeded generators...
> 
This has always worried me slightly.

> In summary, the problem that Sree raises is that RNGs generally only guarantee that their numbers act like IID RVs from a single sequence, they don't guarantee much about how the sequence is initialized from their seeds. By seeding using consecutive seeds as we do we might in fact be getting a whole load of numbers from similar locations in the overall sequence, and so our results will be statistically correlated. As we can't guarantee that different seeds in the same RNG produce statistically uncorrelated RVs, only that a single RNG approximates an IID sequence, there are maybe reasons to be worried... I'll open an issue for the discussion, it would also be good to get Gary in on this as well as Sree.
> 
Seems to me that we should look at the boost documentation.  We don't care about any old RNG, we care about the boost ones, right?  So I just did, and found some interesting statements like "Pseudo-random number generators should not be constructed (initialized) frequently during program execution, for two reasons. First, initialization requires full initialization of the internal state of the generator. Thus, generators with a lot of internal state (see below) are costly to initialize. Second, initialization always requires some value used as a "seed" for the generated sequence. It is usually difficult to obtain several good seed values. For example, one method to obtain a seed is to determine the current time at the highest resolution available, e.g. microseconds or nanoseconds. When the pseudo-random number generator is initialized again with the then-current time as the seed, it is likely that this is at a near-constant (non-random) distance from the time given as the seed for first initialization. The distance could even be zero if the resolution of the clock is low, thus the generator re-iterates the same sequence of random numbers. For some applications, this is inappropriate."

(http://www.boost.org/doc/libs/1_49_0/doc/html/boost_random/reference.html)

Also, can we not carry out a simple experiment to test for independence of random number sequences initialized in the way that we sometimes do it with consecutive seeds?