Fixes a bug in test optimization

Using max instead of min here made this little optimization useless
as instead of limiting tree size to 5 it made to be at least five.
Trees generated with size more than five tend to take a lot memory
and are slow to generate.