lib/rbtree.c: Initialize num_nodes when creating trees

Without this patch, tests sometimes fail due to the value
being uninitialized but used when attempting to create hash
tables to store objects, which then can become ridiculously
large (like, with several billion entries), causing some
malloc() or other to fail horribly.

Signed-off-by: Andreas Ericsson <ae@op5.se>