Fix UBSAN error calculating maxNodeCount

* should use an explicitly uint64 constant
* simplest is to use ULL suffix
  (LLU is apparently standards compliant but not always supported)
* this came up because the nabor R package wrapping libnabo threw a UBSAN error
  for points with dimension = 1
* closes https://github.com/ethz-asl/libnabo/issues/16