test if binary tree has BST property