Fix memory/resource leaks on tree_file_open() failure

Change-Id: I72fc39057458fad80471630d6f772688a95c0e3a
Reviewed-on: http://review.couchbase.org/28630
Reviewed-by: Aaron Miller <apage43@ninjawhale.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>