fix AV in debug assertion, address warnings

Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>