merge method moved to basetable as it is a general enough implementation that works for both flat and database tables