rpm: re-enable the db format, fix build with gcc

Make sure to link with db-5.3 instead of (the default) db-6.0.
Also account for the C++ experiments affecting the bin linkage.

Make the file magic path shorter, put db/sqlite next to another.

Closes #24272.

Signed-off-by: Mike McQuaid <mike@mikemcquaid.com>