Don’t use caulk as a substitute for mortar. It’s only for moving joints, namely the places where different materials come together, like the junction of wood window moldings and brickwork or where concrete meets a brick.