Eventually you will reach Foog's Gate, but take a right before you travel through the gate and follow the path around to the Castle Vaults.
The path goes around to the vaults.