There are a few ways to chisel out a rivet. If the rivet is buried deeply in the surface, you can use a standard chisel to remove it. If the rivet is less deep, you can use a special rivet chisel that has a thin, elongated blade.