Flamethrowers wreak havoc on them. Vikings are so cool that they can beat a knight as if he were an unarmored man. Romans only have half armor. They can be used to supplement knights' armies. Terrain Houses can stop bullets, but a hand grenade can blow a door open.
Tanks can knock down any house.
A wood fort can stop bullets and old fashioned cannons, but can be penetrated by grenades, light modern cannons and all larger weapons. Tanks and half tracks can drive through wood forts. The easiest way to beat walls is to fire over them with mortars and grenades. Some stone forts can be breached by tanks. Big cannons can put holes in stone forts, since a stone fort is not the same as a stone wall.