Because it is vigorous, horseradish does have a tendency to take over and can be hard to remove once established. They can grow again from tiny bits of root left in the ground, and have a reputation for being invasive plants if left unchecked, but can be kept in line if grown in a raised bed or container. As an alternative to horseradish, Amateur Gardening’s organic expert Bob Flowerdew recommends growing relative spoonwort or ‘scurvy grass’ (Cochlearia officinalis), which has a similar taste.