You can, but it has advantages and disadvantages. Against the grain the razor blade cuts very close to the skin, which can cause irritation and raises the risk of getting cut. Usually going with the grain is the better option.