No. Flax only grows in villages. So you have to face the danger of the Fulings to get Flax. One option is to use only stamina foods and run in, gather, and then leave, but that's risky.