These fungi are usually white and stringy. Ferguson explains that when air does get into the bag, harmless fungi can develop. Actinomycetes is another fungus that is stringy and white. Composted manure is a great amendment and nutrient booster. But make sure the manure you use is completely composted.
Raw manure is high in salts that stunt growth.
I suggest that you consider ordering a few cubic yards of a soil mix from a soil yard to build your beds rather than using bagged potting soils. The blends will provide needed nutrients and the correct pH balance for the plants you wish to grow. And they vary in price. The better ones are generally more expensive. I think delivered soil would save you considerable money if you are going to need soil for large areas.