If enough mold is present in the crawlspace, then yes. Due to the "stack effect," moldy air rises and permeates the upstairs. If you've ever been in an old home, perhaps you noticed a musty smell in the air. This likely was mold that ascended into the upper levels of the house. While the smell is unpleasant enough, it's also unhealthy for people with asthma, allergies, and respiratory conditions.
If a crawlspace is moldy, it's probably filled with pests. As these critters die in the area, they create odors that mingle with the mold. This creates an unsanitary and unhealthy living environment.