Not all inappropriate behavior is actionable harassment. To be sexual harassment, it must be sexual in nature. A ‘dumb blonde’ joke maybe sexist, but not necessarily sexual, for example.
Repeated jokes that are not sexual but target or demean women, minorities, or people in other protected classes could also support a hostile work environment claim based on discrimination, even if they are not sexual harassment.
There is also a difference between one off-color joke and a pattern of making them. If a co-worker tells an offensive joke, but then stops after being told it was offensive, that is not likely to constitute harassment. If the behavior continues, or if it is a regular practice among multiple co-workers, it is more likely that it rises to the level of harassment.