Deadlines are the killers. Applications for grants and contracts and user requests that don't allow for the time needed to get the information out. New software can be tough. User training can be a challenge. That can ease the pain at times.
Communication seems to be the toughest part of any job.
Knowing what people mean as opposed to what they say. Knowing why they say what they say and what they feel is really important is all very tough. Totally unrealistic expectations can be a problem. Which is a subset of the communication challenge. Our host organization is a college campus.