Like many things in life, the truth is rather less simple. Fluid can be infected. Just looking at the fluid by examining the ear is not sure fire way of telling. Even so, antibiotics have only a very small effect in clearing fluid when there are no symptoms of acute otitis media . This insensitivity to antibiotic treatment likely has to do with the low level of activity of any bacterial infection present.
Antibiotics are most effective in getting rid of bacteria when they are actively growing.
Middle ear fluid can also come from allergy, and persist from enlarged and infected adenoids and sinuses. This seems to be a result of poor ventilation of the ear. Correcting the underlying problem can help the ear ventilation and accelerate resolution of middle ear fluid. The main danger from middle ear fluid is Hearing Loss and Speech Delay.