This is not a good practice. There are sewer gases in the waste piping system. Normally, water in the traps of the drain pan piping would provide a seal keeping the sewer gases out of the home. However, this only works with water in the traps. When the air conditioning is off for any substantial time, the water in the traps evaporates and the sewer gases can enter the home.
Sewer gases are mostly methane, which is toxic and potentially explosive under the wrong conditions.
Supply ductwork for the air conditioning system should be carefully constructed and sealed to prevent it from leaking the cool air you're paying for.