Penalties for late filing or non-compliance can include fines, interest, and potential legal action.