Some potential downsides to having a 504 plan include not providing as much support as an Individualized Education Program (IEP), schools not having the resources to effectively implement accommodations, stigma or discrimination, and possible inequity in support compared to students with IEPs. It’s essential to carefully consider these potential downsides and work closely with the school to implement the plan effectively.