Secrecy is crucial in dealing with the Skrull invasion because the Skrulls are slowly and quietly removing humans from the planet. If the Avengers were brought in, it would make this classified matter public, potentially causing panic, aiding the Skrulls' plans, and leading to an even greater loss of life.