I don't see where Justice Brennan is arguing that the need for quick action trumps the framework in that excerpt.
I see that as a green light for Congress to do pretty much what it pleases--Brennen's statement captures the idea that we shouldn't let the constitution impede progress.