The judiciary is not the ultimate arbiter of what is lawful; that is congress. Not only can congress simply impeach the entire court, it can also replace almost all articles of the constitution* (and the few that formally cannot be replaced are highly problematic and likely not relevant here anyhow, and congress could do an end-run around those too - formally - by a multistep amendment). * This is of course ignoring states (which would need to side with congress), but much more importantly:

All of this is of course misleading, because far before this point, we'd have guns blazing civil war. When the government is so deeply untrusting of itself - we're talking a level that makes dems and repubs look like best buddies here - what matters is practical power, not formalities (which is also why the constitutional limits on congressional power are unwise). And let's hope we never have to find out what would happen then, because it's unlikely to be pretty.

Frankly: any limits on congress's power (and specifically the house) are hugely problematic, and contribute to side-stepping democracy in favor of other forms of currying favor. There shouldn't be any question about congress's supremacy; these questions should be fought in the court of public opinion via representative democracy, rather than historical happenstance or legal quirks. It's fine to require some high bar for dramatic changes, but it's not fine to have giants like California, Texas and Florida on the same level as Vermont and Wyoming when it comes to real, impactful issues; and just as critically - it's not OK for the constitution to make it all but impossible in practice to be altered even when a large majority in favor can be found. The current system might require lots of different phases including convincing 75% of states, meaning states with just 4% of the population have a veto, and in those you only need a majority - so, say 3% of the population needed for a veto - and additionally people don't get a say directly, so you also need to run the risk of considerable lobbying; since vested interests generally oppose change - that makes it even harder. The final nail in the coffin of change is partisanship; itself a consequence of flaws in how voting works in the US it would need the kind of constitutional amendments to fix that partisanship makes almost impossible - since anything one side propose is invariable a step too far (or not far enough, or somehow a little in the wrong way) for the other side. Unsurprisingly, there have been virtually no amendments since the very early days that actually affect the way the country is run or democratic rights as opposed to publicly impactful but hot air in terms of power stuff like the prohibition. Voters are captured by a system designed to deal with realities of its day, which is by now almost unrecognizably far away, with no route for permanent improvement.

The exceptional amendment that did pass late and influence actual democratic rights (e.g. women's right to vote) kind of proves the point - because the only reason that passed is because by human biology all states have roughly the same proportion; and political parties find it very difficult to use partisanship against them since they would have been in the vast majority of households regardless of which partisan "team" they adhere to.