Couldn't assumevalid fail if the majority of hashrate built on an invalid block for long enough that everyone who knew it was invalid gave up and left and then those who remained set a new `assumevalid block at a higher height?

Then any user running a full validation with assumevalid disabled or set to a lower block than the offending would reject the best chain. Hopefully one of them would raise an alarm. Also, if a block had been invalid and people got forked off, how would the grand conspiracy have kept thousands of naturally suspicious and widely distributed Bitcoiners from getting that message out?