Default shortcuts to speed up initial block downloadDefault shortcuts to speed up initial block download

Turning on a new node can be a real pain in the ass. You have to download and verify all the blocks from genesis...or do you? Currently, all major node implementations ship with some default shortcuts that make it faster to sync with the tip of the chain with the most work. Such shortcuts involve different trust-assumptions. I thought it would be interesting to compare them.

What do these default shortcuts do?What do these default shortcuts do?

assumevalid is the oldest of these shortcuts. It specifies the hash of a block the ancestors of which your block will assume are all valid. Your node will not validate the scripts and signatures of the transactions in these blocks, although it will still check block headers, double spends, and made up inputs. If the assumed valid block is not in the chain being followed, your node will revert back to verifying all scripts and signatures. All blocks after the assumevalid block still have their signatures checked normally.

milestone is Libbitcoin's equivalent to assumevalid, but skips more validation checks. It specifies a block before which all validation except identity checks (block hash, Merkle root, and witness commitment) is skipped. Block headers are always fully validated. While it is the default mode for libbitcoin, the documentation says that milestone sync should only be used if the "user has previously fully validated the chain up to the configured milestone hash." All blocks after the milestone block still have their signatures checked normally.

checkpoint is used by btcd. Prior to the checkpoint block with the highest blockheight, btcd will not perform all checks except those involving comparing the header against the checkpoints. It still performs some "sanity" checks on the blocks before the checkpoint such as ensuring expected proof of work and timestamps that are after the last checkpoint. All blocks after the checkpoint block still have their signatures checked normally.

assumeutreexo is used by Floresta. When a new Floresta node starts up, instead of generating a utxo set, the node uses a merkle tree updated after every block to accumulate references to every unspent transaction output, allowing nodes to skip storing the outputs themselves. The default Floresta mode uses assumeutreexo to accept a snapshot of this merkle tree. Other than ensuring the snapshot is tied to a block in the chain with the most work, no checks on the snapshot are performed. However, Floresta also defaults to a backfill mode similar to assumeutxo which will validate the full chain from genesis over time. All blocks after the snapshot have their signatures checked normally.

assumeutxo accepts a snapshot of the utxo set as generated by some other node as long as it matches the hash hardcoded in the node's software. Core and Knots both hardcode the hashes of snapshots of the utxoset at 840000, 880000, and 910000. Using assumeutxo, your node will check to see if the hash of a snapshot you try to load matches one of the hashes hardcoded in its software. Later, the node will go back and validate the chain from genesis. All blocks after the snapshot have their signatures checked normally. This is an optional mode in Core and Knots, but is not the default.

How are the trust assumptions different?How are the trust assumptions different?

For shortcutted blocks and before:

Note: before backfill/background validation is complete for assumeutreexo/assumeutxo

How bad is a little trust?How bad is a little trust?

I think the main concern with these shortcuts is that someone could lie about portions of the chain history and if the majority of hashpower does not notice it (or accepts it anyway), all nodes using such shortcuts will accept it as truth.

(there is a caveat here: `assumeuxto` and `assumeutreexo` both go back and validate from genesis, so such nodes would notice the problem at some point, but until their backfill function is complete, they could be tricked, too)

None of these shortcuts really put you at an increased risk of a reorganization or increased risk from a 51% attack. Rather, the concern with these shortcuts is that you are trusting the majority of the hashpower to not lie to you. If you don't validate the chain from genesis, you are trusting everyone else that the current state is what they say it is.

We all feel pretty confident that this didn't happen in the past. And It does seem like a small it of trust. However, if the trend of relying on shortcuts continues and few people ever validate from genesis, it is entirely possible that using a shortcut at some future block height could hide someone cheating.

How is this like SPV nodes?How is this like SPV nodes?

I believe, all these shortcuts involve trusting miners not to collude to some extent. In that sense, they start to sound a lot more like Simplified Payment Verification (SPV) nodes:

the verification is reliable as long as honest nodes control the network.

SPV nodes only check to see if a transaction is included in the chain with the most proof of work, not whether it is valid. Which means they have to trust the majority of hashrate that any given transaction is valid.

As we have seen from above, most default node installations have the same trust-assumption for all transactions before their shortcut. The fact that they start validating after the shortcut doesn't seem very valuable: if a block turns out to be invalid, all its descendants are invalid, too -- even if the descendants followed the rules perfectly.

We seem to be okay with this, though, because it is unlikely that invalid transactions would get included in a block without someone noticing or that we'd all just be okay with miners building on top of that block. Yet, if you tell me an SPV node is enough security, I'll probably think you don't know what you're talking about.