I’d like to kick off discussion on Rewards Tree Spec v9. I’d like to see the following changes:
Changes to Specifications
- Rewards merkle tree file format must be fully specified
- It should be a binary format
- It should no longer contain the CID of the minipool performance file
The motivation is:
- Enable the oDAO to run alternative treegen implementations
- Remove the need for compression, which currently ties the resulting CIDs (which are part of the consensus vote) to specific versions of the specific compression library used in smartnode
- Make performance file generation optional (simply reduces specification work)
In more detail, the reason the oDAO cannot currently run independent implementations of treegen is that after building the tree, they must:
- serialize an unspecified performance json file with the exact same bytes as every other oDAO member
- name the file exactly the same as the other oDAO members
- compress that file with the exact same version of this library as other oDAO members
- compute a CID in a manner exactly the same as the other oDAO members
- serialize an unspecified merkle tree json file with the exact same bytes as every other oDAO member, containing the CID from the first file
- repeat steps 3 and 4 with the merkle tree json file
The zstd library used by watchtower specifically warns against this:
[…] the encoding should never be assumed to remain the same, and you should not use hashes of compressed output for similarity checks.
So, instead of using this compression library, we should just remove the need for compression in the first place. In earlier experiments I’ve done, binary formats had roughly the same file size as compressed json, or even smaller, which make them a good option. In keeping with ethereum practices, ssz seems like the obvious format choice.
The process, then, would become:
- Serialize the merkle tree ssz file with the same name as the rest of the oDAO
- Compute the CID
Minipool performance files are not mission-critical and will still be made available, but not necessarily on IPFS.
Additional tooling will be published to convert ssz trees back into json, and json trees will still be usable to claim rewards.
Process Changes
From v9 and onward, tree specifications, both for the generation of the tree, and the format, name, and ipfs tree of the resulting artifacts will be written as RPIPs.
Process note: Past rewards tree versions have required a sentiment vote and (somewhat inconsistently) a full pDAO vote to ratify them. Since part of this proposal is to begin codifying the rewards tree specs with RPIPs, I think we should just do a single poll for the RPIP itself, when ready.
I’m writing this thread now to kick off discussion. A poll will follow when the RPIP is written.