Generalized Rewards Tree Distribution

Governance
1

In this post I’m going to describe a proposal to settle the manner in which the rewards tree is distributed.

Motivation

A bit of history:

  • Originally, rewards trees were posted and pinned by each and every odao seat on IPFS.
  • The IPFS service the oDAO was using more or less made this impossible, and now rewards trees are simply uploaded to github by the core team every interval (manually).
  • While we do not post the trees to IPFS, the oDAO continues to populate the IPFS CID in consensus (a deterministic identifier which, should anyone decide to upload the tree to IPFS, will allow nodes to download it).

We ultimately decided a replacement API for IPFS uploading wasn’t important as:

  • Rewards Trees can’t be tampered with- Smart Node calculates and verifies the merkle root when it parses the tree, and will reject files with an invalid root.
  • You need not get the rewards tree from the oDAO- anyone can generate it themselves
  • You need not generate it yourself- anyone who generates it can provide it to you trustlessly (see point 1).

That said, I do not like using github as a cdn (as it is centralized) and I do not like manually publishing trees.

Proposal

First, the Smart Node maintainers will publish an openapi spec that describes a service that can be used to download reward trees in multiple formats, with or without compression.

Smart Node will implement this spec and allow users to expose a route with this service.

Smart Node will also accept a vector of URLs from which to fetch reward trees, should local generation be disabled.

After a transition period, Rewards Trees will be uploaded to github only for archival purposes, and Smart Node will no longer download trees from github.

Benefits

  1. Anyone can run a service that shares reward trees with other node operators altruistically
    • so long as at least one such node operator runs a service, nodes will be able to claim rewards in timely fashion.
    • RPIP-52 asks the oDAO politely to participate in this process: The Oracle DAO SHOULD upload all four files to a publicly accessible webserver, where individual Node Operators can retrieve them.
  2. Improved robustness against github (ie should microsoft notice that the rewards tree repo is, essentially, parasitic storage).
  3. Anyone who wants to can still publish the tree to IPFS, following the rules established in RPIP-52.
  4. By publishing an API spec we allow non-smart-node implementations of the service as well.

Drawbacks

  • Mostly, development time (though arguably there is a break-even when the manual tree publication process is eliminated).
  • It’s possible that the centralization issue isn’t solved immediately, as there is little motivation to provide such a service except altruism, and once one reliable service is running, there is diminishing returns in running additional ones.

Why is this on the DAO forum?

A few reasons:

  • The IPFS deprecation happened quickly and we never really asked the community / dao how it felt about using github as a stopgap
  • Github was supposed to be temporary and we acknowledged the need for a longer-term solution
  • I will probably be working on this and I am an employee of the pDAO

Vote for up to 2:

  • We should do this
  • We should still try to automate uploads to IPFS, whether or not we do this
  • We should not do this
0 voters
4 Likes
2

I understand/recall that the oDAO didn’t want to host the trees on IPFS themselves because IPFS is peer-to-peer and it would expose their IP addresses. So instead a centralized service was used where you can upload files and they host/pin them on IPFS for you. Their tooling got deprecated and that’s why it was removed from Smart Node.

IPFS still seems like an appropriate protocol for this. Any node operator can generate trees and could upload them to IPFS. oDAO nodes could send the trees to another machine that runs IPFS to protect their privacy. Several IPFS gateways can be used to download files over HTTP (ipfs.io, Cloudflare, etc.).

Alternatively, AWS S3 API could be used for this. Volunteers could set up buckets in AWS S3, Google Cloud Storage, Cloudflare R2, etc. Uploading would use AWS S3 SDK but downloading could be over public HTTP endpoints. Lots of tools already to sync/replicate buckets.

3

S3 is a fine backend but even if smart node can simply use s3:// uris we probably want a standard anyway so we aren’t hitching our horse to amazon.

Keep in mind that even if we publish such a spec we can allow smartnode to support multiple protocol uris, eg

4

I asked a couple of questions and got good answers in discord, so documenting here:

Could folks serve up bad trees (on accident or on purpose) and cause problems?

The Smart Node checks the computed root for the file against the onchain server. If it was a mismatch on the root or the tree doesn’t properly calculate the root, they’d discard that file and get a new one.

Are there any DOS opportunities to consider? Probably of the folks running the service?

This is a real concern, and “we will include language indicating only expert sysadmins should attempt to run a public service”. There will also be “some logic in that prevents it from running on a machine with an odao wallet loaded” to avoid a bigger DOS issue.

5

Hey @Patches, thank you for this.

I believe the smart node already has a way to pull the reward files from multiple endpoints, @fornax can confirm.

From our side, we are going to switch over to using S3 - as credentials are an issue with Github. It was always our intent that others would host the files and are happy to add providers to the list as they come online.

6

Hi, there are few things unclear to me:

  • What is the proposed way to maintain the list of URLs?
  • What is the proposed way for “hosting” operators to get trees?

Also, if peteris is right about oDAO concerns on IP address privacy, then is it ethical to ask node operators to exhibit their IPs?

Asking my own questions, I think that if there is some way for oDAO nodes to upload files on such described services (my main concern here is credentials), then

  • it is preferable to implement a standalone service
  • so anyone can run it home or using any cloud provider
  • and report to pDAO an URL and specs (eg. cloud provider and host instance region)
    • so that smartnode devs or pDAO can compile a list of URLs and specs
      • and encourage new service hosters to use different providers/regions
      • and pay from IMC to service hosters to compensate at least for their expenses
7

Smart Node can have some defaults, otherwise there is explicitly not a proposed way of maintaining a list

They generate them. Or download them from someone else. It doesn’t really matter which.

We wouldn’t be asking them to, only experienced sysadmins / security-knowledge people should create publicly accessible endpoints.

People with the resources to run multiple nodes would be able to, say, generate the tree on their fallback and have their primary download the tree from the fallback, but wouldn’t be asked to create a public service.

This isn’t in the proposal. oDAO operators can choose whether or not to independently run a service which implements the spec, but aren’t required to, and auth/creds won’t be an issue- there is no “upload” route in the proposal, only download.

The answer to the first bullet is “no” in my opinion and I don’t think we need to earmark funds for what should cost a few dollars a month to operate.

This proposal is about specification. I’m not sure what you mean by “standalone service”, but if you have a good spec, you don’t need to worry about the implementation.