A Candidate Design for the Smoothing Pool

Node Operator Experience
1

Note: This is typically the kind of thing I would write up in the research repository, but I think this is a better place for it since we can have a direct conversation around it.

What is the Smoothing Pool?

When we first shared our post-merge plans with the world, they stirred up quite a bit of conversation. In general the feedback was varied - we heard praise for our forward-thinking development mindset, concern over additional Oracle DAO responsibilities, arguments over the dynamics and math behind MEV, and even interest in the gamification potential that could be attached to the protocol. One thing that was almost universally lauded, however, was the Smoothing Pool.

In a nutshell, the Smoothing Pool emulates the behavior that POW mining pools employ today: node operators would assign it as their fee recipient (the address that priority fees and MEV are sent to), and all of those rewards would be pooled together. There would be some mechanism to determine each node operator’s “share”, and they would be paid their share at some period in time. The intent is to “smooth” the extreme variance of priority fees and MEV by giving all users an equal share of all of those fees. As recent data has shown, these kinds of fees tend to be extreme during things such as NFT launches or major macroeconomic events. Since block proposers have very little time to react to being given an assigment (between 0 and 6 minutes), being a proposer for one of these high-value blocks is akin to winning a small lottery. The Smoothing Pool combats this randomness and essentially promises everyone a reliable, consistent reward structure.

The Smoothing Pool’s popularity among node operators is well established. We have already heard from many users who are currently solo staking today and frustrated by this variance that when Shanghai arrived, they would switch their nodes to Rocket Pool minipools just to gain access to the Smoothing Pool. I submit that a proper and timely implementation of it would be a strong attractant for the protocol in a time where node operators are in extremely high demand.

There are a few challenges in building such a system: determining what each participant’s “fair share” is, delivery of rewards, management of ingress / egress, management of defection, and minimizing the need for trust. Marceau wrote an excellent post that detailed some of these challenges, and stated that they would have to be surmounted for the Smoothing Pool to be viable. In this post, I will offer a candidate design for the Smoothing Pool that provides all of above features while addressing all of those concerns.

There’s a TLDR at the end for people that don’t want to read a book.

Before I begin, I want to give a shout out to Butta and Marceau for helping me come up with some of the ideas in this design and workshop it in advance. Thanks guys!

Assumptions

My design is predicated on the following assumptions:

  • Post-merge, Rocket Pool will employ a Trusted Block Builder system where our node operators are given entire blocks to propose from a verifiable third party. That is, node operators cannot assemble their own blocks. This is for two reasons:
    1. It is too difficult to tell whether or not they extracted their own MEV and thus stole rewards from the staking pool if they build their own block
    2. It enforces the acquisition and distribution of MEV, increasing the overall returns and competitiveness of the protocol
  • The new Merkle Tree-based RPL rewards distributon system (affectionately known as Garlic Bread) is in place and successful

High-Level Behavior

The intended behavior of a functioning Smoothing Pool would work as follows:

  • The smoothing pool would be a new smart contract that stores ETH
  • Node operators would “opt in” to the Smoothing Pool via a Smart Node command, which would:
    1. Set their fee recipient address to the Smoothing Pool contract instead of their node’s distributor address so all of their priority fees and MEV would be sent to the pool instead
    2. Set a flag in the contracts, indicating that they joined it and record the timestamp of when they joined
  • There would be a “rewards checkpoint” at regular intervals, analogous to the RPL rewards checkpoint (it could even be the same event if desirable)
  • At a rewards checkpoint, the Oracle DAO would distribute each node operator’s “fair share” of the accumulated ETH
  • The process repeats ad infinitum

Implementation

In this design, the Smoothing Pool would be an extension of the Merkle Tree distribution system that Kane has already built for RPL rewards. Instead of managing newly-minted RPL, however, this would manage ETH received from priority fees and MEV. In this way, all of the functional concerns are essentially delegated to a single event: the rewards calculation and distribution. Here is my design for what happens at a reward checkpoint.

The following would be done by the Oracle DAO (the watchtower) during a checkpoint:

Determine Total Rewards

  1. Get the initial balance of the Smoothing Pool at the beginning of this interval, which is recorded in rocketStorage. Call this initial_pool_balance.
  2. Get the current balance of the Smoothing Pool at the time of the rewards checkpoint. Call this current_pool_balance.
  3. Calculate interval_rewards, the total amount of ETH to be distributed, via current_pool_balance - initial_pool_balance.

Calculate the Eligibility Interval per Minipool

  1. Look at the epoch where this interval started, and call the first slot of it interval_start.
  2. Look at the latest finalized epoch on the Beacon Chain, and call the last slot of it interval_end.
  3. For each node, the Oracle DAO would find the “opt-in” time (call it node_start) and the “opt-out” time (call it node_end). Nodes that did not opt in (or opted out prior to the start of the round) would be ignored.
    1. If node_start happened before the start of this interval, set it to interval_start.
    2. If node_end didn’t happen (because they didn’t exit it), set node_end to interval_end.
  4. Look at node_start and node_end to determine if the node was eligible at any point during this round. If so, add this node to an eligible_nodes list.

Detect Defection

  1. For each node in eligible_nodes, go through each minipool and look at each proposal they sent during their eligibility window.
  2. If the fee_recipient is not set to the smoothing pool, they cheated. Remove them from eligible_nodes and penalize their Beacon balance accordingly.
  3. If the block didn’t come from a Trusted Block Builder, they cheated. Remove them from eligible_nodes and penalize their Beacon balance accordingly.

Calculate Node Share vs. Pool Staker Share

  1. Calculate the average fee of all minipools in all remaining nodes in eligible_nodes. Call it node_share.
  2. Calculate the amount of rewards to send to the staking pool (half of interval_rewards, minus the node_share portion). Call it staking_pool_rewards.
  3. The remaining balance is the total_node_rewards.

Calculate the Share per Node Operator

  1. For each minipool attached to each eligible node, look at every attestation on the Beacon Chain from node_start to node_end.
    1. For each one that was successfully executed, add 1 to a running total called successful_attestations.
    2. Divide the running total by the total number of epochs. Call this ratio participation_rate. This makes attestations on the Beacon Chain analogous to “shares” in a PPS-style mining pool for POW.
    3. Multiply participation_rate by (node_end - node_start) / (interval_end - interval_start) and call it reward_share. This is how much ETH from the Smoothing Pool should be given to this minipool.
  2. Add up all of the reward_shares for a node’s minipools. The combined total is the node_reward_share, which determines how much ETH goes to the node.
  3. Multiply the node_rewards_share of each node by total_node_rewards to determine how much ETH that node should receive - the node_rewards.

Distribution of Funds

  1. Generate a Merkle Tree containing the node_rewards for each node and the staking_pool_rewards, along with any other important ancillary information about the round.
  2. Upload the tree to IPFS, the same way the new RPL rewards system does it. Inform the Rocket Pool contracts of the resulting Merkle Root and IPFS link.
  3. When 51% of the Oracle DAO members come to the same conclusion and upload an identical Merkle Tree, the contracts will canonize the results (as RPL rewards will be done). The staking_pool_rewards are sent to the staking pool, and the total_node_rewards are sent to a distribution contract (as RPL rewards will be done). Record the balance of the pool after distribution so it can be used as the initial_pool_balance for the next interval.
  4. Node operators are free to claim their Smoothing Pool rewards at their convenience.

Advantages of this Design

This design mitigates all of the potential complications the community has brought forth. I will address them topic-by-topic below.

Node operators are less-accountable for their performance, and could flake out while still earning unfair rewards.

  • This is mitigated using the participation_rate calculation. By using attestations as a gauge for uptime, we can accurately calculate a node’s fair share; downtime will not be rewarded in this system. Node operators are incentivized to maximize uptime.

Smoothing will increase the profitability of tip/MEV thefts.

  • By using the interval-based rewards system, it is easy to detect when a user steals from the system and simply remove any smoothed rewards they would otherwise receive for that entire interval. They would gain no benefit in cheating within the Smoothing Pool as opposed to outside of it.

Smart contract risk, gas costs, technical overhead.

  • By piggybacking off of the existing Merkle Tree distribution system for RPL rewards, there is very, very little to be added for Smoothing Pool functionality. The risk is deferred almost entirely to that system which will be employed regardless of the Smoothing Pool’s existence.
  • Gas costs for individual claims can be high if the smoothing pool generates fairly low rewards or the node operator has a fairly low total share. However, as it is based on the Merkle Distributor system, users simply have the option of waiting to claim (even multiple intervals at once) until they are comfortable doing so. Furthermore, they could potentially even assign an L2 as their network of choice for smoothed rewards - this functionality is already in place for RPL rewards, so doing it for ETH rewards in this fashion should be trivial.
  • The technical overhead comes primarily from the generation of the Merkle Tree, which was already described in detail above. It’s not particularly daunting in my opinion.

Economical viability in terms of MEV.

  • As the Smoothing Pool would require the use of a Trusted Block Builder which would provide MEV, the node operator will always do better than simply building their own blocks.

Opportunity cost in terms of developer time.

  • Since this system largely derives from an existing system, I expect developer time to be quite reasonable for this approach. Not zero, but quite reasonable. I think it will be short enough to pay for itself in terms of Quality-of-Life improvements for node operators and further attract / incentivize attention from prospective candidates… especially if deployed before Shanghai, so solo stakers have a chance to see it in action.

Indifference to Pool Size

  • This design for the Smoothing Pool is effective regardless of how many node operators or minipools opt into it. If more pools come in, node operators have a smaller reward_share but the amount of rewards in the pool will increase and these two factors are expected to even out. Thus, even large node operators would be welcome to join - and even encourage to do so because they would decrease the pool’s susceptibility to variance and provide an even “smoother” experience for everyone.

Disadvantages of this Design

Oracle DAO Dependency

  • As with RPL rewards and withdrawability, this depends on the Oracle DAO to act in a trusted capacity. There is contention around adding another layer of responsibility to the Oracle DAO, but I believe that this is mitigated by making the Smoothing Pool opt-in rather than opt-out. Users have the freedom to use it or not based on their confidence and level of trust in the Oracle DAO.

Gas Costs

  • As this system is a pull rather than a push configuration, the user must pay a transaction fee to claim their rewards. For small node operators, this can be prohibitively expensive relative to the rewards they would earn. However, I offer the following as counterpoints:
    • The Merkle distributor is already quite cheap according to Kane’s calculations, taking about 55k gas to claim for one interval and 55k + 15k for each additional interval if claiming for several at once.
    • If the gas cost to claim such a thing is prohibitively expensive, I submit that the gas cost to do anything on Ethereum would be prohibitively expensive, so that ETH would be “locked” in your wallet anyway.
    • Users can simply let their rewards aggregate and claim multiple at once when gas costs are low, and they could even do this on an L2 where fees are even lower.

Trusted Block Builder Requirement

  • Not allowing people to build their own blocks and instead relying on one (or several) trusted block builder(s) goes against decentralization. I submit that this is a larger problem for Rocket Pool beyond just the Smoothing Pool and probably for Ethereum in general, so I will defer discussion of this topic to its own dedicated post. This probably isn’t the correct avenue to argue for or against it, since it may become a protocol-wide requirement anyway.

Interval-based Access

  • The use of intervals means users don’t get their priority fees / MEV as soon as they propose a block; they have to wait for the next claim interval. This is a valid disadvantage, and at the end of the day, a user’s decision to opt in or out of the smoothing pool may come down to this delay in liquidity. I leave it to the community to decide if the opportunity cost outweighs the smoothing effect.

Conclusion & TLDR

  • Copy/Paste the Merkle Tree distrbution-based RPL rewards system (Garlic Bread) for the Smoothing Pool
  • Determine everyone’s “effective share” by looking at opt-in/opt-out time and attestations during that window
  • Cheaters get booted, everybody else wins
  • Node ops can let their rewards build up and claim lots whenever they want, put it on an L2, or claim it right away
  • Size doesn’t matter, the more the merrier
  • Smoothie pool, it’s got what solo stakers crave (consistency)

Alright, now for the fun part. Let’s discuss!

14 Likes
2

To highlight the effect that a smoothing pool has, let’s consider the following scenario:

  • Maximum penalty rate is 100%
  • We have a node with one minipool, commission is 0% (to keep it simple) and we have 10% RPL staked
  • A total of 1,000 minipools (0% commission) have opted in to the smoothing pool, all attestions are successful

20 ETH MEV block happens while we are opted out

What’s supposed to happen: We get 10 ETH (50%), while 10 ETH goes to rETH. Stealing is not profitable, because the stealable amount (10 ETH from rETH holders) is smaller than the amount we can get penalized for.

20 ETH MEV block happens while we are opted in

What’s supposed to happen: 10 ETH goes to rETH, 10 ETH goes to smoothing pool. We can expect to get back 0.01 ETH from the smoothing pool as our fair share of this block. Stealing is now profitable, beceause the stealable amount is 19.99 ETH(10 ETH from rETH holders + 9.99 ETH from other smoothing pool participants), which is more than the maximum penalty. Losing out on one reward period of smoothing rewards is an insignificant penalty in this scenario.

This is fundamental to any kind of smoothing pool and not an issue of this particular design. Smoothing is supposed to remove the luck component. As a direct consequence, if you do get lucky there is more that you are supposed to share and more that you can steal.

4 Likes
3

I already posted this comment to the Rocketpool discord, but have been asked to repost it here for better visibility.

Socialized/average comission
Calculating the average fee for node_share obviously socializes the commission for MEV among the smoothing pool participants, which could disincentivize owners of high commission minipools from participating. I’m not saying a socialized/average commission is a bad thing, just something to be aware of.

Issue of unclaimable Node Operator Shares
The way you are currently proposing to calculate node_rewards, each missed attestation leads to unclaimable ETH in total_node_rewards. I propose to share the rewards due to missed attestations among node operators, as NOs are also taking the risk of missing attestations.

Proposed alternative for calculating the Share per Node Operator
Assuming we’re fine with a socialized node_share as described above, I’d offer the following alternative:

  1. For each minipool attached to each eligible node, count the number of successful_attestations on the Beacon Chain from node_start to node_end. Calculate the sum of all minipools’ successful_attestations belonging to a single node and call this node_successful_attestations.
  2. After calculating node_successful_attestations for every eligible node, sum them up into a new value called total_successful_attestations.
  3. node_rewards_share is now calculated by dividing node_successful_attestations/total_successful_attestations.
  4. node_rewards stays unchanged: multiply node_rewards_share by total_node_rewards.
2 Likes
4

I like this overall design a lot. I see two small issues and have small tweaks to address them.

Issue 1: Can be gamed by “hopping” in and out of smoothing pool

The design allows for opting in at any point. A strategy with positive expected value is to not participate in the smoothing pool and monitor the pool performance. If pool participants get sufficiently lucky during one reward period, then opt in for the remainder. Now you start earning shares of an above average reward pool and dilute people that have been in the pool from the beginning.

Proposed Solution

People can signal to opt-in/opt-out at any time, but it gets executed only at start/end of a reward interval.

Issue 2: Commission rate of nodes is ignored

As @objectObject points out above, commission is socialized. For nodes with low average collateral this design has positive expected value and for nodes with high average collateral it has negative expected value. This could lead to a race to the bottom where the smoothing pool is only attractive to 5% commission node operators and everybody else is forced to opt out.

Proposed Solution

  1. Let node_attestation_share = node_successful_attestations / total_successful_attestations the same thing @objectObject calls node_rewards_share above
  2. Let node_avg_commission be the average over all minipools of one node
  3. Calculate staking_pool_rewards based on the node_attestation_share weighted average of node_avg_commission
  4. Calculate a nodes rewards as 0.5 * node_attestation_share * (1 + node_avg_commission)
5 Likes
5

There was a discussion in the discord about the intended purpose of the smoothing pool. Is it intended to solve the smoothing of tip/MEV due to a) randomness from being selected as a block proposer or 2) variation from the amount of tip/MEV available in your block proposal opportunity? https://discord.com/channels/405159462932971535/857072928155762718/952655885459664936

Ultimately we determined that the smoothing pool is to solve both problems and that the incorporation of your_effectiveness / total_effectiveness is a good surrogate to determining the block proposal contribution by the pool participants.

A question about setting a minimal node effectiveness criteria (e.g. 96%) was also discussed as a prerequisite to entering the pool. This could be used as an incentive to ensure that a NO has worked any kinks out of their node and has optimized it for good performance. Accidents happen and nodes go offline, so we would not want NOs to be penalized once in the pool other than the fraction share dropping in equilibrium with their effectiveness score dropping.

1 Like
6

I like this idea. Agree with @knoshua again here on his two issues and think his solutions are spot on.

I’ll add that it is technically possible to include this in the same merkle tree as RPL rewards to save on gas for both oDAO and users. The leaf nodes would contain an RPL amount and an ETH amount.

4 Likes
7

Well done @jcrtp and collaborators!

This is a very promising design; effective and practical.

I suspect it won’t be a problem but one thing we should investigate early is how intensive looking at every attestation for every validator will be - I suspect it will be fine as they are constrained periods but might be worth checking how it scales.

2 Likes
8

I have intentionally not commented on the commission issue yet, because I feel like it’s more complicated than it might first seem.
Both averaging the commission among all smoothing pool participants, as well as splitting it according to the commission for each node operator has its problems.

The problems with @knoshua’s solution
While a globally averaged commission disadvantages high commission node operators, using the average node commission for the smoothing pool disadvantages the low commission node operators even more: Even though they are bringing the same value to the smoothing pool, because we assume that every attestation is a potential proposal, they are being paid less than high commission node operators for the same “work”. This highly disincentivizes low commission node operators from joining, as they would end up directly subsidising the already higher earning high commission NOs.
The specific proposal by knoshua also has the disadvantage of a variable commission based on who is missing their attestations, which might incentivize rETH holders as a group to DoS high commission NOs, in order to increase their share of MEV rewards.

Possible Solutions that need to be discussed further
As I stated in the beginning, I don’t think there are easy solutions here. I propose 2, hoping to induce further discussion:

  1. A fixed 20% commission on MEV. As this is the highest commission obtainable by any NO, this should be acceptable even for high commission NOs, without disadvantaging the low commission NOs, who get a nice little bonus on MEV.
  2. As we currently don’t have a technical limitation in place that forces NOs to share their MEV with rETH holders, it could be argued that we need an even higher commission on MEV to incentivize them to do so. I guess this is where we can start arguing about whether there already is a social contract / expectation for NOs to share their MEV. But from a purely technical perspective, there isn’t. I don’t know if the benefits of a smoothing pool alone might be enough of an incentive for NOs to share their MEV, and I look forward to hearing your thoughts on that.
2 Likes
9

Summarizing my POV from our discord discussion for visibility:
I approached this issue of preferences under uncertainty with an expected utility theory mindset. The rewards share calculation is supposed to ensure that every individual node operator has the same expected value inside the smoothing pool vs. outside and every node operator reduces their volatility by joining. Thus a rational, risk averse node operator would prefer joining the smoothing pool over opting out. So while a low commission NO is disadvantaged compared to a high commission one in the smoothing pool, this discrimination is necessary because it already exists outside the smoothing pool. I believe this design maximizes NO participation rate, given that other protocol participants don’t pay for it.

Your idea would raise the commission rate for rETH holders, but they do not gain anything from more node operators joining the smoothing pool. So taxing rETH in the design should be out of the question.

I do not see a DoS vector unless an attacker would be able to link specific validators to an IP address. Such an attacker would not only be able to exploit any kind of smoothing pool design, but would also be able to attack Ethereum as a whole. So I do not consider this a disadvantage of my proposal.

The team has stated their design goal with regard to sharing back in June last year:

https://medium.com/rocket-pool/the-merge-0x02-mev-and-the-future-of-the-protocol-c7451337ec40

The technical functionality to penalize node operators that don’t share is already in place.

1 Like
10

This was the only concern I had in mind as I read jcrtp’s proposal. Your solution is great! I only have one question left, what happens to the Node Operators’ balance in between their join signal and the start of the period. If the next period starts on 23-AUG, but they signal their intent to join on 12-AUG, what is the process for those 11 days in between. They continue to earn rewards as normal, or their rewards are handled in a special way? (Same goes for their quit signal)

11

I think the easiest approach from our perspective is something like this:

  • Let a user opt-in whenever they want (e.g. in the middle of an interval) and they immediately become eligible for that interval, though not for the entire thing - only for the fraction of time starting from their opt-in to the interval’s end

  • Require that the user be locked into at least one interval before exiting / opting out, to prevent gaming the system

We can support that both from the Watchtower side and the contract side, and it solves the problem where someone opts in a few weeks before the next interval and they just happen to get a proposal during that interval… thus losing the rewards (since they’ll go to the pool).

As another note, we can’t always reliably restart the user’s VC with the updated fee recipient info the moment an interval starts, so this process lets users pick and choose the time of that restart as part of the opt-in process. I think from a UX perspective, that’s the cleanest way to do it.

3 Likes
12

Proposed Solution

People can signal to opt-in/opt-out at any time, but it gets executed only at start/end of a reward interval.

Issue 1 sounds more urgent and impactful in the long term. We should include this solution in Atlas.

13

The problem with this is that the smoothing pool rewards are not perfectly smoothed over several rewards periods. NOs that are not participating in the SP are incentivized to join in those periods, diluting everyone inside.

One extreme example would be one minipool in the SP and 99 non-participating minipools. Suppose the average income is 0.1 ETH / period / minipool, but that period the one MP SP receives 100 ETH fee on the first day. Now it makes sense for all 99 MPs to immediately join, they contribute with 0.1 for that month, at the end of the period the SP will have ~110 ETH and each of the 99 MP receives ~1.1 ETH.

1 Like