The GMC recently met with bloXroute to learn about their Validator Gateway product, which has the potential to enhance effectiveness across Rocket Pool validators. Example data can be viewed at https://relayscan.io/
bloxRoute is a blockchain distribution network that gave a presentation at Rocket Pool’s Denver Lift-Off event. That video can be viewed here.
Relay Proxy A free service co-locating the relay with the node operator, streaming block bids to node operators in real-time. You can set the timing for accepting bids within a slot.
Blockchain Distribution Network (BDN) A paid service utilizing data stripping to send data via direct routes. Once a validator node accepts a block bid, the BDN ensures rapid propagation across the network. For duties like attestations or sync committees, the BDN Gateway helps keep nodes synchronized with the current head slot, enhancing validator effectiveness and increasing high-paying attestations within block proposals. The BDN Gateway aims for near 100% effectiveness.
Some Questions That Were Answered
How would it work for small solo stakers?
Relays can be set up regionally, and they can be purchased in various locations.
How many relays would make sense for the entire US?
Possibly one per state, or fewer if that is excessive.
What happens if a relay goes down?
There is no risk, as connectivity to other relays remains.
Is it only profitable when maximizing timing games?
Node operators can manage risk by setting delays.
Payment and Billing
The GMC is interested in the fixed price per pubkey option. bloXroute is willing to accept RPL as payment.
The GMC is seeking the pDao’s input. Please ask any questions you have below, or provide any feedback.
–
Should the Smart Node integrate the bloXroute Validator Gateway?
I’m pretty unsure about this. Better effectiveness for Rocket Pool sounds good. But bloXroute is one of the worst relays when it comes to trying to use their public API endpoints, so I feel biased against them. And I would like to understand the centralisation and other risks here better before signing off. That’s why I voted no. It feels shady because of the brand and the money involved, but I want more analysis of risks to make an informed choice.
Relay Proxy seems unremarkable in the sense that bloXroute is already an approved relay, and having better RTT is a strict improvement.
I’d prefer for the BDN decision to fall to individual node operators, who can opt to pay for it. I don’t think it’s sensible for the pDAO to socialize the cost. I do think we should gauge pDAO sentiment before trying to integrate smart node.
I mentioned to bloXroute in Denver that RP NOs receive partial income per public key and the pricing is less attractive unless it accounts for that.
It’s not clear to me what we’re voting on. Is there a concrete proposal with costs? Or it more of a vibe check? (or is the video the proposal, which I haven’t watched?)
I’ll start by caveating that I think my understanding is at like a 5/10 here. This may be its own issue, but it does mean a large bit of salt is appropriate when reading the rest.
My gut feel on BDN is that it creates a second source of truth. If it’s wrong it can cause slashing etc (either direction depending how popular BDN is). If it’s right, it’s providing a more-profitable move than using spec consensus. Bugs me at the ethos level
The relay collocation sounds helpful. That said… I’m rather confused why this isn’t the default behavior? Like wouldn’t it be good for all parties to have a “US east” sorta thing? Variants of this have even used source IP to decide what server to point you at for a given domain. What I’m asking here in context amounts to “why should RP pay this?” If this is cheap enough, sure why not, but I would definitely want to feel very comfortable that we come out ahead (even if bloxroute prevalence changes quite a lot).
I’m also somehow like ramana and just don’t like bloxroute. They have two relays, one “max profit” and one “regulated”. However, at some point they changed the “max profit” one to also censor:
bloXroute hosts two types of MEV relays, Max profit and Regulated, that validators and builders can connect to. Both two relays currently propagate all available transactions/bundles except the ones that interacted with addresses sanctioned by OFAC, and they have the same filtering requirement, although for legacy reasons, bloXroute still maintains the two relay names in parallel.
(MEV Relay (For Validators) | bloXroute Documentation)
This makes everything bloxroute does very uncertain. They might start with something ok and then decide to censor stuff, gaslighting their customers…
Thanks for your feedback, Ramana. First, we would like to understand more about your experience with our public API endpoints. Our data API endpoints are identical to other relays so we don’t understand your comment.
Then regarding your centralization concerns, let’s briefly review the BDN and relay proxy, how they operate, and how they’re not a centralization concern.
The BDN when added to your node is the same as adding another node as a peer, and you can think of the BDN being the best peer node that you can ever have, whereby transactions and blocks are both read and written to the blockchain faster >90% of the time. This helps NOs maintain synchronization, especially in lower-performing regions. So the BDN helping lower-performing regions be more in sync, actually helps decentralization by offering these NOs better synchronization with the blockchain. Then when connecting your node to the BDN, your node is still going to verify, send, and receive transactions and blocks from other peer nodes, and so there isn’t a risk there.
Then for the Relay Proxy - adding it is the same process as adding another relay, and by adding it, you still have the option to connect to any other relays, thus still providing you with optionality and decentralization.
Thanks for the reply. I was not saying that the API is different but rather how the API endpoint behaves. It has been quite flaky - maybe more so in the past? - both for validating (current slots, e.g. this kind of problem error="Post \"https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/validators\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)") and historical data (I have a list of slots I had to special case bloxroute for because the API was returning bogus data).
Thanks for the info on BDN and Relay Proxy. That sounds fine then. (Although I’m confused about how it works under the hood - how e.g. the BDN is able to be such a great peer, and whether there’s any risk if not everyone can afford to use it.) And I think I share Valdorff’s concern about the second-source-of-truth.
Patches, we appreciate the feedback. The Relay Proxy’s gRPC connection significantly reduces latency between the validator and the relay - helping NOs be more in sync with our relays, and this is something that will then actually help NOs set up in lower-performing regions to better decentralize the network.
Then about your take on socializing the costs or having the individual NOs pay it, we’re here to be transparent with this proposal, and believe the decision should be made collectively and provide mutual benefit for everyone involved. So, we welcome your input, and look forward to making something happen for the best.
Thanks for your comment, Valdorff. Let’s unpack it.
First, there’s no risk of the BDN acting as a second source of truth, as connecting to the BDN is no different than connecting to another peer node, and with doing so, your node is still going to be sending, receiving, and verifying transactions and blocks from our BDN as if was a standard peer, ensuring there’s no risk of slashing.
Second, NOs are going to be geo-distributed, especially with Rocket Pool, and so having our BDN setup, it’s automatically going to point to the nearest relay, however, the NO has the option to override this if they so choose.
Lastly, to address “Why should RP pay for this?” This echoes Patches comment questioning whether the cost should be socialized, or be left to the individual NO to pay for, and we’d like to emphasize that we want to bring this proposal with transparency, and come to an agreement that makes the most sense for the community. In short, our take is that the benefits of speed that the BDN provides - better performance (effectiveness), less missed slots, lower-latency, and helping aid decentralization - provides a compelling reason for Rocket Pool to consider this proposal.
Maybe I’m not understanding something – I’ll happily admit I’m not expert in this area.
My understanding is that my state depends on a small set of peers - a couple dozen - that clients try to distribute widely. There are documented eclipse and false friend attacks where essentially any attempt to control which peers are chosen is considered attack surface. I think your quote is saying that you do in fact displace a peer with a centralized “trusted” peer, and thus damage the intended behavior of the blockchain. Am I reading that wrong, or are you simply saying that replacing a single peer is an acceptable tradeoff for the speedup?
We’ve seen data artifacts with the bloxroute endpoint, and not with others.
Eg from an analysis last fall:
There were 13 slots that reported one or more mev_rewards from a bloXroute relay and an mev_reward from another relay with mismatching reward sizes. The bloXroute mev_rewards were deleted in favor of the other relays based on beaconcha.in as a second opinion. The number of slots makes this negligible regardless. This applies to slots 6209620, 6209628, 6209637, 6209654, 6209657, 6209661, 6209675, 6209814, 6209827, 6209867, 6209871, 6209957 and 6209964.
Other relays have shut down in the past. So you not shutting down your “max-profit” relay that is no longer max profit is an act of bad faith in my eyes. Keeping it for “legacy reasons” makes no sense. Just shut it down so that everybody knows that you do not care about Ethereum decentralisation and instead suck up to whatever the US might do.
You talk big about decentralisation, but this change shows that you only care for the US of A and not about all the other Ethereum users that should not be impacted by the stupid overreaching OFAC sanctions!
Edit: Why this matters is that we now need to assume that all your products can become “compromised” by US regulations. I as a European would rather have something from a corporation which respects all countries and doesn’t exclude 70% because 30% might need censoring.
Thanks Valdorff. The attack vector you referenced, Eclipsing Ethereum Peers with False Friends has been fixed in geth as described in this paper: https://arxiv.org/pdf/1908.10141, and is related to go Ethereum client in the POW era before the merge. Now block propagation is done in CL with beacon nodes instead of EL with geth nodes, and so the main idea is that even if a peer provides an invalid/bad block, the beacon node is supposed to reject it. This is the safe guard built into the protocol that ensures no one can successfully act as a second source of truth.
Then regarding the example you presented with 13 slots with artifacts - this was last fall, an issue we took seriously, and we have improved a lot since then.
Thanks Ramana. We have registration issues sometimes and we have improved a lot. This endpoint is used by almost all other validators, and registration issues are common, not only with our relay, but with other relays as well (this is based on the MEV Boost logs which we receive from validators). Given that close to 1 million pubkeys register in a short time, some registration failures are expected. Even if you see that error, even on timeouts we still have a 99.9% success rate with registration and even if one fails occasionally, they are still registered from the last time they registered so they are still safe. We are constantly improving the reliability of the endpoint, and we can commit to work with RP on these issues and fixed them.
Regarding your point about how we work under the hood and the concern about us as a second source of truth, first please reference our recent replies to Valdorff, where we explain how even if your node receives invalid/bad blocks, the beacon node is supposed to reject it. For how bloXroute works under the hood - it’s best to think of our BDN acts as an acceleration layer that propagates transactions and blocks faster than the p2p network through compression, indexing, and smart routing to be able to deliver the data that’s already on the blockchain faster. This speed is where the benefits of our BDN come into play, and translates to better performance (effectiveness), less missed slots, lower-latency, and helping aid decentralization for validators.
I think what Valdorf is getting at though is a legitimate issue, even if eclipse attacks aren’t as big of an issue after the merge as they were under proof of work*. What comes to mind specifically - by giving the validator a “faster” view of the head of the network, this service further enables timing games to be played by builders and relays to delay the signing of the block header further into the slot. If enough validators use your service, “late” arriving blocks that are late because of timing games have better odds of not being orphaned.
Also more philisophically speaking - you can’t mitigate one kind of centralization (lots of validators hanging out in US-EAST to get better bids from relays) with another kind of centralization (“here’s a ‘faster’ view of the ‘canonical’ head of the chain to sometimes get the same advantage of being near a big chunk of the network”).
The better solution is for validators to distribute themselves globally as much as they can and for the timing games to stop. Maybe that’s too idealistic, but decentralization has always been an idealistic value aspiration.
*I think eclipse attacks are still possible - but mainly from a standpoint of a group of evil peers potentially censoring validators and their blocks and attestations.