Funding Support Moving Forward

Governance
1

I’ve got a more thorough post-mortem coming soon™, but one thing that came out of our first round of GMC discussions was a desire for…I don’t know whether to call it clarification or further action or something like that…from the community/pDAO in regards to how we fund “support”. By support I predominantly have in mind the support that people provide in the #support channel on Discord and similar things of that nature.

Inasmuch as the RP protocol is going to pay for support, it is going to come from the pDAO. My sense as a community member is this is a thing that most of us (myself included) think we should be (generously) reimbursing volunteer support personnel for. Support is different from many other potential line items (e.g. marketing, bizdev, research, coding) in that it is likely to recur in every single period and in roughly the same form. In that sense it is perhaps more similar to liquidity incentives (IMC) stuff than the kinds of grants the GMC was otherwise deciding on.

The GMC found it very difficult to decide how much to pay the support-related retrospective award requests this past cycle, including difficulty in deciding how much of our overall budget should go to support and how to weigh the contributions of the two people who submitted RAs (plus the potential other support folks who didn’t yet but may submit one in the future).

One thing that came out of GMC discussions is that it would be great if we could have a broader community discussion about how much we would like to spend of the pDAO’s overall inflation every period on “support” and whether that money should be coming entirely from the GMC’s budget, from GMC + reserves, from trimming IMC, etc. If the funds are coming entirely from the GMC, it would make the GMC’s life a lot easier if the committee knew roughly how much the community wanted spent on support payment every cycle.

The other thing that committee struggled with was determining how to allocate the support payments. In his RA request, Patches proposed number of messages in #support as a rough metric, and Object then based his request on his relative number of messages compared to Patches. That’s probably not the exact thing we want to be incentivizing in the future (e.g. pure message counts) but I think I can confidently speak for the GMC as a whole in saying that no one is interested in becoming the managers of #support, figuring out who deserves what share of the support allocation by monitoring that channel or other locations where support is provided. I think we are open to ideas on this front, whether that be the creation of a Support Management Committee with its own support budget or other metrics that could be used for apportioning support dollars by the GMC.

Any and all discussions related to the above would be most welcome!

1 Like
2

Is there any kind of ticketing system we could use? Some user comes and makes a support ticket, someone else picks it up and finishes with them, support person leaves a ‘billable time.’ At the end of the reward period, a set amount of inflation could be divvied up by the successful ticket completion hours per contributor.

We would rely on the honor system with potentially a recourse for an appeal if you think someone lied and inflated their ticket hours. With how tight nit support tends to be, I think this is a minor issue.

3

One the one hand, a ticketing system like Zendesk is probably the industry standard for support use cases. On the other hand, it feels like introducing a lot of administrative overhead in managing tickets. It does make time and effort spent more measurable, but I wonder if that really is the problem that needs solving most. We have a couple of users very active in support and many who incidentally drop by to help out. For such a small number of ‘support personnel’, the honor system of estimating hours spent could work well enough (with # of messages as a very rough measure as ‘proof’ to back up the estimate.)

An entire support management committee feels overdone as well, but I understand the GMC has enough to do and doesn’t want to micromanage support as well. We could decide a percentage (or absolute amount) of the GMC budget as a maximum recurring support budget and a hourly rate, and then allocate from that based on hours reported by people providing support?

4

Unlike many here, I have no programming background so my solutions tend to be decidedly low tech. But here is mine:

  1. Have a central pool where all funds for support go. (in my dreams, it’s a 2/3 multisig with Patches, Object, and me, since I’d be doing it for free and patches/object have already gotten GMC funding and may donate some of their initial payouts). This collects funds over 3 months.

  2. Every quarter have a pinned message on a thread in #support, which gives an emoji for each person offering support (ie, :one: patches :two: object etc). Anyone can also post in that thread to add a person to the support list.

The message will read something along the lines of:

Please click the emoji for the individual that you feel provided the majority of assistance for you today. If you would like to contribute to funding #support, the recommended tip is 10% of the money saved or 10% of your time saved based on how much you think your time is worth. Please send donations to SupportTipjar.eth [or whatever]

  1. Have a bot which sends a message in #support every x hours (2 hours? that’s usually enough for most to see without feeling obnoxious) that refers any people who feel they’ve been helped to the pinned message.

  2. At the end of 3 months, the GMC either sends a predetermined lump sum or a matching percentage of community contributions to the tipjar. the tipjar is split between the various support staff based on their relative votes.

  3. Pin a new message. Any support staff with less than 5 votes the prior round are taken off.

Some benefits:
  1. the tips can be anonymized, ie your public discord can indicate support for 0xfornax because they publicly helped you, but your tip would not be linked to that interaction.
  2. pseudo sybil resistant: if there are support providers who seem like they are receiving a disproportionately large share (ie, concern for cheating), it is relatively easy to audit who voted for them and check whether actual support was provided.
  3. smoothing benefits: single large contributions get spread amongst many members. Similarly, one donation transaction can efficiently fund multiple individuals
  4. Anyone who does not want to donate can still show their appreciation to the #support staff who helped them, which will funnel funds their way.
Some downsides:
  1. May encourage less knowledgeable or less helpful contributors in the hope of tips (this is probably true for any way of reimbursing support)
  2. Inefficiencies: if patches helps someone 5 times in a quarter, he can only be acknowledged once for that person
  3. only for discord members (although this is where it almost all support happens)
  4. People can just go into the pinned message and vote for people they like. this should be discouraged.
5

I really appreciate all of the thoughts so far. I do worry that for many of the people who provide support, one of the reasons they are willing to volunteer their time is that there is very low overhead for tracking what they do. They pop into #support when they can and do what they can. At a minimum, I think something like @Pieter’s idea of a set budget percentage would help a lot. I was actually thinking that maybe if the community has a preference for that amount, we could apply for a bounty during the next round of applications for support payments. Sort of a standing bounty of X% of GMC budget per quarter, with people able to claim the bounty once per quarter at the end of the quarter. That would still leave the question of how to apportion said bounty. I like @Pieter’s idea of an honor system reporting of rough guess of hours spent on support that quarter. @epineph’s bot/react system is also pretty interesting and seems fairly straight-forward.

6

I’m in favor of the honor system… It doesn’t scale forever (depends on a lot of trust), but it’s a wonderful method while it does. (aka - I agree with @Pieter).

7

The honor system is definitely the simplest, and I agree with a set budget from GMC that they don’t have to discuss every quarter, but I do worry about alignment of incentives. Namely, it’s based on what the support provider feels their time is worth, rather than trying to approximate the value they provide the community.

  1. #support BadEpineph takes 2 hours to solve a problem that patches would solve in 5 minutes. Patches would get paid 5$, but epineph gets paid 120$ (which since it’s zero sum, this reduces patches’ income). Is this a concern?

  2. Who will be responsible for policing overestimates of time worked? This will be touchy because it is a tight knit community. and we tend not to like to ruffle feathers (except oDAO plumage).

  3. Is there a whitelist, and if so who will be in charge of gatekeeping who is allowed to provide support? Do we keep #support slim and trim, or permissionless?

However, if the major players in #support give their approval (they will probably be the most likely to underestimate their hours because of the challenge of tracking large, irregular schedules for 3 months), this seems like a great start and well worth funding.

8

2 and 3 I feel good about…

2 - The supporters should talk among themselves if they think something is off. If they are unwilling to, or they can’t get it figured out, they can ask the GMC to cease payments or file a dispute on the next iteration.
3 - I think the previously paid supporters can gatekeep - so “low bar permissioned?”. I think we have the same safety nets as in 2.

GMC can cease payments
  • If a majority of the GMC agrees that a grant recipient is failing to provide the specified services to the protocol in a timely manner (as documented in the original application and in subsequent monthly updates), the GMC SHALL publicly announce such a decision and cease any future payments. This decision MAY be disputed by anyone through the creation of an RPIP within two weeks of the GMC’s announcement. The RPIP SHALL be subject to a snapshot vote.
Anyone can file a dispute
  • Anyone MAY file an RPIP disputing a grant, bounty, or retrospective award within two weeks of the announcement of recipients. Such an RPIP SHALL be subject to a snapshot vote.

1 - This one’s a tough problem, at RP and major companies :stuck_out_tongue:. I suspect that supporters would tend to want this help even so, but perhaps some of the more active folks there (@Patches @objectObject @fornax) can give their thoughts.

9

Edited because the forum ate a bunch of my text:

Trying to synthesize the above. I think a bounty would work a bit better than a grant support in this case. Structured something like as follows:

Assuming this application was submitted on Apr 1 and funded effective May 15, it would look something like:

Recurring bounty worth 1,000 RPL per quarter (this is approximately 10% of GMC’s budget every quarter - other proposed percentages are welcome!). GMC creates a forum thread every quarter on the 15th day of the final month of the quarter (e.g. June 15). Anyone who provided support during that month can post in the thread between June 15 and June 29 with an estimate of the time they spent on support and the nature of the support they provided (#support, forum thread, twitter, DMs, other?). Any community member can also chime in if they think the estimates sound too high for someone or (more likely) too low.

GMC then calculates on the last day of the quarter (e.g. June 30) the share that each person gets of the 1000 RPL based on their share of the claimed hours. Payout goes out on July 1 and we start the next quarter anew.

This does run into the issue where people claim more hours than they should, but personally I’d rather start assuming everyone is acting in good faith and modify as needed rather than being overly complicated in trying to ensure accurate estimates.

10

This recurring bounty structure is a good approach, I think. First impression of the 10% budget sizing is that this quite significant, especially as it will happen every quarter. Support is very important, but we do need to keep enough budget for everything else we’d like to fund.

One thing though - this bounty is a maximum so it would not necessarily be fully paid out every period, depending on support hours spent. We also still need to determine a hourly rate for support to close the loop - any suggestions? (If hours spent * hourly rate exceed the maximum bounty, the payout per person - and effective hourly rate - would be reduced accordingly.)

For reference, in the first period 2330 RPL was spent on support payments, but this was of course retroactive over a longer period of time (any estimations how much this would be, normalized to a single period? That’d give us an initial ballpark figure for our periodic budget. Preferably we’d also have some slack in the budget to account for growing support requirements as the protocol grows.)

Agree with @Valdorff this is a tough one to tackle. Not even e.g. a ticketing system with a per-ticket reward alone would be of great help here. You could start classifying types of support requests somehow and move away from hourly compensation. Not a simple path.
For now I don’t think it’s a big issue, as people generally know where they can help out effectively, and others can chime in to help tackle tougher issues faster.

11

The 10% was based on doing what Pieter suggested in terms of estimating it. The committee awarded 2330 RPL out of a built up budget of approximately 42000 RPL, which is approximately 5.5%. However, that was only given to two people and there were presumably others who contributed to support over the past year (though likely not at the same rate). If we assume that Patches + Object account for 60% of support then that would give 9.2%, which I rounded up to 10% for idiosyncratic reasons. I imagine you could tweak that number down or up a bit, but something in the 7.5-15% range feels right to me.

I agree with the idea that the bounty is a maximum, but the more you lean into that idea the less this actually does for making life easier for the people on the GMC. My guess is that if we set a bounty at X% of GMC inflation payments for a quarter, the GMC is going to award the full X% and, to be fair, that is my purpose in starting this thread.

1 Like
12

I’m realizing I never replied to this, which is probably because I don’t really have any good suggestions.

On compensating #support

I don’t think the “helpers” are going to be able to come up with a good system. I’m there because I’ve acquired skills completely orthogonal to the abstract ones need to solve this; I’m not good at this sort of stuff. All I can offer is some information about my experiences, which I’ll keep brief:

  • We didn’t do ourselves any favors by being “platform neutral” in the guides. I think Raspberry Pis were probably a mistake. We should have also dissuaded people from using OSX with Docker Desktop- it has been nothing but trouble lately. These users are disproportionately present in #support.

  • The vast majority of questions asked in #support have been answered before. I’d probably prefer to see bounties paid to people who provide self-help resources, like PRs to add content to https://rocketpool.support and the official docs repo.

  • The lack of search functionality in the official docs repo is hurting us. Most people don’t want to ask for help. They only ask for help when they can’t find resources on their own.

  • 10% of the cases take 90% of the time. It may make sense to clock support cases from start to finish and not worry about compensating the long-tail of quick fixes. I’ve spent days to weeks working with a few NOs, and the grueling cases are really the only ones that make me question why I keep doing this. Helping someone, say, delete a swapfile is like lending your neighbor a muffin tin- I don’t think we need to worry about capturing that effort.

  • I think the regular helpers could probably do a better job coordinating with one another. The async nature of some of the problems people face makes us waste time somewhat regularly- eg- helping someone who has already tried a handful of things with another helper, and going through the same paces again.

At the end of the day, though, I think we would benefit from some light data mining. I can’t tell you how many times I told people that sync progress of 100.00% is actually rounded up from 99.999% before it occurred to me that I could just submit a PR to remove that confusion. If we were running a company, we’d have postmortems and ticket labels and all that good stuff, and we’d be working to prevent the need for hands-on support from arising in the first place.

I don’t know how to fix the disparity between ad-hoc volunteer-run casual support and the kind of work that needs to happen to reduce the burden and scale to a larger pool of node operators, but I think that’s what we need to focus on compensating. Of course, we can and should continue to compensate both- but paying people for hands-on support is not anywhere close to as efficient as paying people to prevent that work from needing to be done in the first place.

Ramblings

At any rate, I don’t intend to ask for more compensation for hands-on support than what I have already received. There are a few reasons for this:

  • I’m not in there as often as I was in 2022.
  • I need to work less or I’m going to go completely bald and my partner is going to kill me. I somehow have a full-time job still, on top of everything else. I’d love to give Rocket Pool my full attention for a while- there are a lot of things I want to build- but between cost of living and my particularly hefty health insurance requirements, I’m not sure I can. I’m actively trying to figure this out.
  • I want to go back to building stuff, so that’s what I’m focusing the time I do have on these days. I’ve worn many hats in my career, but rarely the sysadmin hat, so while my time in #support helped me learn new skills that bolstered my primary abilities, there are diminishing returns and I am eager to get back.
  • The compensation I did receive was generous. Lacking for any individuals to thank, I’d like to say thank you all. Very much. Being part of this community has been a joy for me. That said, in #support, I feel like I’m punching below my weight. I feel like helping people one at a time isn’t an effective use of my time. I’d rather build things that help everyone at once.
4 Likes
13

So it’s been a year, so I wanted to bump this. #Support is one thing that makes RP unique, and the protocol would not be nearly as used if not for the relentless efforts of the many people who participate in #support. I don’t think we are much farther along on automating portions, and obviously the docs are an ongoing and large project.

I wanted to see if there was appetite to trying some experiments on reimbursing the support volunteers in an ongoing fashion? I think even if it’s inefficient at first, by iteration hopefully we can get to a decently reimbursed volunteer crew; above all, I’d like to avoid a round of discussion where everyone agrees it should happen but no agreement is reached.

I’ll throw out-
1). Recurrent Retrospective award with a monthly pot of 10k usd
2). Use @haloooloolo s script to calculate the hours spent (under newly chosen criteria)
3). Someone from GMC goes through the top 10 entries; weeds put any people who are mostly requiring support, until 10 true non-bot ‘supporters’ are found.
4). Take a sampling in some way (eg take 10 messages starting at the same moment in time: 1, 11, 21, 31… in #support). Any person with <80% messages directed towards people seeking support (“you’re welcome” would count but !dance would not) is removed for that month.
5) monthly award is split amongst the remaining ‘supporters’ based on hours calculated

Benefits:
low overhead, likely around 30 minutes per month to administer

Neutral

Non-reliant on self reporting

Easily iterative, as the pull criteria from the script can change

Downsides:

Doesn’t entirely remove the benefit of spamming extra messages to increase billed time, although gives it a risk factor.

May take some ‘fun’ out of support since an overabundance of !dance messages may disqualify. We could mitigate by allowing certain messages to be ignored by the script and GMC checks.

By random chance, some ‘supporters’ will inevitably be disqualified erroneously some months, and others may be erroneously paid. However, because these errors simply result in your share being redirected to other ‘supporters’, it should even out over time and is better than status quo.

This is not “what our support crew is worth,” but is more a thank you and recognition for their work.

This does not reimburse low frequency but consistent contributors (I think this is ok for now, as I am going for ‘start with something’).

other thoughts:

‘Supporters’ will have responsibility to self police if there is someone who is frequently using support for non-supportive messages, or is not a knowledgeable/effective supporter.

Would be easier (but not required) if the monthly payment went out to one trusted community member who divided it up as specified by GMC. Or to a non-trusted community member who paid out and then was reimbursed by the GMC.

I don’t think this should be seen as an alternative to long-term structural changes to the ad hoc nature of #support.

I was planning to post this after ethdever wrapped up, and I see haloooloolo posted a retroactive; I don’t think they really interfere with each other

14

I’m wondering if a restriction on the frequency one can be rewarded would help. It discourages spam and ensures that contributors who aren’t at the top have a good chance to be rewarded, if they’re above a threshold we think is worth paying for. It shouldn’t necessarily be “who is #1 at helping everybody” and more like “who is consistently meeting the minimum target”. Anyway I haven’t put more than 5 seconds of thought into this. Throwing it out there.

1 Like
15

The lack of search functionality in the official docs repo is hurting us.

Algolia offers free search integration for FOSS projects. Worth applying for, if the “thing that builds the doc pages” supports Algolia.

1 Like
16

I actually like the retro method we’ve been using. It’s now paid out three times, so there’s some reasonable expectation that big contributors will get rewarded. Having humans (including other support folk) look makes it essentially not worth trying to game.

2 Likes
17

So the individual retro method works great currently; it is extremely cheap for the protocol and easy for the GMC because few people apply. Once more people apply that is no longer the case. I think we have ~250k USD of outstanding liabilities at the going rate for support, and ~150k per year ongoing for the top 20 contributors. Given that #support is only a small part of the protocol, it’s easy to envision a time where “we are technically insolvent, but things are going great so far.”

My opinion is that if we know we need support contributors ongoing, we are signaling that we will pay the going rate if you ask for it, and we have the money currently, then we should just be paying as we go. Expecting people to individually and arbitrarily apply retroactively, at an interval of their choosing, is a bit counterintuitive to me given the consistency of the work.

1 Like
18

image
image646×614 23.7 KB

All right - I had not done numbers out. This is the list I’m seeing for the last 6 months or so. At haloo’s $90/hr number, that hits 75k or so for the list minus bots (duh) and employees (?). This is more time and a lot more people than I expected tbh. I’m convinced it’s worth implementing something less one-off.

I like most of your methodology, but I think it misses the (impressive) breadth of supporters I’m seeing. For example, if poupas is number 18 every month, should they receive $0 forever? I’d suggest using “hours since last paid” numbers instead, so lower-down folks can accumulate and get into the payout list. Should be doable with very small tweaks to the script and a look-up-table of when folks were last paid. There’s a bit of volatility because if you’re a small fry and you get paid in a month where a lot happened you’ll get less than if you get paid in a quiet month. I think this is worth just accepting as a modest inaccuracy.

I would also suggest something like running with 2 different reasonable hold durations for time between messages - maybe 10 and 15 minutes or something. This would serve to flag cases where the order is very different for human attention.

4 Likes
19

When I made that post were using a different “thing that builds the doc pages”.

The new “thing that builds the doc pages” has much better search functionality!

20

So I agree that there is something unsatisfying about having someone work steadily for a long period of time and wind up with nothing to show.

However, based on this, after the 20th place [i know i had said 10 spots, but that’s negotiable], we are getting to 10hrs/6 months = 1.5h/month. I feel that there is likely not a high expectation of reimbursement by the participant for that amount of work (definitely more in the fun/hobby category), the administrative burden/dollar spent to differentiate supporter from supportee would be much higher, and because of the decreased pattern recognition the quality of support offered is more likely (on average) to be less efficient (poupas non-withstanding).

Paying on 'time since last paid" also somewhat negates the benefits of consistent budget item for the GMC (ie, payments would increase when more people decide they want to provide support); it also negates the benefit of people being paid more per hour when there are fewer other supporters (ie if haloooloolo and patches go to a 3 month zen retreat or someone gets a new job).

There’s a bit of volatility because if you’re a small fry and you get paid in a month where a lot happened you’ll get less than if you get paid in a quiet month. I think this is worth just accepting as a modest inaccuracy.

:point_up_2:This i think is actually the big benefit for the protocol for the steady payment structure; kinda an Uber type supply/demand relationship for gig workers.