Project: OpenUX node operator and community user research study
For a version of this grant application with all links included, visit: OpenUX_Rocket Pool grant proposal for dao approval/vote. Final and public INCLUDING LINKS - Google Docs
What is the work being proposed?
An in-depth user research study will be conducted to surface the needs, pains, goals and experiences of RP node operators and community members. The insights and recommendations will:
- Support those making decisions in the DAO; improve engagement and messaging with individuals inside (and outside) of the community, identify new ways to improve governance participation and processes, and optimise documentation and support for new node operators.
- Identify new product and service opportunities that support Rocket Pool’s ethos and goals, and also support others building services or tools in the Rocket Pool ecosystem.
Research Questions this study will answer
Please note, these are example research questions the study will answer, we will go through a refinement and prioritisation exercise with stakeholders in the community at the start of the research.
- Discover who is currently running a node with RocketPool
- Assumption: The APY is high, so why are they not running a node? What is blocking them?
- What are the different needs of the different users? What are the behavioural personas? What statistics can be gathered around what the makeup is of your community and/or node operators?
- How does Rocket Pool’s goals balance against the distinct types of users in the community?
- What are the 80% of quiet people thinking/saying?
- Discover why node operators (and stakers) are using Rocket Pool over alternatives
- What do they think of other protocols in terms of security?
- How confident are they about rocket pool’s battle-tested-ness?
- Learn about current node operators pain points, frustrations and barriers around running a node with Rocket Pool.
- Learn about the current node operators onboarding journey. How competent/confident people are to get their node online themselves, and maintain it?
- Understand the experiences of community members (who may or may not be node operators) to understand
- What are their needs, pains and goals with regards to community?
- What might attract them to greater governance participation?
Our User Research Approach
- We will run a consultation with stakeholders within Rocket Pool and the community to refine the research questions and assumptions to test, and prioritise these.
- OpenUX will design and launch a survey of the community to collect signals around operator (and community) groups, to explore needs-based segmentation and insights. All data will be anonymized and privacy will be protected.
- We will then run 10-15 in-depth user interviews with participants who opt-in to clarify and deepen understanding. All data will be anonymized and privacy will be protected.
- The end deliverable will be a published report that addresses all the research questions and assumptions, and includes user journey maps and personas.
- We will run sessions with the community and Rocket Pool team to make sense of the findings and support you with using them for decision-making in the organisation.
- We will then provide ongoing unlimited support to people wishing to integrate the insights and recommendations into their own product or service development.
How do we run ethical, privacy-preserving user research?
We are aware that there may be resistance from some in the community about sharing their experiences with an outside organization (or even with the Rocket Pool community). Participant privacy is our number one consideration, and success of the research outcomes depends on people being able to trust our research processes. Survey responses will be anonymous. Those who are happy to participate in interviews can opt-in to do so, and their anonymity is respected too - we have plenty of experience researching with participants using pseudonyms and/or no camera.
We will never collect and store Personally Identifiable Information (PII), except for an email address that participants can volunteer - and that is only used for inviting them to an interview with us. Email addresses will be stripped from the raw data to conduct analysis. Video recordings of interviews are only used for the purpose of transcription and analysis, are stored in a GDPR compliant manner, and will be deleted roughly 6 weeks after completion of the study. No one except the research team will have access to videos or PII.
When recruiting participants for research, it should be noted that there will always be a small minority of people who will just not engage at all - that’s the same for any service, product or brand. But there is always a silent majority that is effectively reached by using user research recruitment best practices. In initial consultation with stakeholders, we will determine the profile of user that is needed in order to answer the prioritized research questions. We use screening methods to ensure that we not only invite to interviews/surveys only the right profile of user, but balanced quotas too. For example, it might be decided that we want to interview 8 x experienced mode operators from the community and 8 x outside who are solo stakers/or people in a particular region etc. Whatever the profile we need, we’ll choose the right method for recruitment and use smart screening (and quotas) to get the right people participating.
We use a range of techniques which ensure we get the right level of participation. Firstly, we make it clear who is conducting the research study; the fact that a third party is conducting the work can even support uptake (participants have a different perspective of outside organizations than they do for the org we’re conducting the research on behalf of). We are doxxed individuals, happy to be contacted at any time about the research, and will be available on community calls/AMAs.
We reward people with financial incentives for taking part in the study (silent voices are incentivised to speak up when there is a reward for doing so), and this is user research best practice. Usually for interviews, we pay people $1 per minute of their time, so for hour-long interviews we pay people about ~$60 for their time, but if it is an especially tricky recruit we might pay more (up to ~$200).
We choose the right places to find participants. We advertise the ‘call to research’ well outside of the discord, as well as inside. And we use a range of different recruitment tools/methods, including web3 recruitment platforms such as Web3UX.org, as well as social media.
Is there any related work this builds off of?
n/a
Will the results of this project be entirely open source? If not, which parts will not be, why, and under what license will they be published?
OpenUX sees user research as a public good, and the full report of the research results and recommendation will be published and made available to the entire community (and externally). Anything that identifies individuals will be stripped from the report, anonymising all responses.
Benefits
How does this help people looking to stake ETH for rETH?
Having insights about the staking experience as it stands today will have a systemic benefit to the organisation, and therefore new potential stakers.
The research findings will uncover opportunities for Rocket Pool to invest their resources in grants, roadmap items, and people that lead to overall protocol growth and more successful, happy users.
How does this help rETH holders?
Depending on the focus of the research, innovation opportunities will be identified that help builders in the Rocket Pool ecosystem design and develop better experiences, tools or products for rETH.
How does this help people looking to run a Rocket Pool node for the first time?
The outcomes of the research will support decision-makers at Rocket Pool with data they need to improve the documentation and support for first time node operators. Insights can help support certain key product metrics, for example reducing time to get a node up and running, operator retention, reducing support requests etc. The result will mean that the experience is improved for people looking to run a Rocket Pool node for the first time.
How does this help people already running a Rocket Pool node?
Likely the focus on these users will not be a priority for the research, but outcomes for first-time node operators, and insights that inform better communication and governance will support people already running a node.
How does this help the Rocket Pool community?
The community, including the silent majority, will have an opportunity to have their opinions and views heard. This can build trust and improve engagement in the organisation. Research insights will recommend specific tactical and strategic changes to decision-makers which will improve the experiences of members. Better alignment, understanding and cohesion are positive outcomes from acting on the research recommendations.
How does this help RPL holders?
RPL holders will benefit from actions taken off the back of the research: increased engagement, better brand equity, and increased trust in the long term sustainability of Rocket Pool and RPL, supporting token price.
What other non-RPL protocols, DAOs, projects, or individuals, would stand to benefit from this grant?
Other DAOs and protocols can learn from the research and decide to implement their own changes based on the findings, where suitable to them.
Will the resulting project be open source?
N/a. However, the research findings and recommendations will be published.
Team
Who is doing the work?
- The Lead User Researcher on this project will be Georgia Rakusen (ultimately accountable for the outcomes). Also working on this project will be OpenUX co-founder Manansh Shukla.
- OpenUX is a web3-native collective of the most experienced UX researchers working in the space today. Our and our members’ previous clients include: MetaMask, ConsenSys, Protocol Labs, MakerDAO, Yearn Finance, Liquity, Yearn, dHedge, Stacks Bitcoin ecosystem, Lens Protocol, NEAR, Solana, Infura and Pocket Network. Collectively we have interviewed more than 2000 individual web3/crypto users and developers across all use cases (NFTs, DAOs, infrastructure, DeFi, staking and more).
- Not only are we highly experienced user researchers, we are ETH-natives. We understand the staking landscape and are able to bring our deep knowledge of users (whether stakers or node operators) to add meaningful context to the insights from our research studies.
- We see user research as a public good in web3, and are dedicated to publishing our findings. Previous published research by our core members include the following (URLs can be accessed from the google doc at the top of this grant application):
- We were awarded a grant by NEAR Foundation to conduct an in-depth study of the experiences of web3 users who are new to NEAR attempting to learn and onboard themselves into the ecosystem
- A longitudinal study on the unique needs of Filecoin DeFi users for the company Glif.
- Metaverse user research study for Protocol Labs.
- DeFi user research report for ConsenSys.
- Ethereum 2.0 Staking ecosystem report for ConsenSys.
What is the background of the person(s) doing the work? What experience do they have with such projects in the past?
Georgia Rakusen, Lead User Researcher, has over a decade of professional user research experience, working at start ups and scaled technology companies, including work for the UK Government.
Georgia has been working exclusively in the web3 space for 5+ years. As User Research Lead at ConsenSys she delivered high value strategic insights to a range of product teams building for developers, institutions, and general consumers, including Protocol Labs for the Filecoin launch, and the successful launch of MetaMask’s token swap feature. She is co-founder of OpenUX and a collective of experienced web3 UX researchers on a mission to increase the quality and quantity of UX research in web3. She is also co-founder of Web3UX, a user testing panel for web3-native users and product teams.
Recent talks (links in the google doc)
- "Going on Safari: Researching Users in the Metaverse”, Devcon, Colombia (2022)
- Web3 User experience sucks. Find out why. Poolside podcast 2022
- “Broke AF: Crypto UX is Bad but Getting Better” panel discussion, EthDenver (2021)
What is the breakdown of the proposed work in terms of milestones and/or deadlines?
The project will take approximately ~10 weeks from start to completion. Estimated milestones:
Weeks 1-2: stakeholder consultations and planning
Week 3: survey design
Week 4: survey live
Week 5: survey analysis
Week 6: initial reporting of survey findings
Week 7: recruitment for interviews and interview guide written
Week 8: interviews conducted
Week 9: interviews continue and analysis
Week 10: analysis is completed
Week 11: final report and recommendations delivered with replay to community
Week 12 +: ongoing integration support with those looking to learn more from the research and apply it to their own decision-making.
How is the work being tested? Is testing included in the schedule?
n/a
How will the work be maintained after delivery?
OpenUX will provide unlimited support to those in the Rocket Pool community looking to integrate the research findings and recommendations into their own decision-making and workstreams.
Payment and Verification
What is the acceptance criteria?
A full written report that addresses the prioritised research questions and assumptions to test, backed by evidence from the multi-phase robust research, and includes both tactical and strategic recommendations to the community, culminating in a read-out of the findings to the entire ecosystem.
If OpenUX is unable to reliably and robustly find the answers to the priority research questions and assumptions (as defined in weeks 1-2), we will continue the work at no extra cost until this outcome has been reached.
What is the proposed payment schedule for the grant? How much RPL and over what period of time is the applicant requesting?
The total cost for the research is $50,000 USD (happy to be paid in stablecoin or RPL for the equivalent value).
We propose the following milestones but are happy to adjust:
- 25% in week 4 when the survey goes live
- 25% delivered at the end of week 6 when the survey results are completed
- 50% (remainder) when the final report and recommendations have been delivered in week 11.
How will the GMC verify that the work deliveries match the proposed cadence?
OpenUX will provide weekly updates on the schedule of work in the Rocket Pool discord. We will ‘work in the open’; all planning docs will be publicly available. We will keep an updated schedule so anyone in the RP community can check on the status of the work.
What alternatives or options have been considered in order to save costs for the proposed project?
We anticipate that the hours to conduct this research work will cost more than the proposed grant amount, but have capped it at that amount in order to encourage a successful grant application. We eagerly want this work to be done and believe it will be incredibly valuable to the Rocket Pool ecosystem.
From experience, integrating the research findings is the most important step of the work (research just existing is not enough to make change happen). Rather than cost integration time, we have decided to provide unlimited support for free, because we want the outcomes to be successful and ensure that any builder or decision-maker in Rocket Pool has what they need to turn recommendations into actions.
Conflict of Interest
Does the person or persons proposing the grant have any conflicts of interest to disclose? (Please disclose here if you are a member of the GMC or if any member of the GMC would benefit directly financially from the grant).
None.
Will the recipient of the grant, or any protocol or project in which the recipient has a vested interest (other than Rocket Pool), benefit financially if the grant is successful?
No.