Off-box alerting for Rocket Pool nodes: pages you when a client hangs or the whole box is gone (open-source agent, looking for feedback)

Disclosure: I’m the developer. NodeBeat is a hosted alerting service
with a free tier; the agent that runs on your node is open source
(Apache-2.0).

Why another monitoring tool, when Smartnode has alerts?

Smartnode’s built-in alerting is good, and I’d keep it. Its default rules
cover clients not synced, low disk, low ETH balance, proposals, sync
committees and updates. Two gaps remain:

  1. It runs on the node. Alertmanager lives in the same Smartnode
    stack, so if the machine freezes, loses power or drops off the
    network, nothing is left to page you. You find out from beaconcha.in,
    after the misses.
  2. There’s no rule for a client that’s down or hung. “Not synced for
    5 minutes” catches a node that falls behind, but not reliably an EL
    that stops answering or a beacon node that stops following the head.
    That’s the failure that quietly costs attestations.

NodeBeat covers those two gaps and is meant to run alongside Smartnode.

How it works

  • A small Go agent polls your beacon node (Beacon API) and execution
    client (JSON-RPC) every second. Read-only: no keys, no signing
    endpoints, no restarts, and it never touches your node wallet. It
    listens on loopback only and makes outbound connections only.
  • A client that’s dead or hung pages you in seconds. In
    fault-injection runs on a test devnet, fault to page took 5.6–8.6s
    (9 runs). A client that still answers but stops advancing pages in
    about a minute.
  • Alerting runs off-box, so a frozen, unplugged or offline node still
    pages you, via Telegram, Discord, Slack, email, PagerDuty, Opsgenie or
    webhook.
  • Given your validator indices, it also alerts on missed attestations and
    proposals, effectiveness and slashing, plus disk, peers, sync and clock
    drift.
  • Trust: signed releases (cosign), SBOM, a public
    threat model
    and alert rules,
    one-command uninstall.

What I’d like from Rocket Pool operators

  1. Would you run this next to your node, and if not, why? Criticism
    of the trust model is the most useful feedback.
  2. Which alerts would be noise for you, and what’s missing?
  3. Testers: the free tier covers 1 node. For operators with several
    machines, I’m giving a few people 3 months of the paid plan free (up
    to 5 nodes, no card) in exchange for a short weekly feedback note.
    Reply here or email [email protected].

Setup on a Smartnode host: Smartnode keeps the client APIs closed to
the host by default, so set Expose RPC Ports (Execution Client) and
Expose API Port (Consensus Client) to Open to Localhost in
rocketpool service config first. They bind to 127.0.0.1 only. The
install guide covers the rest (Smartnode’s P2P port, optional metrics).

Tested on Smartnode v1.24.3 with Geth + Lighthouse, and outside it with
Geth, Reth, Lighthouse and Teku. Nethermind, Besu, Nimbus, Prysm and
Lodestar are auto-detected but untested on my side, so if you run one of
those, I especially want to hear from you.

Code: GitHub - nodebeat/agent · GitHub
Smartnode setup: NodeBeat — Validator Monitoring That Pages in Under 10 Seconds | Ethereum & Cosmos