Relay Manager implementation plan

Current direction

The approved Fetch / Coverage / Sync design is the source of truth for the next implementation. It supersedes the earlier 500-event split-window strategy, one-hour presence expiry, global Coverage table, and proposed mandatory per-ID existence-check stage. The completed sections below remain the historical implementation record, not a claim that the new design is implemented.

• Record the Q&A decisions, UI flow, persistence direction, migration constraints, and deferred scope in the design document.
• Reconcile README/architecture links and the active implementation backlog.
• Implement the milestones below. This documentation-only update changes no runtime code, database, relay settings, or existing fetch jobs.

Runtime status at planning time

Working: account/relay settings, automatic metadata/socket checks, public-event archive with legacy split-window checkpoints, NIP-65 publication, and persisted positive presence facts. Pending: NIP-77, incremental per-relay fetching, scoped query/range history, revised Coverage UI, eligible missing-copy sync and read-back.

Next implementation milestones

1. Evidence, query history, and migration

• Add additive per-account/relay cursor, range/attempt, query-log, and range-event evidence persistence around the existing archive and operation identities.
• Separate known Found IDs from validated archived bodies for reconciliation.
• Record exact scope/strategy/limit, sent/completed times, EOSE/NEG completion, NOTICE messages, errors, timeouts, disconnects, and counts per query/attempt.
• Replace TTL-based coverage with a provenance-backed latest-applicable-evidence projection. Failed queries retain prior facts; invalid observation metadata is reported separately instead of silently becoming Unknown.
• Preserve existing signed events, accounts, publication versions, and operation history. Label legacy observations honestly; do not fabricate missing query data.
• Preserve or explicitly transition legacy partial checkpoints without changing their limit/strategy silently or discarding the archive.

Exit: repeat-safe migration preserves old data; an event can have independent observations in multiple ranges/runs; partial and newer unrelated scopes do not overwrite older evidence; projection can be rebuilt from recorded provenance.

2. Ordinary paging and incremental fetching

• Add configurable page size, default 100, saved with each run.
• Replace normal split-window backfill with inclusive-earliest until paging, boundary-ID deduplication, and explicit stalled-page detection.
• Persist history completion, fixed run bounds, latest successful forward boundary, active paging cursor, and request outcomes independently per relay.
• After history completes, default to since = successful boundary - 10s and a fixed current run-start until; advance only after the full interval completes.
• Resume unfinished work with its original bounds/settings; leave failed or saturated scopes partial without skipping same-timestamp events.
• Add Recheck history and scoped Refetch as new observations of previous scopes.
• Replace ambiguous received totals with distinct Found/Fetched/newly archived counts; keep repeated delivery counts in query details.

Exit: repeated runs avoid unnecessary full backfills; interrupted snapshots resume without jumping forward; timestamp overlap does not duplicate archive bodies; rechecking can discover newly imported old events; short-page completion records its relay-limit assumptions rather than claiming universal completeness.

3. NIP-77 reconciliation

• Select a compatible Negentropy implementation and assess dependency/runtime integration before adding it to this independent plugin.
• Implement bounded NEG-OPEN/NEG-MSG/NEG-CLOSE negotiation and record NEG-ERR, unsupported responses, version negotiation, timeout, and cancellation outcomes.
• Compare local eligible timestamp/ID sets for the exact selected scope; persist remote-only/common/local-only evidence with run/range provenance.
• Download and validate remote-only bodies through REQ. Never upload to extra sources or automatically publish during Fetch.
• Use ordinary paging for unsupported relays. Failed reconciliation cannot establish absence; fragmented/scoped reconciliation must record completeness.
• Share range/cursor/UI contracts between NIP-77 and fallback strategies.

Exit: differences in both directions are correctly attributed; common bodies are not redownloaded unnecessarily; same-timestamp IDs reconcile; unsupported relays still fetch; partial/error scopes never generate unsupported missing conclusions.

4. Range Coverage and automatic comparison

• Replace the global state table with relay -> range -> kind/date/evidence tree.
• Display Beginning/date bounds, unique Found count, Fetched/Not fetched count where relevant, Complete/Partial status, and checked-at/attempt details.
• Put Refetch/Resume in each range's ⋮ menu with active-run disabled states.
• Expose Recheck history separately from default recent Fetch.
• Automatically recompute Sync candidates after committed applicable evidence; no additional Compare button and no automatic publication.
• Use completed comparison scopes to establish absence; never infer it from a limited page, failed query, age, or an unrelated incremental interval.

Exit: the same UI works for both transports; scope history remains inspectable; partial scopes are not displayed as complete empty ranges; multiple overlapping observations do not inflate the unique archive total.

5. Eligible missing-copy Sync and verification

• Define replay eligibility for deletion requests, expiration, superseded replaceable/addressable events, and private/ephemeral exclusions.
• Render per-write-relay missing-copy groups by kind/year/month; default to all eligible confirmed-missing copies with optional relay/kind/date narrowing.
• Snapshot selected candidates/scopes and persist resumable event/relay deliveries.
• Publish unchanged signed events only on explicit Sync; distinguish Pending, Accepted/already stored, Rejected, Verified, and Unverified with reasons.
• Read back corresponding IDs, validate bodies, and append verification evidence. Resume unfinished delivery/verification without resending verified copies blindly.

Exit: extra/read-only sources receive no sync uploads; one event missing on two destinations counts as two deliveries; acknowledgement alone is not verification; rejection and inconclusive read-back are separate; archive-only historical versions are not accidentally replayed as current events.

6. Integration review and documentation

• Update command metadata, registration, README, and current architecture as each milestone becomes implemented; keep remaining items unchecked.
• Perform targeted lint/static verification and narrowly scoped isolated checks appropriate to migrations, paging boundaries, reconciliation, and delivery state.
• Obtain user review of live relay behavior and the timeline/range/overflow UI. Browser checks require an explicitly confirmed authenticated context.

No new test files or test-suite runs are included unless requested by the user.

Deferred scope

• Specialized-relay explicit-kind fallback/capability policy (postponed by user).
• Individual archive selection, Select all/Clear selection, and JSONL export/import.
• Relay authentication and optional preferred-signer references.
• Creation of deletion requests, media-file backup, and unattended scheduling.

Historical implementation record

First timeline iteration

• Persist accounts as independently validated JSON documents keyed by public key.
• Keep archive facts and relay observations independent of presentation.
• Replace list scaffold with compact account, relay, event tree, and operations sections.
• Support account/relay edits, profile and NIP-65 discovery, and measured connection checks.
• Initial sample design mode was provided, then removed during UI iteration.
• Update documentation, regenerate command registration, and lint changed files.
• User review of the rendered timeline design and real relay network behavior in authenticated AppWeaver.

Verification: targeted ESLint passed; TypeScript reported zero diagnostics in the plugin; a one-off in-memory check validated empty/preview/populated WebNode schemas, npub account creation, normalized relay updates/removal, invalid URL rejection, account deletion, and preview non-persistence. No browser or live relay checks were run.

Subsequent iterations

Account selection convenience

• Add browser-authenticated and named bunker public-key choices to Add account, alongside manual npub entry.
• Verify the reusable select source and update documentation.

Verification: targeted ESLint passed; changed-file TypeScript checks reported zero diagnostics; a one-off schema check confirmed the dynamic public-key picker and independent manual npub form in list. Authenticated browser review remains pending.

Compact layout iteration

• Remove Reload accounts and add-account Cancel; use + Account to toggle the section.
• Put account selection/manual input and their submit buttons on shared rows.
• Align relay headers/body with explicit element cells on identical grid tracks.
• Remove design-preview controls, command option, fixtures, and sample sync/progress renderers.
• Update docs/registration and verify changed files.

Verification: targeted ESLint and changed-file TypeScript checks passed; registration generation completed. A one-off in-memory schema/layout check verified empty/populated list output, six element cells per header/body row, inline account submit controls, and removal of the preview command option and render paths. Visual review in authenticated AppWeaver remains pending.

Account actions and automatic refresh

• Move account removal beside + Account.
• Move fetched-list, JSON configuration, and extra recovery sources to Sync.
• Automatically refresh profile/NIP-65 snapshots and relay connection observations in the background, coalescing concurrent requests and caching results.
• Update documentation and verify the new layout and refresh lifecycle.

Verification: targeted ESLint passed; plugin TypeScript diagnostics were zero. A one-off in-memory check with stubbed network responses validated concurrent refresh coalescing, five-minute cache reuse, URL-change invalidation, profile and connection observations, preservation of local settings, polling stop on completion, DM list without networking, and visible control placement. Live relay and browser review remain pending.

Account label and alignment cleanup

• Remove the fixed Watch-only identity label and use Add account wording.
• Center controls in both add forms and the account selector to prevent Open stretching.
• Verify the renderer and update the UI documentation.

Verification: targeted ESLint passed; static inspection confirmed the identity label removal and centered flex alignment for account add/selector forms.

Agreed fetch / coverage / sync structure

• Separate Relays, Extra fetch sources, Fetch, Coverage, Sync, and Operations in list.
• Derive fetch sources from configured NIP-65 write relays plus extras; derive sync destinations from write relays only.
• Add timestamped present/missing/unknown persistence with an additive migration of existing positive observations.
• Query per-relay coverage and missing relay/kind/year/month groups without counting unknown/stale records as missing.
• Render missing event copies per destination relay and move shared JSON configuration to account settings.
• Update docs and verify persistence/query and renderer contracts.

Verification: targeted ESLint passed and plugin TypeScript diagnostics were zero. A one-off in-memory SQLite check validated additive/repeated migration, existing account/archive retention, write-only scopes, per-relay duplicated-copy counting, stale/future/unknown classification, monotonic observation writes, independent source/relay edits, and the ordered WebNode section/tree schema. No live relay or authenticated browser checks were run.

The initial UI and presence-query foundations were arranged here. The archive and NIP-65 publication actions were implemented in the following section. Their next fetching/coverage changes are tracked in the active milestones above.

Enable publication and backup actions

• Enable Publish NIP-65 using the generic signer picker restricted to the selected account.
• Validate signed relay lists server-side and publish with per-relay acceptance/error results.
• Prevent older discovery responses from overwriting newer published snapshots.
• Rename Load published relay list to Reset to published relays and explain the local replacement.
• Implement signature-verified public-event backup with bounded windows, durable checkpoints, partial recovery, and resume.
• Display real operation progress and archive counts with polling while running.
• Verify isolated signing/publication and archive flows; update docs/registration.

Verification: targeted ESLint passed, changed-module TypeScript diagnostics were zero, and command registration generation completed. A one-off in-memory check using actual signed events and stubbed relay transport verified invalid-signature/ wrong-author/private-kind filtering, archive deduplication, observed presence, partial backup resume, concurrent-run rejection, interrupted-run recovery, selected-account signer restrictions, signed-list/config validation, per-relay publication outcomes, stale snapshot protection, and enabled widget actions. Live relay and authenticated browser review remain pending.

Initial event-engine backlog (superseded by active milestones)

• Archive public authored events from configured NIP-65 write relays plus extra fetch sources with bounded/resumable fetching and signature validation.
• Kind/year/month lazy browsing and archive selection remain deferred; sync scope narrowing is tracked in milestone 5.
• Mandatory per-ID presence queries are superseded by completed NIP-77/range comparisons. ID requests remain useful for downloading bodies/read-back.
• Missing-copy delivery/read-back is tracked in milestone 5; automatic age expiry is superseded by provenance-backed observations in milestone 1.
• JSONL export/import remains deferred.
• Integrate authenticated/bunker account selection for NIP-65 signing and publication.
• Relay authentication/preferred-signer references remain deferred.
• Replay semantics are tracked in milestone 5; creating deletion requests remains deferred.

The current iteration includes local configuration, public-event backup, and NIP-65 publication. Existence checks, event selection/export, and sync are still pending. No dock widget registration. Runtime restart must be done by the user outside the active chat.