digest stale-open nudge reports on RFIs whose live fetch always fails — Procore RFI detail decode error (15/24h) #15

Open
opened 2026-11-02 17:54:16 +00:00 by gal-bot · 0 comments
Owner

Problem

The digest task's stale-open nudge is reporting against RFIs it cannot fetch live. Every fetch fails with a body-decode error, and the design deliberately proceeds on the stale local copy:

WARN constructionbot::digest: stale-open live fetch failed — keeping (nudge, don't suppress) rfi_id=183681
     error=procore decode error: rfi detail: error decoding response body
...
INFO constructionbot::digest: stale-open nudge sent count=16

Observed 2026-11-02 (last 24h window, tenant gal, project 361980):

  • 15 occurrences of procore decode error: rfi detail: error decoding response body
  • 1 stale-open nudge sent count=16
  • 2 procore 429 — backing off (unrelated, normal rate limiting)

Why it matters

The nudge path intentionally fails soft — "keeping (nudge, don't suppress)" — because silently dropping a stale item is worse than nudging on possibly-stale data. That trade is defensible for an occasional failure. It is not defensible as a permanent steady state: if every live fetch fails, the digest is nudging people about the state of 16 RFIs using local data that may no longer match Procore.

The decode failure is on the RFI detail endpoint (GET /rest/v1.0/projects/{id}/rfis/{rfi_id} shape), not the list endpoint. Most likely a Procore API response-shape change: a field added as an incompatible type, a null where a value was expected, or a renamed key. The 429s are incidental and not the cause.

What to find out

  1. Capture the actual response body for one failing rfi_id and diff it against the struct the app deserializes into. The error is error decoding response body — serde's message with detail would name the field; the current log line drops it.
  2. Fix the deserializer (likely: make the drifted field optional or accept both shapes).
  3. Decide whether the nudge should degrade differently when every fetch in a run fails: nudging 16 items on 0/16 live evidence arguably warrants a distinct signal (or a skip) rather than the same "don't suppress" path.
  4. Improve the log line to include serde's field-level cause, so the next drift is diagnosable from logs alone.

Scope note

Nothing is submitted to Procore by this path — the nudge is an internal digest notification. This is a data-fidelity and diagnostics defect, not a write-safety one.

Verification

  • Re-run the digest; confirm procore decode error count returns to 0 and stale-open nudge still sends for genuinely stale items.
  • Confirm a genuinely malformed response still fails soft (does not crash the digest task).
## Problem The digest task's stale-open nudge is reporting against RFIs it cannot fetch live. Every fetch fails with a body-decode error, and the design deliberately proceeds on the stale local copy: ``` WARN constructionbot::digest: stale-open live fetch failed — keeping (nudge, don't suppress) rfi_id=183681 error=procore decode error: rfi detail: error decoding response body ... INFO constructionbot::digest: stale-open nudge sent count=16 ``` Observed 2026-11-02 (last 24h window, tenant `gal`, project 361980): - **15** occurrences of `procore decode error: rfi detail: error decoding response body` - **1** `stale-open nudge sent count=16` - **2** `procore 429 — backing off` (unrelated, normal rate limiting) ## Why it matters The nudge path intentionally fails soft — "keeping (nudge, don't suppress)" — because silently dropping a stale item is worse than nudging on possibly-stale data. That trade is defensible for an occasional failure. It is not defensible as a permanent steady state: if every live fetch fails, the digest is nudging people about the state of 16 RFIs using local data that may no longer match Procore. The decode failure is on the RFI **detail** endpoint (`GET /rest/v1.0/projects/{id}/rfis/{rfi_id}` shape), not the list endpoint. Most likely a Procore API response-shape change: a field added as an incompatible type, a null where a value was expected, or a renamed key. The 429s are incidental and not the cause. ## What to find out 1. Capture the actual response body for one failing `rfi_id` and diff it against the struct the app deserializes into. The error is `error decoding response body` — serde's message with detail would name the field; the current log line drops it. 2. Fix the deserializer (likely: make the drifted field optional or accept both shapes). 3. Decide whether the nudge should degrade differently when *every* fetch in a run fails: nudging 16 items on 0/16 live evidence arguably warrants a distinct signal (or a skip) rather than the same "don't suppress" path. 4. Improve the log line to include serde's field-level cause, so the next drift is diagnosable from logs alone. ## Scope note Nothing is submitted to Procore by this path — the nudge is an internal digest notification. This is a data-fidelity and diagnostics defect, not a write-safety one. ## Verification - Re-run the digest; confirm `procore decode error` count returns to 0 and `stale-open nudge` still sends for genuinely stale items. - Confirm a genuinely malformed response still fails soft (does not crash the digest task).
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gal/constructionbot#15
No description provided.