Skip to content

fix(telemetry): consistent invocation ID per run and telemetry acceptance tests#271

Merged
leggetter merged 8 commits intomainfrom
chore/telemetry-repro-v2
Mar 19, 2026
Merged

fix(telemetry): consistent invocation ID per run and telemetry acceptance tests#271
leggetter merged 8 commits intomainfrom
chore/telemetry-repro-v2

Conversation

@leggetter
Copy link
Collaborator

Summary

  • Fix: Set invocation ID only when empty so all API requests in one CLI run share the same invocation_id (fixes multiple IDs when gateway PreRun called ValidateAPIKey and connection PreRun overwrote telemetry).
  • Telemetry ATs: Add acceptance tests that run the CLI through a recording proxy and assert every API request sends the same command_path and invocation_id (helpers: StartRecordingProxy, AssertTelemetryConsistent). Run in slice 1 with request and event (telemetry build tag).
  • Cleanup: Remove TestTelemetryThreeRequestsOneCommandPath (the three-request / three-command_path bug is not reproducible with the current CLI).

Commits

  1. fix(telemetry): consistent invocation ID per run and add telemetry test infra — root fix, unit test, recording proxy, telemetry_test.go and listen_test.go telemetry tests, README.
  2. chore(acceptance): run telemetry tests in slice 1, remove unreproducible test — tag telemetry tests with telemetry, add to slice 1 in CI and run_parallel.sh, remove unreproducible test.

Made with Cursor

…st infra

- root.go: set invocation ID only when empty so all API requests in one CLI run
  share the same invocation_id (fixes multiple IDs when gateway PreRun calls
  ValidateAPIKey then connection PreRun overwrote telemetry)
- pkg/cmd: add TestInvocationIDPersistsAcrossMultipleInitTelemetryCalls (fails
  without fix, passes with it)
- test/acceptance: recording proxy (StartRecordingProxy, AssertTelemetryConsistent,
  RunListenWithTimeout, RunWithEnv, NewCLIRunnerWithConfigPath/NoCI), telemetry_test.go
  proxy-based tests, TestTelemetryInvocationIDConsistentWhenValidateAPIKeyInPreRun,
  TestTelemetryListenProxy in listen_test.go, README section

Made-with: Cursor
…amples

Update Example strings in connection create, delete, get, list, and upsert
so help text shows the preferred gateway-scoped form instead of the
deprecated root-level 'hookdeck connection'.

Made-with: Cursor
Run go run ./tools/generate-reference after connection example updates
so REFERENCE.md reflects hookdeck gateway connection in examples.

Made-with: Cursor
…ble test

- Tag telemetry_test.go with 'telemetry' (was 'connection'); run in slice 1 with request, event
- CI and run_parallel.sh: add telemetry to slice 1 tags
- Remove TestTelemetryThreeRequestsOneCommandPath (three-request bug not reproducible with current CLI)

Made-with: Cursor
- Add telemetry_listen_test.go (//go:build telemetry) with TestTelemetryListenProxy
- Remove test from listen_test.go so matrix slices with telemetry disabled never run it
- CI/run_parallel: single go test -tags=telemetry (no separate listen invocation)
- README: document telemetry-tagged proxy tests

Made-with: Cursor
- Set HOOKDECK_CLI_TELEMETRY_DISABLED=0 so repo/org defaults cannot disable headers
- AssertTelemetryConsistent: clearer fatal when all telemetry headers empty
- run_parallel.sh and README match CI

Made-with: Cursor
…y only

- Add //go:build connection|destination|source to *_config_json* and connection_rules_json tests
- Untagged *_test.go files were included in go test -tags=telemetry (acceptance-telemetry CI)
- README + AGENTS: document that untagged tests also run in the telemetry job

Made-with: Cursor
- Run, RunWithEnv, RunFromCwd: up to 4 attempts, 2s backoff
- Detect 502 via CLI error patterns only (not 503/504)
- Log each retry and final give-up with output excerpts
- Document in README; add basic-tagged detection tests

Made-with: Cursor
@leggetter leggetter merged commit 2d1574d into main Mar 19, 2026
12 checks passed
@leggetter leggetter deleted the chore/telemetry-repro-v2 branch March 19, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant