JA Pipeline Observatory JobAutomation / public telemetry
System state Loading telemetry

System primer

How the daily pipeline works

The system finds company job boards, extracts recent US openings, then runs two independent branches: a broad public board and candidate-specific fit scoring.

How to read the numbers: “This run” is activity from the latest completed stage run; “current total” is the latest stored or published inventory; rolling metrics state their time window.

Telemetry generated
Latest successful stage
Combined stage runtime
Snapshot ID

30-day run history

Daily pipeline activity

New postings New assessments

Independent review

Gemma usage

Current period
01

Latest discovery run

Job Discovery

View source and board diagnosticsDiscovery sources · new additions

Discovery sources

New board additions

02

Latest extraction run

Job Extraction

View ATS reliability diagnosticsFailure rate · error taxonomy

Board reliability by ATS

ATS familyBoards selectedBoards failedFailure rateSignal

Error taxonomy

03

Latest public-board build

Public Discovery Board

View enrichment diagnosticsFive model passes · cache · descriptions

Five-pass enrichment

Cache and description intake

04

Latest market-analysis run

Market Analysis

View Spark pipeline diagnosticsStage timings · data quality · deduplication

Spark stage timings

Aggregate output

05

Latest assessment run

Relevancy Assessment

View model and sync diagnosticsRetries · fallbacks · tokens · calls

Assessment reliability

StepAttemptedFailedFallbackError rate

Token and call ledger