SKIP TO CONTENT

// METHODOLOGY

How the State of Agent Engineering dataset was built

Everything needed to reproduce the 119-posting, 32-employer preliminary snapshot. Same compliance-first approach as the FDE study: only sources whose own terms permit collection, extracted facts and link-backs rather than republished text, and a deterministic coding rubric anyone can re-run.

1. Sources, and why only these three

Data came from exactly three public, unauthenticated ATS job-board APIs, chosen because each operator's own documentation permits fetching public postings:

  • Lever Postings API - Lever's own docs state published jobs "may be scraped by third parties."
  • Greenhouse Job Board API - "Job Board data is publicly available, so authentication is not required for any GET endpoints."
  • Ashby Public Job Posting API - a documented public, no-auth endpoint for currently published postings.

Deliberately excludedbecause their terms prohibit collection: LinkedIn, Indeed, Wellfound/AngelList, YC Work-at-a-Startup, Google for Jobs, and any re-scraped "open" dataset that launders one of those. Lever's robots signalai-train=no is honored: this data is for statistical/reference analysis, never model training.

2. Collection

A hand-curated seed list of roughly 170 mostly US, venture-backed, AI-native and infrastructure companies, extending the seed list used for the FDE study with AI-agent-tooling and applied-AI companies (model labs, agent-framework vendors, evals platforms, LLM-infra startups). Each company's board slug was resolved from its public careers URL, then its one public JSON endpoint was fetched once per ATS, rate-limited to no more than one request per second, with an identified User-Agent. No login, no paywall bypass, no IP rotation. 504 requests were made against 507 candidate (company, ATS) pairs; 102 boards returned a non-empty public job list. The full per-board probe log ships alongside the dataset.

COLLECTION PIPELINE
Flow diagram: 7 stepsFlow diagram: 7 steps. Seed list, then Resolve board slugs. Resolve board slugs, then Fetch public ATS JSON. Fetch public ATS JSON, then 102 boards returned jobs. 102 boards returned jobs, then Title gate. Title gate, then Temperature-0 coding rubric. Temperature-0 coding rubric, then 119 rows, 32 employers.1Seed list~170 mostly US, venture-backed,AI-native and infra companies, curatedby hand.2Resolve board slugsFrom each company's public careersURL. No login, no paywall bypass, noIP rotation.3Fetch public ATS JSON504 requests across 507 candidate(company, ATS) pairs, <=1 req/sec,identified User-Agent.4102 boards returned jobsThe rest were empty, moved, or not onGreenhouse/Lever/Ashby.5Title gateAgent / AI-engineer title familyonly. ML Engineer and Data Scientistare out of scope.6Temperature-0 coding rubricAgent-framework, evals and autonomysignals coded deterministically,auditable to text spans.7119 rows, 32 employersExtracted fields and link-backs only.No verbatim posting prose.
Every step is deterministic and logged. The per-board probe log ships with the dataset so the request counts below can be checked independently.

3. Inclusion and exclusion

Included if the posting was live and its title matched the agent/AI-engineer-family lexicon: AI Engineer, Applied AI Engineer, Agent Engineer, Forward-Deployed AI Engineer, Prompt Engineer, or an evals/AI-safety specialist title.

Excluded: generic Software Engineer or ML Engineer postings with no AI-agent-family title match, pure data-science/research-scientist roles, internships, expired postings, and duplicate cross-posts. Deduplication is keyed on canonical employer name (not raw board slug) plus normalized title and location, so a company listing the same role under two slug aliases or on two ATS platforms is counted once, not twice.

4. The agent-signal coding rubric

Unlike the FDE study's five-axis authenticity score, this study codes four independent binary signals per posting, each keyed to a specific regex-matched vocabulary and auditable back to the text span that triggered it:

SignalTriggered by
Agent framework / agentic patternLangChain, LangGraph, CrewAI, AutoGen, AutoGPT, LlamaIndex, “agentic,” “multi-agent,” tool use, function calling, orchestration
Production codinglanguage/stack requirements consistent with shipping production software, not aptitude-only screening
Evals / red-teaming / guardrails“evals,” “red-team,” “guardrails,” safety-review language
Autonomy / human-in-the-loop“autonomy,” “human-in-the-loop,” “escalation,” oversight language

LLM-stack mentions (RAG, OpenAI API, Anthropic/Claude, vector databases) and programming languages are coded the same way, as independent keyword matches, not a composite score. All coding is deterministic and temperature-0 - the same input text always produces the same output - and not human-adjudicated for every row, so spot-checking is recommended.

5. What gets published (and what doesn't)

Published: aggregate statistics, a per-posting fact table of extracted attributes, and a link back to each live posting. Notpublished: verbatim job-description prose (that is the employer's copyright), and anything from a prohibited source. Extracted facts plus a link are citable without republishing anyone's creative text.

6. Limitations

This is agent-engineering hiring among the AI-native and infrastructure companies in the seed list, on public ATS boards, not the whole market. At 119 postings and 32employers, this round is smaller than the FDE study's 519/57 minimum-viable bar - read every percentage here as directional, not precise, until a larger round runs. It is a point-in-time snapshot (open roles, not hires), so trend claims need a second edition. This corpus is never blended with the 519-posting FDE dataset at /state-of-fde- different title family, different collection round. Full limitations are listed on the report.

▸ BACK TO THE REPORTDOWNLOAD THE DATASET