AI Apply — Michael Evans

Overview

Fully AI-Orchestrated Build

AI Apply is a multi-tenant SaaS designed to remove the most repetitive part of a job hunt. A user signs in, points the app at their CVs, defines one or more job-search criteria, and clicks Run. A background worker scrapes the chosen boards (Seek AU/NZ, Wanted KR), picks the best CV per listing, asks Claude to draft a tailored cover letter, and stores each one as an editable draft in the Apply UI. Drafts never leave the app — there is no auto-submit; the candidate still reads, edits, and marks each one sent.

The entire codebase — FastAPI backend, SQLAlchemy data model, vanilla-JS frontend, worker pipeline, three OAuth flows, Stripe billing with idempotent webhook crediting, envelope-encrypted secret store, Bicep infrastructure templates, and the GitHub Actions deploy pipeline — was orchestrated through Claude. Architecture, invariants, and product direction were author-led; every line of implementation was AI-driven. The one piece that was hands-on was provisioning and wiring the Azure environment itself: resource group, App Service, Postgres Flexible Server, Key Vault, container registry, federated identity for the deploy workflow, and OAuth client registrations across GitHub, Google, and Microsoft Entra.

Architecture

FastAPI + Background Pipeline

⬡

FastAPI API Layer

One router per concern — auth, settings, runs, applications, billing — with a single current_user dependency that handles both real OAuth sessions and the local-dev auto-login path. The static frontend mounts alongside; no build step.

⬡

Worker Pipeline

Framework-free orchestrator: scrape → dedupe against the user's job table → load CVs from the upload table or GitHub → call Claude with a strict cover-letter contract → debit tokens → save draft. Lives in BackgroundTasks today; the orchestrator is self-contained so it can be lifted onto Container Apps Job or Storage Queue without touching the API.

⬡

Azure Hosting

App Service (Linux Python) running gunicorn, Postgres Flexible Server, Key Vault for user secrets via the App Service managed identity, ACR for container builds. All provisioned by a single Bicep template; deploys on push to main via federated-identity GitHub Actions.

⬡

Local Dev Mode

Same codebase runs single-user against SQLite with no auth, an envelope-encrypted secrets sidecar, and (optionally) the local claude binary so generations consume the developer's Claude subscription instead of an API key. One command: python main.py.

Billing

Three Billing Modes, One Worker

Tokens (hosted default) — Anthropic calls go via the host's API key; per-call cost is debited from the user's centitoken balance. Stripe Checkout funds top-ups; the webhook credits idempotently against a unique stripe_session_id so replays and concurrent /checkout/verify calls can't double-credit.
BYOK — the user supplies their own sk-ant-... key. Stored envelope-encrypted (Fernet locally, Key Vault in production) behind an opaque ref; no balance touched.
System (local only) — generation routes through the local claude binary as a subprocess. Rejected by the API in hosted mode.

Token balances are stored as integer centitokens (×100) and gated by a credit/debit ledger so every change is auditable. Webhook idempotency, race-safe credit, and the local/hosted gate are all covered by tests in the pytest suite.

Technical

Invariants & Implementation

Single mode flag (SECRETS_BACKEND != "azure-key-vault") drives every local-vs-hosted behaviour: auth, billing, generation source, secrets backend. One source of truth, exposed to the frontend via GET /api/config.
OAuth across three providers — GitHub, Google OIDC, and Microsoft Entra email/password — with email as the canonical account ID and signup-provider lock to prevent account takeover via a different login route.
Cover-letter contract bans markdown and meta-commentary but deliberately does not blacklist "Claude" or "Anthropic" — those legitimately appear in letters for jobs at Anthropic; the human review step is the safety net, and a regression test enforces the boundary.
User secrets (Anthropic key, GitHub token) never sit in the DB as plaintext — only an opaque ref like kv:anthropic-42. The SecretStore abstraction has Fernet (local) and Key Vault (production) implementations behind a common interface.
Background runs are recovered after restart via a boot-time recover_orphaned_runs() sweep over live-status rows so a process death mid-run can't strand a user's batch.
Two-phase scraper architecture (per-site adapters behind a single registry) already supports Seek AU/NZ and Wanted KR; adding a new site is a four-file change.

Python 3.11 FastAPI SQLAlchemy 2.0 Anthropic SDK Azure App Service Postgres Key Vault Bicep Stripe OAuth / OIDC Pytest GitHub Actions

AI Orchestration

What Was Hands-On vs Claude-Driven

The project was a deliberate test of running a complete shipped product end-to-end through AI-orchestrated development — not just code generation, but architecture decomposition, invariant enforcement, test-suite design, deployment scripts, and the CI pipeline that builds and ships it.

AI-driven (everything else): the FastAPI app, SQLAlchemy models, worker pipeline, scraper integration, Anthropic SDK call, Claude CLI subprocess path, three OAuth flows, Stripe Checkout + webhook, token ledger, envelope-encrypted secret store, vanilla-JS frontend, pytest suite, Dockerfile, gunicorn entrypoint, Bicep templates, and the GitHub Actions deploy workflow.
Hands-on (Azure setup): standing up the resource group, configuring App Service / Postgres / Key Vault / ACR, registering the OAuth applications across GitHub, Google, and Entra, wiring federated identity for the deploy workflow, and verifying that managed identity correctly resolves against Key Vault.
Author-led (across both): architecture, invariants (anonymity contract, billing idempotency, secret handling, the local-vs-hosted gate), product scope decisions, and review of every change before merge. The repo's CLAUDE.md encodes those invariants so AI agents enforce them on every change rather than re-discovering them.