I built a model that rates careers like stocks — then caught it lying, and fixed it.
by Ashley Libasci — Math + CS @ UIUC · GitHub · LinkedIn
CareerStar scores the viability of a career in an AI-shaped economy as a single 0–100 risk-adjusted number — the way a portfolio weighs an asset’s return against its risk. It’s a solo summer project, built with an AI-native workflow and deployed on AWS. This page is the story of how it works and why I trust it.
What is this page & how do I use it?
This is the story behind the product — how the model was designed, the bug I caught in my own methodology (and chose to disclose rather than hide), and the evidence that the scores hold up. It’s written for anyone deciding whether to trust the site, including recruiters kicking the tires.
How to use it: read top to bottom like an essay — or jump straight to the pipeline diagram if you just want to see how a number gets made.
A sequel, not a first draft
My freshman project was an AI stock-recommendation engine — predictive models, risk profiles, back-testing. CareerStar is the same machine pointed at a harder, more personal question: rates stocks → rates careers. The framing that makes it work is that a career, like an asset, isn’t just risky or safe — it’s risk-adjusted: a field can be exposed to AI and still be worth pursuing if its growth, pay, and fit are strong enough.
Every number is math, not an LLM
A pure, deterministic scorer is the single source of truth. Growth and pay are percentile-ranked across all 730 occupations; risk blends AI-exposure research with a constructed volatility proxy; the two combine into a risk-adjusted value, then blend with personal fit:
The actual equations
Return = wGrowth·growth + wPay·pay Risk = wExposure·exposure + wVolatility·volatility RAV = Return · (1 − γ·Risk) Score = 100 · [ α·RAV + (1 − α)·Fit ]
The language model never computes a number. It only writes the plain-English “why,” from the already-computed score — and it never even sees your raw text, only controlled tags. The math is auditable; the AI is a narrator.
I caught my own model overclaiming — and fixed it
So I rebuilt it for real. Each occupation now carries a genuine 68-dimensional O*NET capability vector — 35 skills + 33 knowledge areas — and Fit is cosine similarity in that space, weighted by how distinctive each capability is across the labor market (a per-dimension z-score), because otherwise the skills every job shares drown out the ones that actually differentiate them. The redirect to a “stronger adjacent path” is a true nearest-neighbor in that same space.
The point isn’t that I wrote a bug. It’s that I went looking for where my own model was weakest, found it, and made the claim true instead of quietly deleting it. Don’t take my word for it — the full reconciliation document is committed in the repo, verdict by verdict.
Back-tested against a real decade
The strongest evidence isn’t a formula — it’s history. I recovered the 2014–24 BLS projections vintage from the Internet Archive, scored the 2014 labor market with today’s model, and compared against what actually happened by 2024 — 647 occupations across a real decade. The score tracked reality (rank correlation ρ = 0.39; it flagged 48% of the careers that actually declined vs a 33% base rate). And the part a marketer would cut: the AI-risk adjustment added nothing for that decade (exposure ρ ≈ 0.1) — exactly as it should, because 2014–24 was a pre-LLM decade and exposure is a forward-looking bet. The misses are published by name, including the oil-crash casualties no occupation model catches.
Robust to ±20%, and the risk axis isn't redundant
Robustness. The weights are a modeling choice, so the fair question is whether the answer survives disagreement about them. Every comparison is re-scored across 729 weightings — every weight moved ±20% on a fixed grid — and the app reports how often each career keeps its rank, flagging a “close call” instead of selling a shaky #1 as certain.
Validation.A fair objection is that AI-exposure just restates “declining.” It doesn’t: across all 730occupations, exposure and projected growth are almost uncorrelated (Spearman ρ = 0.075; exposure-quartile growth is flat — Q1 0%, Q2 1.6%, Q3 3.6%, Q4 1.3%). So the risk axis carries information growth doesn’t — which is why a risk-adjusted score beats ranking on growth alone, and why “exposure ≠ displacement” is literally true in the data.
Five models, not one — because a formula is an opinion
The deepest critique of any scoring app is: who says your formula is right? My answer is to stop pretending there’s one formula. Every comparison is also scored under four rival models — a growth maximalist that ignores AI risk entirely, a moat-first defensive model, a Sharpe-style efficiency ratio, and a naive equal-weight control — and the app shows the spread. When all five agree, the ranking isn’t an artifact of my assumptions. When they crown different winners, the app says so, because that disagreement is the honest finding: it means the answer turns on how much AI risk you believe in, and no formula can settle that for you.
(The equal-weight control is there on purpose: a clever model that can’t beat 1/N isn’t earning its complexity — a lesson borrowed from portfolio research.)
Solo, one summer, AI-native, on AWS
- One person, one summer — from brief to PRD to architecture to build, using a structured AI-native (BMAD) workflow.
- Real public data: U.S. BLS Employment Projections, O*NET 29.0 (skills + knowledge), Eloundou et al. AI-exposure, and College Scorecard for the education-ROI layer.
- Deterministic & stateless: no database, no accounts, no stored input — the whole model is a committed data file + pure functions.
- Deployed on AWS Amplify with CI/CD, HTTPS, security headers, rate limiting, and prompt-injection-resistant LLM use.