Accuracy registry
Verification & the open eval suite
Language models can't do deterministic financial math. This is the open benchmark that measures it — and the registry of exactly what Pascaline has verified.
The open eval benchmark
The Pascaline Deterministic Tax Eval Suite is a public set of defining U.S. tax, estate, and benefit scenarios — each paired with its correct, statute-cited answer, produced by the same pure engine Pascaline serves. Run it against any calculation tool or language model to measure real accuracy on the cases that matter: the cliffs, phase-outs, and edge cases that are easy to get plausibly — and expensively — wrong.
- Download: eval-suite.json — 35 cases, CC‑BY‑4.0.
- Run it against the live API, or any endpoint that returns the same
{ "result": … }shape:
npx tsx scripts/evals/run.ts --base https://pascaline.io # → 35/35 passed (100%). Point --base at any tool or model proxy to compare.
Each case is a POST to its endpoint with a JSON request; the target's result is compared field-by-field to the cited ground truth. Pascaline scores 100% by construction — the informative number is how a general model scores on the same suite.
Covered scenarios
| Scenario | API | Why it's hard | Cites |
| Massachusetts full cliff — $12M estate | state-estate-tax | MA taxes the ENTIRE estate once it exceeds the threshold, not just the excess. The single most common six-figure error in estate-tax tools. | 1 |
| Oregon excess-over-$1M — $1.1M estate | state-estate-tax | Oregon is NOT a cliff state: ORS 118.010(4) taxes only the amount above $1,000,000 via a graduated table, so a $1.1M estate owes 10% x $100,000 = $10,000 — not tax on the whole estate. Corrected 2026-08-18 (GH #27); the prior expectation of $111,500 encoded a modelling bug. | 2 |
| New York 105% cliff — $8M estate | state-estate-tax | NY phases out the exemption between 100% and 105% of the threshold; above 105% the whole estate is taxed. A 3-zone cliff almost no tool models. | 1 |
| Zeroed-out GRAT — $2M, 5yr, §7520 4.6% | grat-grut | The GRAT annuity that produces a ~$0 taxable gift depends on the §7520 rate and the Rev. Proc. 2019-44 annuity factor. Rate pinned to the spec's worked example. | 3 |
| CLAT charitable deduction — $1M, 10yr | grat-grut | The charitable deduction is the annuity times the term-certain factor at the §7520 rate; remainder passes to heirs. | 3 |
| Non-spouse 10-year rule with RMDs | inherited-ira-rmd | A non-spouse beneficiary of a decedent who had reached their required beginning date must take annual RMDs AND empty the account by year 10 (SECURE Act). | 2 |
| ISO exercise-and-hold AMT | iso-amt | Exercising 50,000 ISOs at $1 with an $8 FMV creates a $350k AMT preference item; the tentative minimum tax vs regular tax delta is the AMT owed. | 4 |
| PTET SALT-cap workaround — CA S-corp | ptet-salt | A pass-through entity tax election converts non-deductible personal SALT into a deductible entity expense; the net federal benefit accounts for the QBI interaction. | 3 |
| §199A QBI with W-2/UBIA limits — MFJ $600k | qbi-199a | Above the MFJ threshold the deduction is capped by the greater of 50% of W-2 wages or 25% of wages plus 2.5% of qualified property, computed per business. | 2 |
| §1202 QSBS gain-exclusion eligibility | qsbs | Exclusion depends on original-issue C-corp stock, the $50M gross-assets test at issuance, an active qualified trade, and the holding period. | 2 |
| Social Security PIA at full retirement age | ss-pia | The primary insurance amount applies the bend-point formula to 35 years of wage-indexed earnings; claiming age then adjusts the benefit. | 5 |
| SSDI benefit with workers'-comp offset | ssdi | SSDI plus workers' compensation cannot exceed 80% of average current earnings; the excess reduces the SSDI benefit (the reverse-offset states differ). | 8 |
| Dependent NY student, married parents (AGI $87K) -> SAI 6,890 | fafsa-sai | The spec README's defining family worked through the real 2026-27 Formula A worksheet: parent contribution 6,590 + student assets 300 = SAI 6,890, and AGI just over the 275% Pell ceiling means no Pell despite moderate need. | 7 |
| Canada Express Entry FSW -> CRS 429, PNP path +600 | immigration-points | The defining cross-system scenario: 29yo bachelors, CLB 9, 4 yrs foreign experience passes the 67-point FSW gate at 70 and scores CRS 429 off the official IRCC tables, with a provincial-nomination path jumping to 1,029. | 14 |
| Attorney credential lookup: Kamala Harris, CA bar #146672 (inactive) | credential-verification | The golden real-record scenario: exact-ignoring-middle name match against a live-verified State Bar of California record yields license #146672, inactive status, confidence 0.95, plus the mandatory seed-dataset warning. | 6 |
| FDA enforcement search: India drug manufacturers -> 9 CGMP letters | fda-warning-letter | The defining faceted-search case over the 22-letter seed: country=IN + product_type=drug returns 9 Indian drug-manufacturer warning letters, all faceted as cGMP violations. | 2 |
| SaaS ToS compliance: OpenAI terms of service | saas-tos | The spec README's defining lookup: OpenAI's terms resolve to commercial-use allowed but ML-training-on-outputs and scraping prohibited, attribution-required redistribution, confidence 0.94, with the exact competing-models key clause. | 12 |
| Supplier diversity certifications: Acme Diverse Solutions (3 certs, 0.97) | supplier-diversity | The README defining scenario: exact-name match on a synthetic seed record returns 3 certifications (MBE/WBE/WOSB across NMSDC/WBENC/SBA) at confidence 0.97 with deterministic days-until-expiration derived from the as-of date. | 11 |
| Sponsored YouTube script with no paid disclosure | content-policy | The README defining scenario: a sponsored video script that never says 'paid promotion' is non-compliant with a single required MISSING_PAID_DISCLOSURE violation — the disclosure rule only fires for non-organic use cases. | 6 |
| EU261 long-haul, 215-min delay, ATC (possibly-extraordinary) cause | flight-compensation | The defining scenario: a >3500 km EU-departure flight delayed 3-4h pays 50% (EUR 300, not 600) under Art 7(2)(c)/Sturgeon, and the possibly-extraordinary ATC cause drops confidence to 0.87 — the current-law figure diverges from the spec README's EUR 600. | 18 |
| US9876543 status — 20-year term plus 127 PTA days | patent-expiration | The README defining seed record: active utility patent whose expiration is filing + 20 years PLUS 127 days of §154(b) patent term adjustment (2034-07-20, not the spec's 2034-03-15), with a full maintenance-fee schedule. | 12 |
| LIPITOR (atorvastatin) Paragraph IV settlement entry | generic-entry | The defining seed scenario: a Paragraph IV settlement with an agreed date licenses first-filer entry on 2011-11-30 (overriding patent expiry), and the 180-day first-filer exclusivity runs to 2012-05-28 — pinning the Hatch-Waxman settlement-stacking logic. | 9 |
| $20M estate, MA domicile, surviving spouse | federal-estate-tax | Combines the federal 40% tax on the excess above the $15M exclusion with the Massachusetts full-cliff (tax on the ENTIRE estate, not just the excess) and a portability/DSUE section in one call. | 4 |
| $50K ILIT premium, 3 beneficiaries, $250K corpus | gift-tax-exclusion | The Crummey 5-and-5 hanging-power case: withdrawal rights ($16,666.67 each) exceed the greater-of-$5,000-or-5% lapse limit, so $4,166.67 per beneficiary hangs and accumulates year over year. | 5 |
| NY first spouse, $20M estate, $10M QTIP, 15-year survivor | qtip-marital-trust | The NY cliff makes the two-death total-tax curve non-monotone, so the optimal QTIP election fraction can't be found at the endpoints — it requires a grid search over the sensitivity table. | 5 |
| Married couple, both 62, PIA $3,000 vs $1,200 | ss-filing-strategy | The survivor-benefit asymmetry drives the optimizer to have the higher earner delay to 70 (raising the survivor benefit) while the lower earner claims at 62, across all 81 claiming combinations. | 10 |
| US citizen in Germany, general + passive baskets | foreign-tax-credit | High-taxed passive income triggers the §904(d) high-tax kickout (reclassed into the general basket), and per-basket FTC limits leave only $3,681 of US tax after credits. | 6 |
| Pre-revenue C-corp startup, $1.5M QRE wages | rd-tax-credit | Under 5 years of gross receipts, so the ASC start-up rate applies and the full §280C-reduced credit ($80,100) is taken as a payroll-tax offset spread across four quarters rather than an income-tax credit. | 7 |
| Contaminated backdoor Roth ($200K pre-tax rollover IRA) | form-8606 | Pro-rata rule makes a $7,000 nondeductible-basis conversion 96.6% taxable because a $200K pre-tax IRA contaminates the aggregate; nearly all of the conversion is taxable. | 11 |
| 62-year-old male: $450K lump sum vs $2,400/mo single-life annuity | pension-lumpsum | Mortality-weighted NPV at 6% ($316K) favors the lump sum with a ~age-78 nominal break-even, and the $2,400 benefit sits below the age-62 PBGC cap. | 8 |
| 1968-born Canadian retiring at 65: CPP base + CPP2 enhancement, full OAS | cpp-oas | Combines the contributory-period breakdown (child-rearing dropout + 17% general dropout), CPP2 enhancement math, full 40/40 OAS, and three deferral scenarios in one call. | 8 |
| Mid 3-year bring-forward: $30,000 NCC remaining | aus-super | Bring-forward total cap is locked at the trigger year (3 x $110K = $330K), so $300K used leaves $30K — not $60K under the current $120K cap; getting it wrong costs 47% on the excess. | 14 |
| Office, 45,000 sq ft, 4 stories, 200 spaces, new construction, CA | ada-compliance | Exercises the full requirement stack: Table 208.2 parking (200 spaces → 6 accessible / 1 van, not the README's 8), elevator requirement, 44-in route at occupancy 450, and CA Title 24 overlay. | 33 |
| NY CSSA income-shares, 2 children, with childcare and health-insurance credits | child-support | Income-shares model: 25% basic obligation on $9,700 combined, 67% obligor share, plus obligee-paid childcare added and obligor-paid insurance subtracted → $2,014 transfer. | 3 |
| Alaska minimum wage as of 2026-07-05 | employment-law | Point-in-time statutory lookup: returns $14.00 effective 2026-07-01 (Ballot Measure 1 step) with the $15.00 change scheduled 2027-07-01 and the AS 23.10.065 citation. | 2 |
A flagship seed across the estate, premium, and retirement suites; it extends to all 32 APIs as scenarios are added.
Accuracy registry — all 32 APIs
Every API, its primary route, and its current verification tier. See how we verify for the beta→verified process — a number reaches verified only once a human confirms its defining scenarios against primary sources.
| API | Suite | Route | Tier |
| State Estate & Inheritance Tax | Estate Planning Suite | /estate/calculate-state-tax | beta |
| Federal Estate Tax & Portability | Estate Planning Suite | /estate/calculate-federal-tax | beta |
| Inherited IRA RMD Schedule | Estate Planning Suite | /inherited-ira/schedule | beta |
| Gift Tax Annual Exclusion Optimizer | Estate Planning Suite | /gift-tax/optimize-annual-gifts | beta |
| GRAT / CLAT / SLAT Trust Calculator | Estate Planning Suite | /estate/calculate-grat | beta |
| QTIP / QDOT Marital Trust Optimizer | Estate Planning Suite | /estate/optimize-qtip-election | beta |
| Social Security Filing Strategy | Premium Tax Suite | /ss/optimize-filing-strategy | beta |
| Social Security PIA Estimator | Premium Tax Suite | /social-security/estimate | beta |
| ISO Exercise AMT Calculator | Premium Tax Suite | /equity/calculate-tax | beta |
| QSBS §1202 Exclusion | Premium Tax Suite | /qsbs/check | beta |
| PTET / SALT Cap Optimizer | Premium Tax Suite | /ptet/calculate-benefit | beta |
| §199A QBI Deduction | Premium Tax Suite | /qbi/calculate-deduction | beta |
| Foreign Tax Credit Limitation | Premium Tax Suite | /international/calculate-ftc | beta |
| R&D Tax Credit (§41) | Premium Tax Suite | /tax-credit/calculate-rd-credit | beta |
| Form 8606 / Backdoor Roth | Retirement & International Suite | /ira/form-8606 | beta |
| Pension Lump Sum vs Annuity | Retirement & International Suite | /pension/lump-sum-comparison | beta |
| SSDI Benefit Estimator | Retirement & International Suite | /ssdi/estimate-benefit | beta |
| CPP + OAS (Canada) | Retirement & International Suite | /canada/retirement-benefit | beta |
| Australian Superannuation | Retirement & International Suite | /australia/superannuation/contributions | beta |
| ADA Compliance Requirements | Compliance & Data Suite | /ada/requirements | beta |
| Child Support Calculation | Compliance & Data Suite | /child-support/calculate | beta |
| Multi-State Employment Law | Compliance & Data Suite | /employment-law/lookup | beta |
| FAFSA Student Aid Index | Compliance & Data Suite | /fafsa/sai | beta |
| Immigration Points & Eligibility | Compliance & Data Suite | /immigration/calculate | beta |
| Professional Credential Verification | Compliance & Data Suite | /credentials/verify | beta |
| FDA Enforcement & Warning Letters | Compliance & Data Suite | /fda/enforcement/search | beta |
| SaaS Terms of Service Compliance | Compliance & Data Suite | /tos/check-compliance | beta |
| Supplier Diversity Certification | Compliance & Data Suite | /supplier-diversity/certifications | beta |
| Platform Content Policy Compliance | Compliance & Data Suite | /content-policy/check | beta |
| Flight Delay Compensation | Compliance & Data Suite | /flight/check-compensation | beta |
| Patent Expiration & Maintenance | Compliance & Data Suite | /patent/status | beta |
| Generic Drug Entry Date | Compliance & Data Suite | /pharma/generic-entry | beta |