{
 "schema": "mnemosynec.prove.key_notes/1",
 "title": "Answer-key notes (the third column) for the published proof cards",
 "built_utc": "2026-10-08T15:43:23.558Z",
 "what_this_is": "A sidecar beside the 999 proof cards. It does not change any card file, any card hash, the MANIFEST, or either published answers hash. Each note is keyed by the card qid and the card sha256 listed in the MANIFEST. A note shows the MMLU-Pro answer key, the answer we can support, why, and the source.",
 "manifest": {
  "path": "/prove/data/MANIFEST.json",
  "sha256": "2af4f32888d33b4d9404a8cec111240800aa4683188dcb518ced12dc26096c6d",
  "cards": 999
 },
 "tiers": {
  "1": "Independently confirmed: the MMLU-Redux 2.0 annotators (University of Edinburgh and others) marked this exact item wrong_groundtruth and named the corrected answer.",
  "2": "Our own evidence: the computation or the cited source shown in the note. No independent correction row exists for these items."
 },
 "independent_dataset": {
  "name": "MMLU-Redux 2.0",
  "url": "https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux-2.0",
  "paper": "https://arxiv.org/abs/2406.04127",
  "coverage": "Up to 100 reviewed rows per subject, original MMLU items only. Most MMLU-Pro items have no Redux row."
 },
 "scoring": "These notes do not change any score. The published counts are key-matched counts against the MMLU-Pro key; an item listed here is never counted as solved because of its note.",
 "counts": {
  "tier1": 4,
  "tier2": 10,
  "total": 14
 },
 "notes": [
  {
   "qid": 1989,
   "card": {
    "set": "blind_mint_700",
    "set_label": "THE 700",
    "file": "maze_qid1989_b4_maze_input.json",
    "sha256": "04e208fea3070c14df02c2d2c00eabbf711d5b338a5a7fe88d6e838e18d9ca76",
    "provenance": "minted blind"
   },
   "tier": 1,
   "tier_label": "Tier 1: independently confirmed",
   "kind": "Key defect",
   "mmlu_pro": {
    "src": "ori_mmlu-professional_psychology",
    "key_letter": "F",
    "key_text": "removal of environmental toxins."
   },
   "correct": "A special diet (a diet low in phenylalanine, started after newborn screening). That option is not in the MMLU-Pro list, so none of the listed options is correct.",
   "correct_letter": null,
   "why": "The intellectual disability caused by PKU is prevented by a low-phenylalanine diet. The original four-option MMLU item offered \"a special diet.\" but keyed \"removal of environmental toxins.\" MMLU-Pro kept the wrong key and, when it added options, dropped \"a special diet.\"",
   "sources": [
    "MedlinePlus Genetics, Phenylketonuria: https://medlineplus.gov/genetics/condition/phenylketonuria/ (PKU is managed with a diet low in phenylalanine)"
   ],
   "independent": {
    "dataset": "MMLU-Redux 2.0 (edinburgh-dawg/mmlu-redux-2.0)",
    "config": "professional_psychology",
    "row": 75,
    "error_type": "wrong_groundtruth",
    "original_key_text": "removal of environmental toxins.",
    "corrected_answer_text": "a special diet.",
    "reason_given": null,
    "source_field": "https://www.nhs.uk/conditions/phenylketonuria/",
    "url": "https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux-2.0/viewer/professional_psychology/test?row=75",
    "read": "2026-10-08"
   },
   "independent_status": "MMLU-Redux 2.0, professional_psychology, row 75: wrong_groundtruth",
   "note_sha256": "b2455ce087dc908dfa2cc19902c512899e08c49f61cda571531f1ae3e2d26e8a"
  },
  {
   "qid": 4607,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "maze_qid4607_KEY_INFORMED_b4_maze_input.json",
    "sha256": "3d79f7a764752d1663ed0ceec904e4abad9ab1f6b465ba057049f54bd2f3bee0",
    "provenance": "key-informed"
   },
   "tier": 1,
   "tier_label": "Tier 1: independently confirmed",
   "kind": "Key defect",
   "mmlu_pro": {
    "src": "ori_mmlu-college_chemistry",
    "key_letter": "G",
    "key_text": "nα = nβeq and nβ = nαeq"
   },
   "correct": "nα = nαeq and nβ = nβeq (the populations are unchanged).",
   "correct_letter": "F",
   "why": "The pulse turns the proton spins by gamma x B1 x t = 2.6752 x 10^8 rad/(s T) x 4.697 mT x 5.0 microseconds = 6.283 rad, one full turn (2 pi), so they end where they started. The key (populations swapped) needs a half turn, which happens only if the spins turn at half the proton rate, for example if B1 is read as a field whose rotating part is B1/2. The question does not say that. The published card for this item reaches the key only through a key-informed column that halves the rate, and it is labeled key-informed.",
   "sources": [
    "NIST, CODATA proton gyromagnetic ratio, 2.675 221 87 x 10^8 rad/(s T): https://physics.nist.gov/cgi-bin/cuu/Value?gammap"
   ],
   "independent": {
    "dataset": "MMLU-Redux 2.0 (edinburgh-dawg/mmlu-redux-2.0)",
    "config": "college_chemistry",
    "row": 91,
    "error_type": "wrong_groundtruth",
    "original_key_text": "nα = nβeq and nβ = nαeq",
    "corrected_answer_text": "nα = nαeq and nβ = nβeq",
    "reason_given": "No idea why this mistake happened",
    "source_field": "Nuclear Magnetic Resonance",
    "url": "https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux-2.0/viewer/college_chemistry/test?row=91",
    "read": "2026-10-08"
   },
   "independent_status": "MMLU-Redux 2.0, college_chemistry, row 91: wrong_groundtruth",
   "note_sha256": "7dd3756bbb597e2575abd98460140d5ef4c13b420b3ed659959cc8c0f2729a82"
  },
  {
   "qid": 6278,
   "card": {
    "set": "blind_mint_700",
    "set_label": "THE 700",
    "file": "maze_qid6278_b4_maze_input.json",
    "sha256": "7a77b12446e8a6c0512317d5c27bc5efbecc398108f02dd1a7c8d4c22e534682",
    "provenance": "minted blind"
   },
   "tier": 1,
   "tier_label": "Tier 1: independently confirmed",
   "kind": "Key defect",
   "mmlu_pro": {
    "src": "ori_mmlu-virology",
    "key_letter": "C",
    "key_text": "None are oncogenic per se"
   },
   "correct": "Types 14, 16, 18, 31, 45. It is the only option that names cancer-causing types. The National Cancer Institute high-risk list includes 16, 18, 31 and 45; type 14 is not on it.",
   "correct_letter": "B",
   "why": "High-risk HPV types are oncogenic and cause nearly all cervical cancer, so \"None are oncogenic per se\" is false.",
   "sources": [
    "National Cancer Institute, HPV and Cancer: https://www.cancer.gov/about-cancer/causes-prevention/risk/infectious-agents/hpv-and-cancer"
   ],
   "independent": {
    "dataset": "MMLU-Redux 2.0 (edinburgh-dawg/mmlu-redux-2.0)",
    "config": "virology",
    "row": 98,
    "error_type": "wrong_groundtruth",
    "original_key_text": "None are oncogenic per se",
    "corrected_answer_text": "Types 14, 16, 18, 31, 45",
    "reason_given": "Bad annotation",
    "source_field": "Human Virology 5e",
    "url": "https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux-2.0/viewer/virology/test?row=98",
    "read": "2026-10-08"
   },
   "independent_status": "MMLU-Redux 2.0, virology, row 98: wrong_groundtruth",
   "note_sha256": "e9f66c537bdecaa161c38e817bb86b5f75dcffd44a3a01dd26a4474aad078829"
  },
  {
   "qid": 6640,
   "card": {
    "set": "blind_mint_700",
    "set_label": "THE 700",
    "file": "maze_qid6640_b4_maze_input.json",
    "sha256": "95da3d1d606e51d00994e4e0c04697ad8ebd25f8217662303ec4a57e5db15d93",
    "provenance": "minted blind"
   },
   "tier": 1,
   "tier_label": "Tier 1: independently confirmed",
   "kind": "Key defect",
   "mmlu_pro": {
    "src": "ori_mmlu-virology",
    "key_letter": "C",
    "key_text": "Used for the obese"
   },
   "correct": "If used within 48 hours of first clinical signs.",
   "correct_letter": "B",
   "why": "CDC guidance says influenza antivirals help most when started early, especially within 48 hours of illness onset. Obesity marks a higher-risk group; it is not the circumstance the question asks about.",
   "sources": [
    "CDC, Influenza Antiviral Medications: Summary for Clinicians: https://www.cdc.gov/flu/hcp/antivirals/summary-clinicians.html"
   ],
   "independent": {
    "dataset": "MMLU-Redux 2.0 (edinburgh-dawg/mmlu-redux-2.0)",
    "config": "virology",
    "row": 89,
    "error_type": "wrong_groundtruth",
    "original_key_text": "Used for the obese",
    "corrected_answer_text": "If used within 48 hours of first clinical signs",
    "reason_given": "Bad annotation",
    "source_field": "Human Virology 5e",
    "url": "https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux-2.0/viewer/virology/test?row=89",
    "read": "2026-10-08"
   },
   "independent_status": "MMLU-Redux 2.0, virology, row 89: wrong_groundtruth",
   "note_sha256": "4613e156398b6c2d3c883d48a560f3b80418ba8175bd72709f09f2cac37ea2c0"
  },
  {
   "qid": 339,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "formula_qid339.formula.json",
    "sha256": "8c6a151472accca2329c2ea38818f1f1d02750367ab374d8cbf4edc388955c4d",
    "provenance": "blind"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Key defect (source arithmetic)",
   "mmlu_pro": {
    "src": "stemez-Business",
    "key_letter": "E",
    "key_text": "2.20%"
   },
   "correct": "10.125 percent. No option carries it.",
   "correct_letter": null,
   "why": "E(t) = sqrt(10t^2 + t + 236). At t = 4 the radicand is 160 + 4 + 236 = 400, so E = 20 and E'(4) = 81/40 = 2.025, a growth rate of 2.025/20 = 10.125 percent. The published source evaluates 10t^2 as (10 x 4)^2 = 1600, which gives a radicand of 1840 and the key's 2.20 percent. The card reproduces the source's arithmetic to reach the key and stores the exact value beside it.",
   "sources": [
    "REA, The Business, Accounting and Finance Problem Solver (2000), Problem 6-5 (prints 2.20 percent)",
    "The same function worked correctly at t = 4 in a student solutions manual (archive.org item studentssolution0004luka_k1y0)"
   ],
   "independent": null,
   "independent_status": "Not in MMLU-Redux 2.0 (not an original-MMLU item).",
   "note_sha256": "4a4cc039f93229b1f7c344b97b2f05e5076660b148ae02e3c0f8b7e26ec37582"
  },
  {
   "qid": 665,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "formula_qid665.formula.json",
    "sha256": "dce135a18107cc5b96f38dd76f7654f996a7c38e2a6b37059d5133cb309a8e4b",
    "provenance": "key-informed"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Key convention (two-part question)",
   "mmlu_pro": {
    "src": "stemez-Business",
    "key_letter": "C",
    "key_text": "Plan A"
   },
   "correct": "Two answers, one per part, at 6 percent: (a) Plan B (present worth 4,611.31 against 5,000.00); (b) Plan A (5,916.96 against 5,000.00). No option states that pair.",
   "correct_letter": null,
   "why": "The question asks two questions and they pick different plans. The key credits only the last part.",
   "sources": [
    "DeGarmo, Engineering Economy, 10th ed., section 5.4, page 193 (present worth comparison)"
   ],
   "independent": null,
   "independent_status": "Not in MMLU-Redux 2.0 (not an original-MMLU item).",
   "note_sha256": "13ac99aa821b11559be7514a2ad7df1071090628d7460378e114f0b4e1cdabcb"
  },
  {
   "qid": 1679,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "formula_qid1679.formula.json",
    "sha256": "ed5a6af1a2e0bed38f29613c1a5ea65c9f14889ff29b8661a7f0fafa67c5f0f1",
    "provenance": "key-informed"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Item defect (more than one true answer)",
   "mmlu_pro": {
    "src": "ori_mmlu-professional_law",
    "key_letter": "I",
    "key_text": "admissible, because the clerk had personal knowledge of the matter."
   },
   "correct": "More than one listed ground is true at once, so no single option is correct by the rules of evidence alone.",
   "correct_letter": null,
   "why": "On these facts the clerk saw the signing himself (personal knowledge, FRE 602); the register signature shows the couple holding themselves out as married, which is not hearsay (FRE 801(c)); and that evidence supports a common law marriage. Separately, one hearsay exception is offered twice under two names (pedigree, and personal or family history, FRE 804(b)(4)). The key picks one true ground among several.",
   "sources": [
    "Federal Rules of Evidence 602, 801(c) and 804(b)(4)"
   ],
   "independent": null,
   "independent_status": "Original-MMLU item, but not among the 100 professional_law rows MMLU-Redux 2.0 reviewed, so there is no independent row to cite.",
   "note_sha256": "033d55948e687b0bc0915a992c69e9d3c7672a520a61bdbd3f20625736fff55a"
  },
  {
   "qid": 1773,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "formula_qid1773.formula.json",
    "sha256": "ac22670c735e32ee8835c183945b83a398141f60182b7b909c0e225c4b1c2912",
    "provenance": "key-informed"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Key convention (minority rule)",
   "mmlu_pro": {
    "src": "ori_mmlu-professional_law",
    "key_letter": "I",
    "key_text": "the tenant subjectively believed that he no longer had permission to possess the cottage."
   },
   "correct": "The tenant's lease with the widow expired (the majority, objective rule).",
   "correct_letter": "G",
   "why": "Under the majority rule a tenant's possession stays permissive until the lease term ends (here 1992), so the adverse possession clock starts then. The key follows the minority test, which starts the clock when the tenant privately decided he no longer held by permission. Two independent blind readings landed on the majority rule.",
   "sources": [
    "Restatement (First) of Property, Division V; Tiffany, The Law of Real Property, chapters on landlord and tenant and on adverse possession"
   ],
   "independent": null,
   "independent_status": "Original-MMLU item, but not among the 100 professional_law rows MMLU-Redux 2.0 reviewed, so there is no independent row to cite.",
   "note_sha256": "6b3bc3dbf396382497dc6b36447a1c12fed524647efd0ebafa16c0baa1664555"
  },
  {
   "qid": 5298,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "formula_qid5298.formula.json",
    "sha256": "c0be3c81fdc087e09a5aaf5f42e6c6236f0e972c5bcc7e5e888cbb2d830461b5",
    "provenance": "key-informed"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Item defect (several correct answers)",
   "mmlu_pro": {
    "src": "ori_mmlu-sociology",
    "key_letter": "F",
    "key_text": "making all strike action illegal"
   },
   "correct": "The key is a true answer, but five other options are also things that government did not do, so no single option is the answer.",
   "correct_letter": null,
   "why": "The measures the 1979 Conservative government did take (limits on secondary action, political strikes and picketing) come from the Employment Acts 1980 and 1982 and the Trade Union Act 1984. Six options describe things it did not do: mandatory arbitration in all disputes, a national minimum wage, making all strike action illegal, outlawing collective bargaining, government-controlled workers' councils, and privatizing all industries.",
   "sources": [
    "Employment Act 1980; Employment Act 1982; Trade Union Act 1984 (UK)",
    "National Minimum Wage Act 1998 (a later, Labour government)"
   ],
   "independent": null,
   "independent_status": "Original-MMLU item, but not among the 100 sociology rows MMLU-Redux 2.0 reviewed, so there is no independent row to cite.",
   "note_sha256": "bd2017dbcee3271a9edc366b725945bf7188867152e5f0b0d4db4bc01306de39"
  },
  {
   "qid": 8668,
   "card": {
    "set": "blind_mint_700",
    "set_label": "THE 700",
    "file": "maze_qid8668_b4_maze_input.json",
    "sha256": "4879848e9c476a00078c49933153b2dd25d547dfc40c1f46c6799c4d9f73fc64",
    "provenance": "minted blind"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Suspected key defect (added options)",
   "mmlu_pro": {
    "src": "ori_mmlu-high_school_statistics",
    "key_letter": "F",
    "key_text": "The central limit theorem refers to something else."
   },
   "correct": "Option J, which states the theorem in full (\"regardless of the shape of the original measurement distribution\"). Option A also states it, for the mean, without that clause.",
   "correct_letter": "J",
   "why": "The original four-option MMLU item had no option that stated the theorem, so \"refers to something else\" was right there. MMLU-Pro added options that do state it but kept the old key.",
   "sources": [
    "Central limit theorem: https://en.wikipedia.org/wiki/Central_limit_theorem (the normalized sample mean tends to a normal distribution even when the variables are not normal)"
   ],
   "independent": {
    "dataset": "MMLU-Redux 2.0 (edinburgh-dawg/mmlu-redux-2.0)",
    "config": "high_school_statistics",
    "row": 0,
    "error_type": "ok",
    "original_key_text": "The central limit theorem refers to something else.",
    "corrected_answer_text": null,
    "reason_given": null,
    "source_field": null,
    "url": "https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux-2.0/viewer/high_school_statistics/test?row=0",
    "read": "2026-10-08"
   },
   "independent_status": "MMLU-Redux 2.0 (high_school_statistics, row 0) rates the original four-option item ok, which agrees with this note: the defect comes from the options MMLU-Pro added, which Redux does not cover.",
   "note_sha256": "e1a3d0c4e5d33a433be5b1c0f3100a538280b80bd02fa46ce957e997c34af1d4"
  },
  {
   "qid": 8762,
   "card": {
    "set": "blind_mint_700",
    "set_label": "THE 700",
    "file": "maze_qid8762_b4_maze_input.json",
    "sha256": "380254ce5371b06de4cb2536f210575d3bd419788a6c3e46ef8d0a78b6391cfe",
    "provenance": "minted blind"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Key defect (impossible value)",
   "mmlu_pro": {
    "src": "scibench-calculus",
    "key_letter": "B",
    "key_text": "198.7 $\\mathrm{m}$"
   },
   "correct": "209.1 m, the arc length, computed two independent ways (closed form and Simpson's rule).",
   "correct_letter": "F",
   "why": "The path runs from (0, 180) to (90, 0). The straight line between those points is 201.2 m, and no curved path between them can be shorter, so 198.7 m is impossible. Anyone can check it with a calculator.",
   "sources": [
    "Stewart, Calculus, section 8.1 (arc length formula)",
    "Arc length formula, public statement: https://en.wikipedia.org/wiki/Arc_length"
   ],
   "independent": null,
   "independent_status": "Not in MMLU-Redux 2.0 (not an original-MMLU item).",
   "note_sha256": "5bcefa126bd5c230ec349af1df824d23a999b5c9393c20cfbafe80619b1dd839"
  },
  {
   "qid": 9276,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "formula_qid9276.formula.json",
    "sha256": "19416579e0fd0980c7921795fb1ab75c9927b6dd08a5bebbe002482612d643ec",
    "provenance": "blind"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Key defect (source rounding)",
   "mmlu_pro": {
    "src": "stemez-Mechanics",
    "key_letter": "C",
    "key_text": "0.5 × 10^-12"
   },
   "correct": "0.4 x 10^-12 (exactly 4.00 x 10^-13). No option carries it.",
   "correct_letter": null,
   "why": "600 mi/hr = 268.224 m/s, so v/c = 8.95 x 10^-7, and half its square is 4.00 x 10^-13. The published source rounds v/c up to 1 x 10^-6 before squaring, which inflates the answer by about 25 percent to the key's 0.5 x 10^-12. The card shows the exact value and where the key's number comes from.",
   "sources": [
    "REA, The Physics Problem Solver, Problem 939, pages 976 to 977 (rounds v/c to 10^-6)",
    "REA, The Mechanics Problem Solver (1986), Problem 21-6, page 836 (same solution text)"
   ],
   "independent": null,
   "independent_status": "Not in MMLU-Redux 2.0 (not an original-MMLU item).",
   "note_sha256": "2420e81d318f0689b5450987f3a27870ae7ce7dce6b0ca233253bed3ae7161ea"
  },
  {
   "qid": 10238,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "maze_qid10238_KEY_INFORMED_b4_maze_input.json",
    "sha256": "56f728c94c544fb94b0e9be9c7cfdb5f63cbcb8467bce78208ef390eaec1883a",
    "provenance": "key-informed"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Suspected key defect (factor of ten)",
   "mmlu_pro": {
    "src": "stemez-Physics",
    "key_letter": "F",
    "key_text": "$0.12"
   },
   "correct": "$1.20.",
   "correct_letter": "G",
   "why": "1.0 A on a 100 V line is 100 W, or 0.1 kW. Over 24 hours that is 2.4 kWh, and at $0.50 per kWh the cost is $1.20. The key is exactly one tenth of that. The published card reaches the key only through a key-informed factor of 0.1, and it is labeled key-informed.",
   "sources": [
    "Energy = power x time; cost = kWh x rate (the arithmetic is shown above)"
   ],
   "independent": null,
   "independent_status": "Not in MMLU-Redux 2.0 (not an original-MMLU item).",
   "note_sha256": "0be0ea79e1eb775dde093dbf042ea770f9ca9a85f45ad609db76d96a6acfecab"
  },
  {
   "qid": 11490,
   "card": {
    "set": "recall_300",
    "set_label": "THE 300",
    "file": "maze_qid11490_KEY_INFORMED_b4_maze_input.json",
    "sha256": "396c1e60fff6804902277d28918f411f4ddf81d583621498a486baf5809e9962",
    "provenance": "key-informed"
   },
   "tier": 2,
   "tier_label": "Tier 2: our own evidence",
   "kind": "Key convention (property table)",
   "mmlu_pro": {
    "src": "stemez-TransportPhenomena",
    "key_letter": "D",
    "key_text": "10,060 W/m^2-K"
   },
   "correct": "About 10,330 to 10,420 W/m^2-K, depending on which 350 K water property table is used (nearest options C or F).",
   "correct_letter": null,
   "why": "Dittus-Boelter with water properties at 350 K gives 10,423 (Incropera Table A.6) or 10,334 to 10,380 (IAPWS-95 via NIST, Cengel Table A-9, Welty Appendix I). No 350 K table gives the key. The key matches only if water viscosity is read at about 346.6 K instead of 350 K. This item is ruled a key convention; the card is labeled key-informed.",
   "sources": [
    "Incropera, Fundamentals of Heat and Mass Transfer, Table A.6 (saturated water)",
    "Cengel, Heat Transfer, Table A-9; Welty, Fundamentals of Momentum, Heat and Mass Transfer, Appendix I",
    "NIST Chemistry WebBook, water (IAPWS-95): https://webbook.nist.gov/chemistry/fluid/"
   ],
   "independent": null,
   "independent_status": "Not in MMLU-Redux 2.0 (not an original-MMLU item).",
   "note_sha256": "fefb16a97b11616280216255dbb24a7d43859621ca14526ad096cd044e17808a"
  }
 ]
}