CHECKPOINT 1  -  SET B  -  ANSWER KEY  -  OPEN ONLY AFTER YOU FINISH THE SET
Every output and traceback below was copied from a real run of the named file.
Captured with Python 3.13. On 3.11 and 3.12 the marker lines under the code in a traceback differ; line numbers and the last line do not.
Mark each part complete, partly there or missing against 'What a complete answer contains'.

QUESTION 1   [Mechanism]
  (a) The code names she typed were in the context. Each turn, the whole conversation so far is sent to the model again as input, so later answers could use them. The weights were set during training and did not change. Nothing was learned; the model only read text that was in front of it.
  (b) The new chat's context does not contain the code names, and the team's code names were not in the training data. The model predicts something plausible instead, which is why it gets them wrong, often confidently.
  What a complete answer contains
   - (a) The code names are in the context, and the conversation is re-sent each turn; the weights were fixed in training and did not change; all three terms are used correctly.
   - (b) The new chat has none of it in context; it was not in the training data; so the model produces a plausible guess.
        Standard | Example answer and why
        Meets | "Everything she typed stayed in the context, and the whole chat gets sent back to the model every turn, so it could use the code names. The weights came from training and never changed. The new chat had none of that in context and the code names were never in training data, so the model guessed something plausible and got it wrong."
        Partly | "It only remembers within a chat, so the next day it forgot." Correct outcome, but "remembers" and "forgot" describe a memory the model does not have. No mention of context being re-sent, weights, or training data.
        Misses | "It learned them, but it only saves what you teach it for one day." Accepts that the model learned from the chat, which is the misconception the item tests.

QUESTION 2   [Mechanism]
  (a) The prompt is turned into tokens. The model computes a probability distribution over every possible next token. One token is picked (sampled) from that distribution and added to the text. The model repeats this, one token at a time, until the answer ends.
  (b) Each token is chosen because it is a likely continuation, not because it was looked up. Citations in the training data share a pattern (title, clause number, quotation), so a citation-shaped continuation is highly probable even when no real document matches. The model has no step that checks the citation exists.
  (c) Nothing. Confidence is a property of the prose: confident wording is a likely continuation too. It is not evidence about whether the content is true.
  What a complete answer contains
   - (a) Tokens; a distribution over the next token; one token sampled and appended; repeat until done.
   - (b) Chosen as a likely continuation; the pattern of a citation is learned, not a lookup of a real one; no step checks that it exists.
   - (c) The tone tells you nothing about correctness, with a reason tied to the mechanism or to confidence being a property of the text.
        Standard | Example answer and why
        Meets | "It turns the prompt into tokens, computes how likely each next token is, samples one, adds it and repeats. Citations all look alike, so a citation-shaped string is a likely continuation even when no such document exists; nothing in that process checks. The confident tone is just more likely-sounding text. It says nothing about whether the clause is real."
        Partly | "It predicts words. It made up the citation because it didn't know the answer and wanted to help. Confidence doesn't mean it's right." Has prediction, but (b) gives the model intentions instead of a mechanism, and (c) gives no reason.
        Misses | "The model lied because the real standard is behind a paywall." Treats the output as a deliberate choice and invents a cause; no mechanism.

QUESTION 3   [Mechanism]
  (a) Training data. A widely published public fact; very likely in the training data.
  (b) Context. The times are in this chat, so they are sent with the question.
  (c) Inaccessible. A local file the model cannot reach and was never trained on.
  (d) Inaccessible. Spoken this morning: not in the training data and not in the chat. Any answer would be invented.
  What a complete answer contains
   - Each part: the correct circle, and a reason that says where the information is. A correct circle with no reason, or a wrong reason, is partly there.
   - "Context, if I paste it in" for an inaccessible item has the right reasoning, but the question says the information has not been supplied. Count it partly there.

QUESTION 4   [Code reading and tracing]
  Captured output:
    Captured run: L18_B_q4.py (Python 3.13):
        VAN-2
        5.0
        71 8
  What a complete answer contains
   - All three lines exact. Quotes around a string are not held against you.
   - 5 instead of 5.0 is the right value with the wrong type, so that line is partly there: an int times a float is a float.
   - The last line is wrong if its first value is written as 8: "7" + "1" joins two strings and gives 71.

QUESTION 5   [Code reading and tracing]
  (a) The elif branch. wind > 25 is 18 > 25, which is False, so the if branch is skipped. temp > 90 is 95 > 90, which is True, so the elif branch runs and decision becomes "MODIFY". The else never runs.
  (b) Captured output:
    Captured run: L18_B_q5.py (Python 3.13):
        MODIFY
        -7
  What a complete answer contains
   - (a) The correct branch; the first condition evaluated with the actual values and shown False; the second evaluated and shown True.
   - (b) Both lines exact. The second line is negative: -7. 7 misses the sign.

QUESTION 6   [Code reading and tracing]
  (a) Trace table, from the captured output:
        leg | miles after line 3
        2 | 6
        3 | 15
        4 | 27
        5 | 42
  (b) The last line printed is total 42.
  (c) 4 times. range(2, 6) stops before 6.
    Captured run: L18_B_q6.py (Python 3.13):
        2 6
        3 15
        4 27
        5 42
        total 42
  What a complete answer contains
   - (a) One row per pass with the right miles. If you made one arithmetic slip and then applied the rule correctly to your own wrong running value, count only the first wrong row.
   - A fifth row for leg = 6 is the off-by-one on range. Count that row as wrong, and count (c) as wrong only if the number you wrote disagrees with your own table. One off-by-one is one mistake, not two.
   - (b) Exact text. (c) 4.

QUESTION 7   [Code reading and tracing]
  (a) The loop keeps going while fuel > 30, so it stops once fuel is no longer above 30: the first time the test at the top finds fuel at 30 or less. Yes, fuel can end below 30. The test is only checked at the top of each pass, and the subtraction inside the loop can carry fuel past the limit before the test sees it.
  (b) fuel goes 100 -> 75 -> 50 -> 25. After the third pass, 25 > 30 is False, so the loop stops with legs = 3. Captured output:
    Captured run: L18_B_q7.py (Python 3.13):
        3 25
  What a complete answer contains
   - (a) A correct stopping condition is complete in itself: "it stops once fuel is no longer above 30" says everything the first half asks. The second half needs a yes, with the reason: the test is checked at the top, so the update inside can overshoot.
   - (b) Both values. 3 30 has the count right and the overshoot wrong, so it is partly there.

QUESTION 8   [Code reading and tracing]
  (a) and (b): count_at_least takes a collection of values and a minimum. It returns how many of the values are greater than or equal to the minimum.
  (c) 78, 90, 75 and 75 are at least 75; 62 is not. So the first line is 4. No score is 91 or more, so the second line is 0.
    Captured run: L18_B_q8.py (Python 3.13):
        4
        0
  What a complete answer contains
   - (a) Both inputs named.
   - (b) "How many", a count, with the comparison stated correctly: greater than or equal to.
   - (c) Both lines. 2 on the first line drops the two 75s and has misread >=.

QUESTION 9   [Fault diagnosis]
  (a) Line 4: total = total + int(line). It is the last frame in the traceback, inside total_sold.
  (b) The list log (line 8) contains an empty string "". int("") cannot turn an empty string into a number. The error line shows the value it choked on: ''.
  (c) total = 7 and line = ''. The first two lines were added (4 + 3 = 7) before the empty string arrived. Captured from the check script:
    Captured run: L18_B_q9_check.py (Python 3.13):
        stopped with total = 7 and line = ''
  What a complete answer contains
   - (a) Line 4. Line 9, the call, appears in the traceback, but the error was raised inside the function, so line 9 alone is partly there.
   - (b) Names the offending value, the empty string; says why int() fails on it; says where it came from, line 8. "A value error" with nothing else is partly there.
   - (c) total is 7 (4 + 3); 12 is wrong. line is the empty string: "nothing", "blank" or '' all meet the standard.

QUESTION 10   [Fault diagnosis]
  (a) Captured output:
    Captured run: L18_B_q10.py (Python 3.13):
        Batch 1 sold. Tickets left: 30
        Batch 2 sold. Tickets left: 20
        Batch 3 sold. Tickets left: 10
        Batch 4 sold. Tickets left: 0
        Batch 5 sold. Tickets left: -10
        Sold out after 5 batches
  (b) No. The comment says stop when none are left, but a fifth batch is "sold" after the tickets have already run out, tickets goes to -10, and the count says 5. Line 5 holds the fault: while tickets >= 0 lets the body run once more when tickets is already 0. It is off by one at the bottom end.
  (c) The corrected line is while tickets > 0: and the program then stops after four batches, so its last line is Sold out after 4 batches. Captured from the check script, repaired version:
    Captured run: L18_B_q10_check.py, repaired version, last two lines (Python 3.13):
        Batch 4 sold. Tickets left: 0
        Sold out after 4 batches
  (d) The colleague's version prints three batches and then Sold out after 3 batches, with 10 tickets still in the drawer. That is not a fix. It trades one off-by-one for another in the opposite direction, and again nothing crashes, so nobody is told. The desk would believe it had sold out with ten tickets unsold. A crash stops the program and names a line. A silent wrong answer looks like a result, gets trusted, and is acted on.
  What a complete answer contains
   - (a) The exact output, including the negative number and the last line.
   - (b) Names line 5, and says that >= lets the loop run once more when tickets is already 0. Naming line 6 or line 9 as the fault is wrong: they do what they should.
   - (c) while tickets > 0:, and the new last line Sold out after 4 batches.
   - (d) Says what prints, says it is not a fix because ten tickets go unsold while the program reports sold out, and gives a reason a silent wrong answer is worse than a crash: it looks like a result and gets trusted.
        Standard | Example answer and why
        Meets (d) | "Three batches, then 'Sold out after 3 batches'. Not a fix: it stops too early now instead of too late, and ten tickets never sell. Both versions run cleanly, so nothing tells you. A crash at least says 'look here'. This one says 'all done' and is wrong."
        Misses (d) | "It fixes the negative number, so it works." Checks only that the symptom in (a) went away. The requirement was to sell every ticket, and the output has to be checked against that, not against the previous output.

QUESTION 11   [Method]
  (a) One decomposition that meets the standard:
   - Read one hourly reading and split it into its three values.
   - Check wind against its limit.
   - Check rainfall against its limit.
   - Check temperature against its limits.
   - Combine the checks into one warning, saying which limit was broken.
  (b) Any two of these, or others that are genuinely open:
   - What the limits are. The request gives none; they must come from the events team, not be invented.
   - Is one bad value enough for a warning, or must several be bad together?
   - Is a value exactly at the limit acceptable or a warning?
   - What to do with a missing or garbled reading: warn, or ignore?
  (c) Example: wind_too_high(wind_mph, limit_mph) returns True or False.
   - wind_too_high(12, 25) returns False.
   - wind_too_high(25, 25) returns False. Boundary: exactly at the limit is acceptable (stated assumption; the events team must confirm).
   - wind_too_high(26, 25) returns True.
   - wind_too_high(0, 25) returns False.
  What a complete answer contains
   - (a) At least three sub-problems, each something one function could do and describable in one sentence, which together cover the request end to end. Splitting by step of a recipe ("the first part of the code") instead of by responsibility does not meet it.
   - (b) Two things the request genuinely leaves open. "Use Python" or "make it fast" are not open questions about the request.
   - (c) A signature with named inputs and a stated return; three criteria that each name a specific input and expected result; one of them sits exactly on a boundary and says which side it falls on. "Works for edge cases" is not a criterion.
        Standard | Example answer and why
        Meets (c) | "wind_too_high(wind_mph, limit_mph) -> True/False. AC1: (12, 25) -> False. AC2: (25, 25) -> False; at the limit is acceptable, to confirm with the events team. AC3: (26, 25) -> True."
        Partly (c) | A signature and three criteria such as "returns True when the wind is too high", "returns False when it is not", "handles edge cases". The signature is complete; two criteria state a testable direction but none names an input and expected result; there is no boundary.
        Misses (c) | "The function should work correctly and be efficient." No signature, nothing testable.

QUESTION 12   [Verification reasoning]
  (a) Three explicit claims: the function meets the specification; over_limit(31) returns True; over_limit(29) returns False. The reply also concludes that "the boundary is handled"; (b) shows why that does not follow.
  (b) The specification is the authority for the expected result; running the code is the authority for the actual result. Tests: 29 -> False, 30 -> True, 31 -> True. The test at exactly 30 is the one that settles it. The model's examples, 29 and 31, sit either side of the boundary and never touch it, so "the boundary is handled" was never tested.
  (c) over_limit(30) returns False, because 30 > 30 is False. The specification says 30 or more is over the limit, so it should be True. The function does not meet the specification; line 2 needs >=. Captured run of 29, 30 and 31:
    Captured run: L18_B_q12.py (Python 3.13):
        False
        False
        True
  What a complete answer contains
   - (a) The three explicit claims: meets the specification; 31 gives True; 29 gives False.
   - (b) A test at exactly 30 with expected True, and the observation that the model's examples are not the boundary; tests either side; the specification named as the authority for the expected result and running as the authority for the actual result.
   - (c) False; does not meet the specification; the reason: > excludes 30.
        Standard | Example answer and why
        Meets | "Claims: meets the spec; 31 gives True; 29 gives False. The spec decides what should happen, running it decides what does. Test 29 -> False, 30 -> True, 31 -> True. 30 is the boundary and the model never tried it. 30 > 30 is False, so over_limit(30) gives False and the spec says True. It fails."
        Partly | "Run it with 29 and 31 and see if they match. over_limit(30) returns True." Tests only the model's own examples, which cannot find the fault; the prediction in (c) is wrong.
        Misses | "The model gave two examples that work, so the boundary is handled." Accepts the model's reasoning as evidence.
