Before you start

This course follows How LLMs work. Here's the short version of what it uses from there, plus the bit of maths and Python behind the examples.

A large language model (LLM) takes text in and writes text back, one small piece at a time. The text you send is the prompt; it's measured in tokens, chunks of about four characters of English, and you pay per token. The context window is the most tokens one request can hold, so you can't paste in every document you have. The model remembers nothing between requests: anything it should use has to be in the prompt. Temperature is a setting for how varied its wording is; low means it sticks to its likeliest answer.

An eval (evaluation) is a set of test questions with known good answers, run after every change to see whether things got better or worse. When you're finding the right items in a big pile, recall is the share of the right ones you found, and precision is the share of what you found that was right.

A vector is just a list of numbers, like [0.9, 0.1, 0.3]. You can picture a list of two numbers as an arrow on a map: so far east, so far north. Lists of hundreds of numbers work the same way, in more directions than we can draw. The dot product of two vectors multiplies them item by item and adds the results: [1, 2] and [3, 4] give 1 × 3 + 2 × 4 = 11. It's big when the two point the same way.

The examples are short Python (Programming basics covers the language). A few shortcuts appear here: [x * 2 for x in xs if x > 2] builds a new list from an old one (a list comprehension); zip(a, b) pairs up two lists item by item; enumerate(xs, start=1) counts the items as it goes; and lambda c: c["score"] is a tiny unnamed function, used here to say what to compare when picking the biggest.

xs = [3, 8, 1]
print([x * 2 for x in xs if x > 2])
print(max(["ab", "c", "def"], key=lambda w: len(w)))
for i, w in enumerate(["a", "b"], start=1):
    print(i, w)
print(sum(a * b for a, b in zip([1, 2], [3, 4])))