Pilot: Prompt caching › Ep 2
Ep 2 · What the model does with your text
Length 1:14Today's goal: What does the model actually do with your 21,000-token request?
What you'll learn
- Tokenisation: the text is cut into pieces, and each piece becomes a number.
- Embedding: each token becomes a vector, a point in a space with hundreds of dimensions.
- Attention: each vector looks back at the ones before it. That's the expensive step, and the work the cache saves.
- The cache keeps that maths, not your words.
Sources
- Burke Holland, “Your AI credits are vanishing - here's the fix” (YouTube): the chapter on how caching actually works