One-paragraph summary
Syntax builds small examples to explain language models, from rule-based chatbots and neural networks to tokenization, embeddings and transformers.
Syntax builds small examples to explain language models, from rule-based chatbots and neural networks to tokenization, embeddings and transformers.
Tokens are pieces of input, embeddings represent relationships, and attention helps the model weigh context when making predictions. The demonstrations make those distinctions concrete.
The presenter starts with an ELIZA-style chatbot and a small neural network. Later sections introduce a token vocabulary, vector representations and transformer attention, then connect prediction to sampling, fine-tuning and tool use.
Edited from the saved transcript summary.