Let's build the GPT Tokenizer
Date : 2024-02-20
Description
In this lecture, Andrej Karpathy builds from scratch the Tokenizer used in the GPT series from OpenAI. In the process, he shows that a lot of weird behaviors and problems of LLMs actually trace back to tokenization and discusses why tokenization is at fault, and why someone out there ideally finds a way to delete this stage entirely.
Watch and like on Youtube
Recently on :
Artificial Intelligence
Information Processing | Computing
PITTI - 2026-10-10
Who Answers for AI’s Actions?
Just months after the Institut Présaje’s symposium on AI liability, the rise of highly persistent agents coordinating together ...
PITTI - 2026-07-14
What Happens When AI Agents Have to Find Their Own Market?
Implementation of the Scaling Trust project : early lessons from simulating discovery, negotiation, and trust between autonomou...
PITTI - 2026-03-05
Scaling Trust : a Missing Piece in Multi-Agent Worlds
Humanity’s ability to build complex civilizations relies on an "invisible infrastructure" - the shared culture, institutions, a...
PITTI - 2026-01-14
Cultural, Ideological and Political Bias in LLMs
Transcription of a talk given during the work sessions organized by Technoréalisme on December 9, 2025, in Paris. The talk pres...
WEB - 2025-11-13
Measuring political bias in Claude
Anthropic gives insights into their evaluation methods to measure political bias in models.