
A GPT-4 Capability Forecasting Challenge
Date : 2023-07-15
Presentation
This is a game that tests your ability to predict ("forecast") how well GPT-4 will perform at various types of questions. (In case you've been living under a rock these last few months, GPT-4 is a state-of-the-art "AI" language model that can solve all kinds of tasks.)
Many people speak very confidently about what capabilities large language models do and do not have (and sometimes even could or could never have). Nicholas Carlini built this game as he got the impression that most people who make such claims don't even know what current models can do. So: put yourself to the test.
How likely do you think GPT-4 is to answer the question below correctly? Try it out here, and learn how models work
Recently on :
Artificial Intelligence
PITTI - 2026-07-14
What Happens When AI Agents Have to Find Their Own Market?
Implementation of the Scaling Trust project : early lessons from simulating discovery, negotiation, and trust between autonomou...
PITTI - 2026-03-05
Scaling Trust : a Missing Piece in Multi-Agent Worlds
Humanity’s ability to build complex civilizations relies on an "invisible infrastructure" - the shared culture, institutions, a...
PITTI - 2026-01-14
Cultural, Ideological and Political Bias in LLMs
Transcription of a talk given during the work sessions organized by Technoréalisme on December 9, 2025, in Paris. The talk pres...
WEB - 2025-11-13
Measuring political bias in Claude
Anthropic gives insights into their evaluation methods to measure political bias in models.
WEB - 2025-10-09
Defining and evaluating political bias in LLMs
OpenAI created a political bias evaluation that mirrors real-world usage to stress-test their models’ ability to remain objecti...