Description

While few people predicted the current revolution in AI, it is actually a 'revolution of predictability', with companies spending huge amounts of money on training runs and feeling secure due to the predictable input-output relations. The post also touches upon the role of architecture in AI development, suggesting that in the limit, everything converges to the same place, albeit at slightly different rates. Furthermore, the post highlights how scaling has been a key factor in recent advancements in image generation, with very little of it being architectural. Overall, the post provides a nuanced perspective on the current state and future direction of AI research.