Posts Tagged ‘scaling’

Karl Mehta on X: “Ilya Sutskever says the AI scaling era is running out of road because pre-training is about to hit a finite-data wall. “At some point though, pre-training will run out of data. The data is very clearly finite.” “Then, okay, what do you do next? Either you do some kind of a https://t.co/nZ3uxadaVa” / X

June 23, 2026

https://x.com/karlmehta/status/2068997581959819473

When Your Digital Life Vanishes | The New Yorker

May 31, 2026

https://www.newyorker.com/magazine/2026/04/27/when-your-digital-life-vanishes QT:{{” The memorabilia dated back to the eighties. Back then, hard drives stored so little and cost so much that they were generally more valuable than the files they contained; one forty-megabyte drive on display in the lobby originally retailed for twenty thousand dollars. Advances in storage density, and the digitization of everything from filing taxes to laying out magazines, changed this calculus. “It was like two crossing lines,” Jay Hagan, who co-founded DriveSavers, later told me. “The cost of drives was going down, and the value of data was going up.””}}

Chris on X: “GPT-5.1 (Thinking High) is about 300 times cheaper per task than o3-preview (Low) while scorin g only a few points lower on ARC-AGI-1. 1 year later intelligence has gotten 300 times cheaper. This is why I can’t st and people who say “wahh the models too expensive” it will become https://t.co/VkfepKVTgV” / X

December 23, 2025

https://x.com/chatgpt21/status/1990516566073729362

Limits to economic growth | Nature Physics

December 24, 2024

https://www.nature.com/articles/s41567-022-01652-6

Murphy, T. W. (2022). Limits to economic growth. Nature Physics, 18(8), 844–847. https://doi.org/10.1038/s41567-022-01652-6

Scaling Laws for Neural Language Models

November 16, 2024

https://arxiv.org/pdf/2001.08361

Jared Kaplan ∗
Johns Hopkins University, OpenAI
jaredk@jhu.edu
Sam McCandlish∗
OpenAI
sam@openai.com
Tom Henighan
OpenAI
henighan@openai.com
Tom B. Brown
OpenAI
tom@openai.com
Benjamin Chess
OpenAI
bchess@openai.com
Rewon Child
OpenAI
rewon@openai.com
Scott Gray
OpenAI
scott@openai.com
Alec Radford
OpenAI
alec@openai.com
Jeffrey Wu
OpenAI
jeffwu@openai.com
Dario Amodei
OpenAI
damodei@openai.com

Supersized AI – ScienceDirect

November 2, 2024

https://www.sciencedirect.com/science/article/abs/pii/S0262407921018017

Rorvig, M. (2021). Supersized AI. The New Scientist, 251(3355), 36–40. https://doi.org/10.1016/s0262-4079(21)01801-7

BioNeMo | Generative AI Platform | NVIDIA

April 21, 2024

https://www.nvidia.com/en-us/clara/bionemo/

Where the internet lives | Feb 3rd 2024 | The Economist

March 10, 2024

https://www.economist.com/technology-quarterly/2024-02-03

nice discussion on the scaling economics of data centers

What will drive computer performance after Moore’s law?

November 11, 2020

https://science.sciencemag.org/content/368/6495/eaam9744

What will drive computer performance after Moore’s law?

June 6, 2020

https://science.sciencemag.org/content/368/6495/eaam9744