Discussion about this post

User's avatar
Ahnaf Ibn Qais's avatar

Talk of LLMs, AGI, & “recursive self-improvement” consistently ignores first principles that have constrained every complex system.

Everything (emphasis on “thing” 😉) reduces to four binding constraints… namely energy, materials, demography, and ecology… none of which can be hand-waved away by software progress or clever abstractions… for these things ultimately exist only via the interplay of the former four fundamentals… & so Unless one of those constraints is breached in a thermodynamically unprecedented way, their interaction inevitably generates negative feedback loops that dominate outcomes:

The intelligence-explosion narrative rests on a basic category error… whereby it treats recursive self-improvement as if it can operate independently of the physical world, which itself supervenes on said fundamentals aforesaid:

& this is why Sustained exponential growth doesn’t exist outside simplified models; in reality, exponentials are transient phases that resolve into logistic curves once constraints assert themselves & we see the negative hits crush all growth yielding hysteresis…

For this specific talk of “recursive self improvement” in particular… Compute requires energy, & said energy requires infrastructure, with said infrastructure then requiring materials & labor…

All of that then depends on demographic capacity & political stability. 😘 (Hint— These are already dead 🥰):

Materials are finite, demographics are aging or shrinking, ecological limits are non-negotiable, & each layer constrains the next through hard feedbacks….

& hence, every supposed “runaway” dynamic activates countervailing pressures that slow growth, raise costs, increase fragility, & impose ceilings in an inevitable, DOOM-ed & Destined manner 😍

The relevant question isn’t whether intelligence can improve itself in theory, but whether intelligence can outrun the systems that physically instantiate it to begin with… systems without which nothing else can even prevail anyhow:

Thermodynamics, History & Theology… answers that question clearly & repeatedly, time & time again:

All Growth eventually saturates, negative feedbacks dominate… & limits reassert themselves…. 😊

“Recursive self-improvement” is thus not a law of nature; constraints are… & they will simply make themselves felt this century as the 🌍 moves decisively into Negative Sum conditions & Perpetual Conflict, coupled with Low Energy Dominance.

Tl;dr— Pater OPTIMIST confirmed! 😘 🥰😍🤭

William M Briggs's avatar

One doubts. In fact, one disbelieves. I'll have to read the papers.

E.g., these reasons. https://wmbriggs.substack.com/p/the-limitations-of-ai-general-or

33 more comments...

No posts

Ready for more?