No, I don’t think AI is archaic. The way we are doing it of course isn’t archaic — AI currently represents the pinnacle of human engineering. However, I strongly feel that down the line, we will look back at how AI is being done right now and laugh.
Neural networks are remarkable—but they’re woefully inefficient. The sheer amount of processing power, water, and electricity to power a frontier model is truly mind-boggling. We have massive data centers to power frontier models. And while it is truly remarkable, while it is the current pinnacle of human engineering, “scaling laws” might later appear like a crutch. The way AI is being done right now, yeah, more is more—but I think the real path forward is how we can do more with less. A fundamental shift in how AI is done such that you can achieve the same (or better) intelligence on far, far less.
This idea seems laughable—but think back to supercomputers/mainframes in the 60s. The modern iPhone makes them seem like dumb behemoths. 1960s mainframes typically had around 1 megabyte (or less) of RAM. Modern iPhones have hundreds of thousands of times more memory (e.g., 6 to 8 gigabytes of RAM) and hundreds of gigabytes of flash storage. A single iPhone offers hundreds of thousands of times the processing speed and memory, consuming a tiny fraction of the power.
We are awe-struck by modern AI—but decades down the line, I think we might look at data centers the way we look at mainframes in the 60s, or even the way we look at 90s-00s PCs. The brain itself is 20-watt proof that the opportunities for efficiency may be enormous.
To me, frontier labs should start focusing a lot less on making models smarter, and a lot more on making them more efficient. This will likely require fundamental architectural leaps and theoretical shifts in how AI is done at all.