How many lines of code would it take to code a strong AI/AGI?
How many lines of code would it take to code a strong AI/AGI?
It depends on how much computing power you have. In theory, given powerful enough supercomputer with shared memory to allow running the simplest neural network algoritms on a single computer with huge amount of weights (maybe 100 trillion weights to match synapse count in human brain) you could have complete system in maybe a 100–500 lines of code when you use already existing open source libraries.
You’d need something like 200 terabytes of VRAM for 100 trillion weights and you would preferably have interconnect technology to access that amount of RAM maybe 10 times a second, meaning your memory bandwidth should be about 2000 TB/s. If you take a single most powerful consumer GPU we have today, RTX 5090, it has 32 GB or 0.032 TB VRAM and it has about 1790 GB/s or 1.79 TB/s of memory bandwidth. And you cannot simply run 1000 cards in parallel because you would need the 2000 TB/s memory bandwidth over all the VRAM, not just within a single card and then whatever the PCIe bus can transmit (about 63 GB/s or 0.063 TB/s). As you can see, high end consumer GPU has about 0.1 % of the required performance today.
You would also need a lot of data for training. Like every page on the whole internet, all books ever published, maybe about 10% of YouTube videos and hopefully lots of bodycam and traffic camera footage.
However, the computational requirements for the state of art AI systems are so insanely high that right now it makes sense to use “only” a couple of billion dollars on the hardware and then a few hundred million dollars on software developers to try to extract the best possible performance from that hardware. Those developers will then write a lot of code to allow distributing all the necessary computing to various compute nodes with less than perfect interconnect technologies.
If you implemented full blown AGI using todays technology, the hardware and electricity cost would be so high that it would never pay itself back because the cost would be probably higher than $10K per every human on Earth. The hard part is to make the system economical enough to be worth doing in practice. This may require better algoritms than LLM with attention and mixture of experts architecture of the current state of art.
Good point and well explained. The limiting factor for AGI at present is neither software size nor complexity but rather compute and bandwidth, data availability, and cost. Even with relatively simple algorithms, the available hardware and connectivity do not favor large-scale systems and would be very costly.
Genuine progress will probably be made by more efficient algorithms rather than just larger models or more GPUs.
This is a great explanation thanks for sharing. Basically, building human level AGI with today’s tech isn’t realistic. Even the most powerful GPUs are way below what’s needed, and the memory, bandwidth, and data requirements are astronomical. Right now, it’s all about optimizing algorithms and spreading computation across many machines, because doing a full-blown AGI would cost more than $10K per person on Earth. It’s not just a tech problem it’s a money problem too.