13th March 2022
“Ultra-intelligence” computer planned for 2024
UK-based AI chipmaker Graphcore has announced a project called The Good Computer. This will be capable of handling neural network models with 500 trillion parameters – large enough to enable what the company calls “ultra-intelligence”.
In recent years, artificial neural networks (ANNs) have reached some impressive sizes. The emergence of deep learning in the early 2010s led to a dramatic acceleration in their progress, enabling models to grow at a faster rate than Moore’s Law. This reduced their doubling times from 20 to just six months.
GPT-3, launched in 2020, is perhaps the most well-known language model. It works with 175 billion parameters, which are the individual variables used to tweak and train the AI. Functioning like an enormously powerful and more advanced version of the auto-correct on your smartphone, it can generate portions of text that are remarkably similar to those written by a human. The newest and largest networks have since exceeded GPT-3 and tech companies are already planning for the arrival of models equivalent to the number of synapses in a human brain. Cerebras Systems in California, for example, is now making a system that could allow up to 120 trillion parameters in a single computer.
One company is developing a machine with even greater capabilities. Graphcore, founded in 2016 and based in Oxford, UK, has just announced a project that aims to support a massive 500 trillion parameters, nearly 3,000 times larger than GPT-3 and more than quadruple the synapses in a human brain.
Graphcore claims its machine will be an “ultra-intelligence AI computer” and is naming it The Good Computer, in honour of Irving John Good, who originated the concept now known as the “intelligence explosion” or technological singularity. In 1965, he wrote:
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind… Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control. It is curious that this point is made so seldom outside of science fiction. It is sometimes worthwhile to take science fiction seriously.
Graphcore may be overhyping its claims about intelligence – and its focus will be on hardware, rather than software – but the machine specifications are nevertheless impressive. If successfully built and fully operational by 2024, The Good Computer will feature 10 exaFLOPS of AI compute power (10 billion billion floating point operations per second), 4 petabytes (PB) of memory and 10 PB/s of bandwidth.
“We expect to be able to deliver this technology as a product for about $120 million,” said Simon Knowles, co-founder and chief technologist of Graphcore. “It’s a lot of money, but actually, quite a lot less than many of today’s supercomputers.”
With neural networks continuing to double in capability on a regular basis, alongside the hardware needed to support them, the first quadrillion parameter AI language model could emerge by the middle of this decade.
In addition to its Good Computer, the company revealed a new 3D chip, called “Bow”. This consists of two semiconductor dies stacked one on top of each other and will boost the speed of applications such as deep learning by 40% while cutting energy use. Bow is the first chip to use such “wafer-on-wafer” technology and its development occurred in close collaboration with Taiwan Semiconductor Manufacturing Company (TSMC).
One wafer is for AI processing, while the second wafer is a power delivery die. By adding deep trench capacitors in the power delivery die, right next to the processing cores and memory, the Bow chip delivers power far more efficiently, as illustrated in the diagram below. Previously, stacking silicon has caused problems with heat density, but Graphcore claims that Bow’s low power draw mitigates this. Each processor will provide 350 teraFLOPS of AI compute.
“We’re going to leverage this wafer-on-wafer technology to drive the next phase of AI computing,” said Nigel Toon, co-founder and CEO of Graphcore.
“TSMC has worked closely with Graphcore as a leading customer for our breakthrough SoIC-WoW (Wafer–on-Wafer) solution, as their pioneering designs in cutting-edge parallel processing architectures make them an ideal match for our technology,” said Paul de Bot, general manager of TSMC Europe. “Graphcore has fully exploited the ability to add power delivery directly connected via our WoW technology to achieve a major step up in performance, and we look forward to working with them on further evolutions of this technology.”
—
• Follow us on Facebook
• Follow us on Instagram
• Join us on Reddit
• Follow us on Twitter
• Subscribe to us on YouTube
Comments »