Increasingly powerful chips driving artificial intelligence

Display portlet menu

Increasingly powerful chips driving artificial intelligence

electronic circuit board

From the graphics processing unit through the neuromorphic chips to the quantum computer—the development of AI chips is supporting many new advances

Note: A version of this article originally appeared in The Quintessence.

AI-supported applications must keep pace with rapidly growing data volumes and often have to respond simultaneously in real time. The classic CPUs that you will find in every computer quickly reach their limits in this area because they process tasks sequentially. Significant improvements in performance, particularly in the context of deep learning, would be possible if the individual processes could be executed in parallel.

Hardware for parallel computing processes

A few years, ago, the AI sector focused its attention on the graphics processing unit (GPU), a chip that had actually been developed for an entirely different purpose. It offers a massive parallel architecture, which can perform computing tasks in parallel using many smaller yet still efficient computer units. This is exactly what is required for deep learning. Manufacturers of graphics processing units are now building GPUs specifically for AI applications. A server with just one of these high-performance GPUs has a throughput 40 times greater than that of a dedicated CPU server.

artificial intelligence chip graphicHowever, even GPUs are now proving too slow for some AI companies. This in turn is having a significant impact on the semiconductor market. Traditional semiconductor manufacturers are now being joined by buyers and users of semiconductors – such as Microsoft, Amazon and even Google – who are themselves becoming semiconductor manufacturers (along with companies who want to produce chips to their own specifications).

For example, Alphabet, the parent company behind Google, has developed its own Application-Specific Integrated Circuit (ASIC), which is specifically tailored to the requirements of machine learning. The second generation of this tensor processing unit (TPU) from Alphabet offers 180 teraflops of processing power, while the latest GPU from Nvidia offers 120 teraflops. Flops (Floating Point Operations Per Second) indicate how many simple mathematical calculations, such as addition or multiplication, a computer can perform per second.

Different performance requirements

Flops are not the only benchmark for the processing power of a chip. With AI processors, a distinction is made between performance in the training phase, which requires parallel computing processes, and performance in the application phase, which involves putting what has been learned into practice – known as inference. Here the focus is on deducing new knowledge from an existing database through inference.

“In contrast to the massively parallel training component of AI that occurs in the data center, inferencing is generally a sequential calculation that we believe will be mostly conducted on edge devices such as smartphones and Internet of Things, or IoT, products,” says Abhinav Davuluri, analyst at Morningstar, a leading provider of independent investment research.

Unlike cloud computing, edge computing involves decentralized data processing at the “edge” of the network. AI technologies are playing an increasingly important role here, as intelligent edge devices such as robots or autonomous vehicles do not have to transfer data to the cloud before analysis. Instead, they can acquire the data directly on site – saving the time and energy required for transferring data to the data center and back again.

Solutions for edge computing

For these edge computing applications, another new chip variant – Field-Programmable Gate Array (FPGA) – is currently establishing itself alongside CPUs, GPUs and ASICs. This is an integrated circuit, into which a logical circuit can be loaded after manufacturing. Unlike processors, FPGAs are truly parallel in nature thanks to their multiple programmable logic blocks, which mean that different processing operations are not assigned to the same resource. Each individual processing task is assigned to a dedicated area on a chip and can thus be performed autonomously. Although they do not quite match the processing power of a GPU in the training process, they rank higher than graphics processing units when it comes to inference. Above all, they consume less energy than GPUs, which is particularly important for applications on small, mobile devices. Tests have shown that FPGAs can detect more frames per second and watt than GPUs or CPUs, for example.

“We think FPGAs offer the most promise for inference, as they can be upgraded while in the field and could provide low latencies if located at the edge alongside a CPU,” says Morningstar analyst Davuluri.

More startups are developing AI chips

More and more company founders – and investors – are recognizing the opportunities offered by AI chips. At least 45 startups are currently working on corresponding semiconductor solutions, while at least five of these have received more than $100 million from investors. According to market researchers at CB Insights, venture capitalists invested more than $1.5 billion in chip start-ups in 2017 – double the amount that was invested just two years ago.

British firm Graphcore has developed the Intelligence Processing Unit (IPU), a new technology for accelerating machine learning and Artificial Intelligence (AI) applications. The AI platform of Mythic performs hybrid digital/analogue calculations in flash arrays. The inference phase can therefore take place directly within the memory, where the “knowledge” of the neural network is stored, offering benefits in terms of performance and accuracy. China is one of the most active countries when it comes to AI chip startups. The value of Cambricon Technologies alone is currently estimated at $1 billion. The startup has developed a neural network processor chip for smartphones, for instance.

New chip architectures for even better performance

Neuromorphic chips are emerging as the next phase in chip development. Their architecture mimics the way the human brain works in terms of learning and comprehension. A key feature of these chips is the removal of the separation between the processor unit and the data memory.

Launched in 2017, neuromorphic test chips with over 100,000 neurons and 100 million-plus synapses can unite training and inference on one chip. When in use, they should be able to learn autonomously at a rate that is a 1 million times better than the third generation of neural networks. At the same time, they are highly energy-efficient. Quantum computers represent a quantum leap for AI systems in the truest sense of the word. The big players in the IT sector, such as Google, IBM and Microsoft, as well as countries, intelligence services and even car manufacturers are investing in this technology. These computers are based on the principles of quantum mechanics. A quantum computer can perform each calculation step for all states at the same time. This means that it delivers exceptional processing power for the parallel processing of commands and has the potential to compute at a much higher speed than conventional computers. Although the technology may still be in its infancy, the race for faster and more reliable quantum processors is already well underway.

Take a deep dive into other components necessary for artificial intelligence like sensors in the latest edition of The Quintessence.


Related Articles
interior of autonomous car
The State of Automotive Only Starts with Autonomous Driving
March 6, 2020
Learn how the state of automotive spans a variety of applications, from electrication to in-cabin AI.
man wearing virtual reality headset with graphic overlays
How LoRaWAN and AI at the Edge Revolutionize the IoT
January 3, 2020
Learn how LoRaWAN and artificial intelligence (AI) at the edge can revolutionize the internet of things by reducing data transmission and improving latency.
person touching globe with finger
Today's case for AI at the edge for engineers
May 16, 2019
AI might feel far away, but engineers are innovating everyday. Should you consider edge AI?
A group of colleagues working in a computer security room having a discussion
Avnet and Microsoft Azure Sphere: IoT security in hardware, software and at the edge
March 5, 2019
IoT has transformed various industries by automating them at large, but the security concerns associated are just as large as the market opportunity this technology presents. That’s where Azure Sphere comes in.
A doctor writing on a clipboard
How AI-powered virtual assistants can transform healthcare
February 26, 2019
We have witnessed a paradigm shift in the way patients are being treated by the doctors with the help of AI and machine learning.
field of zeros and ones with shadow of letters AI in center
AGI, artificial general intelligence, the AI bubble and more: Q&A with Toby Walsh
February 25, 2019
We sat down with "rock start of AI" Toby Walsh to talk about the buzz around AI, how far true intelligence really is and how to operationalize AI today for benefits in your business tomorrow.
shopping cart in grocery store
4 ways AI connects retailers and customers
February 25, 2019
Technologies like machine learning and AI help retailers to process massive amounts of customer data. And with this data in hand, retailers can thoroughly understand their customers’ buying behavior.
computer monitor in factory
How leading manufacturing companies are reaping various benefits of AI
February 25, 2019
Manufacturing companies are applying AI-powered analytics for multiple use cases, such as, cutting unplanned downtime, better product designing, increasing product quality, ensuring employee safety, and many more.
depiction of brain surrounded by icons
The rise of machine intelligence in the AI era
February 24, 2019
Whether you know it or not, this technology has a lot of potential to deal with business-specific challenges and the benefits of deep learning outnumber its drawbacks.
futuristic city scene
The Car: A Rolling Smart Device
February 5, 2019
Be it for streaming your favorite music, sending emails or getting real-time information on traffic jams: cars have long since played host to mobile internet—and will continue to via diagnostics, hotspots and ADAS.
man using tablet computer in industrial setting
5 of the best artificial intelligence use cases
January 3, 2019
AI’s upgraded algorithms make predictive analytics, parse data, and help businesses make smarter decisions from the boardroom to the factory floor.
group of young people wearing virtual reality headsets
Top 6 Trends: Get ready to meet the demand for IoT solutions
December 11, 2018
Gartner predicts that in less than two years, more than 20 billion Internet of Things (IoT) devices will be deployed. Here are the top six technology trends to ensure your IoT strategies are successful.
Person going through security using IoT-enabled thumbprint scanning technology
2018 Was IoT’s Breakout Year
December 11, 2018
From connected devices at home to security cameras and sensors in factories, the acceleration of IoT in 2018 has ignited the imagination and pushed product development in new directions.
Doctor checking on a patient from a remote patient monitoring device -- artificial intelligence
AI From A-Z: The biggest buzzwords around artificial intelligence
November 28, 2018
Companies across markets from consumer electronics to factory floors are finding power in the introduction of two letters: AI (artificial intelligence).
Edge AI robotic hand using laptop
IoT at the Edge: How AI Will Transform IoT Architecture
By Lou Lutostanski   -   October 26, 2018
Futurists say artificial intelligence (AI) and the Internet of Things (IoT) will transform business and society more profoundly than the industrial and digital revolutions combined, and we’re now starting to see how that world might shape up.
Bill Amelio
Learning to utilize the benefits of Artificial Intelligence
May 30, 2018
Learn about the impact of Artificial Intelligence (AI) and the market opportunities it presents.
Bill Amelio
As technology reshapes our world, ecosystems reshape our business models
By Bill Amelio   -   April 25, 2018
See why businesses are now increasingly reliant on formal structures that tap into the power of partnership - “the company you keep.”
Related Events
Ultra96 Technical Training Courses
Date: January 14, 2020
Location: Multiple Locations