AI PCs: This Backpack Just Ate A Four-Stroke V6!

There’s a limit to which a CPU-based PC can be stretched for AI workloads. But add some GPU/NPU pistons and you could be looking at something that’s still small but now ready with all AI-chops. Is the formula for PCs moving towards AI? Will it run?

What’s better than watching Verstappen in an F1 Grand Prix? Perhaps watching a First-Person-View (FPV) drone that’s right around him, cruising effortlessly at 220 mph! It is exciting. Because it is new. And hard.

Something that AI PCs are trying to do as they chase the new-world workloads that run as fast and as furiously as an F1 beast. But how do you turn a drone into a racing horse to get that kind of speed? How do you shrink AI-scale computing brutes to the size of a PC shelf?

It can be beneficial for both the enterprise segment as well as any individual professional working in gaming, simulation, coding or AI/ML related activities. - Prof. Nityesh Bhatt Information Management Area, Institute of Management, Nirma University

A lot is going on to crack that ‘how’. Maybe because now is the perfect lap to do that. Or at least, try to do that. The world is guzzling up advances in AI and ML with a never-before appetite. The contrails of excitement are all across the enterprise sky too.

In fact, it is being expected that by 2028, 50 percent of enterprise platforms will leverage specialized infrastructures to support AI infusion (way up from fewer than 10 percent in 2023) as surmised by Gartner. Canalys’ latest forecast augurs that about 48 million AI-capable PCs will ship worldwide in 2024 - making up around 18 percent of total PC shipments. They will keep making further inroads here. AI-capable PC shipments could surpass 100 million in 2025, carving out 40 percent of all PC shipments. In 2028, vendors could ship 205 million AI-capable PCs. From the lens of IDC too, the surge is inevitable. Shipments of AI PCs are growing from nearly 50 million units in 2024 to more than 167 million in 2027. By the end of the forecast, IDC pegs AI PCs to represent nearly 60 percent of all PC shipments worldwide.

We can see the signs already- especially with all the noise drummed up in this area by biggies like Intel and NVIDIA (rolling out dedicated offerings on specialized AI hardware).

But what are these creatures called AI PCs? Are they mutants or a new species altogether?

What’s an AI PC? What’s strapped inside?

Is AI PC just a new nomenclature or something new and different?

Some analysts define them as personal computers with specific system-on-a-chip (SoC) capabilities designed to run generative AI tasks locally. Some call it the same PC – but dressed up for the AI party.

AI PC is a new nomenclature for a nicely packaged, bundled PC with CPU, GPU and more specifically an NPU in a single computer (desktop or laptop) – is a picture that Prof. Nityesh Bhatt, Information Management Area, Institute of Management, Nirma University sketches. “In this way, it is an innovative concept that offers faster throughput for high-end graphical and specialized AI and ML specific applications without (or reduced) dependence on external cloud-based, edge-based or enterprise servers.”

The introduction of Ryzen AI early last year marked a significant milestone in PC technology. - Vinay Sinha, Corporate Vice President, India Sales at AMD

The biggest differentiation for this new set of wheels is the silicon cylinders inside – NPUs (Neural Processing Units), GPUs (Graphics Processing Units) instead of only/pure CPUs (Central Processing Units).

AI PCs represent a new category of computing devices and the defining difference between a traditional PC and an AI PC lies in the hardware architecture, explains Vinay Sinha, Corporate Vice President, India Sales at AMD. “While traditional PCs consist of CPUs and GPUs, AI PCs incorporate specialized AI processors in addition to a CPU and GPU. These processors, designed to handle AI tasks more efficiently, minimise power consumption, heat production and longer battery life as well as offering extra security for AI applications that run locally on the device. The added efficiency of dedicated hardware is crucial, especially in the enterprise segment where reliability and performance are paramount alongside battery life from mobile devices. The introduction of a NPU alongside a CPU and GPU is important in enabling AI PCs to handle diverse AI workloads seamlessly and more efficiently than ever before.”

Drilling deeper, Sinha underlines how NPU is a critical component for AI PCs, complementing both - the CPU and GPU by specifically accelerating certain AI tasks. “While both the CPU and GPU can accelerate some AI workloads, using the dedicated NPU can help to improve the overall system efficiency, enhances security and potentially allow the system to include new AI-driven capabilities in the future.”

Ranjit Atwal, Sr. Director Analyst at Gartner expands on that, underlining how the NPUs are key to optimizing and running the AI models on the PC. “AI PCs are equipped with dedicated AI accelerators or cores, NPUs, Accelerated Processing Units (APUs) or Tensor Processing Units (TPUs), designed to optimize and accelerate artificial intelligence tasks on the device. This provides improved performance and efficiency in handling AI and GenAI workloads without relying on external servers or cloud services. Gartner’s PC definition includes ARM-based products.”

At AMD’s ‘Advancing AI’ event in December 2023, AMD announced the 2nd generation of its AI processor, the Ryzen 8040, Sinha adds. “The new series builds on the success of its predecessors, offering even more powerful and efficient processing capabilities tailored for the most demanding AI applications.”

And is all this shake-up needed?

Ask Arup J Paul, Visual Artist, Multi-disciplinarian, and AI-technologist-cum-Creative professional and he paints all the user-side pixels for us. “As of now, almost all the PC vendors are thinking about a new era of AI-powered PC (an example, Samsung Galaxy Book 4 PC). These PCs are integrating AI technologies deeply into their architecture, enabling functionalities much like those borrowed from their cousin Smartphone. In essence, AI PCs are designed to leverage AI algorithms and neural networks to enhance various aspects of computing, from performance optimization to intelligent automation of tasks.”

From automatic framing, eye contact correction, blurring backgrounds during office calls, and enhancing image processing in real-time to enabling more sophisticated AI models for language translation and voice recognition, the NPU is at the heart of delivering these experiences smoothly and effectively. Sinha reasons, “As AI applications continue to evolve and demand more from our computing devices, the hardware that powers these experiences—NPUs, CPUs, GPUs, and beyond—becomes not just important but essential. These components are the foundation upon which the future of computing is being built, enabling more intelligent, responsive, and personalized technology that will shape our interactions with digital environments.”

What makes AI PCs race?

Will that help your line of work, we ask Paul and he leans closer to this PC with a creative person’s eye. “When it comes to NPU and GPU, things can get interesting. For creative professionals, especially those of us building Gen AI data models, we need a solution where we can have a PC that can be connected to an on-demand cloud GPU and NPU combo software-agnostic platform. What I mean by that, is when we are building a custom data model, we need to be able to train a lot of data and it means a lot of raw GPU power. Most consumer and professional PCs don’t have that kind of power but what if we can scale up and scale down GPU and NPU as we need?” This way the PC will stay portable and GPU and NPU acceleration will be a SaaS platform with a flat monthly fee, he points out. “I know there are a couple of services currently available like Google Colab and Runpod etc., but none of them plug and play consumer-ready add-on to your PC. So, we have a market gap and scope of innovation for the next-gen AI PC service.”

Businesses are buying AI PCs for future-proofing needs, as most AI/Gen AI needs are currently being met through the cloud. - Ranjit Atwal, Sr. Director Analyst at Gartner

In the reckoning of Atwal, the AI PC phenomena is supply-led with the CPU vendors providing the capabilities to run dedicated AI models. “The demand or need for AI will catch up over time, as software vendors develop workloads and use-cases that benefit from on-device AI processing.”

IDC notes the rationale of bringing AI workloads from the cloud to the client: to enhance performance by eliminating the round trip that current AI workloads must make to the cloud and back over the network; to enhance privacy and security by keeping data on the device versus in motion; and to lower the cost by limiting the need to access costly cloud resources.

Where would AI PCs stutter or shine?

The last-mile test of any hardware is always in the software. The same would apply to AI PCs too. Can they run with the software required for AI tasks?

Software is the crucial part, otherwise, it’s just a capability, avers Atwal. “We expect these areas to evolve. Media handling (photography, image quality, audio quality, upscaling); Translation; Searching your files and emails and Security enhancements and monitoring.”

However, issues like power consumption, e-waste and battery life have been pet peeves in the PC market for long. AI PCs could fix them. Specially if they can unlock longer refresh cycles.

Not for business, Atwal dismisses the idea adding that “Over the longer term, AI models will evolve and need to be updated on the device and hardware will incrementally get better as well.”

AI PCs are likely to have longer refresh cycles compared to traditional PCs due to their advanced capabilities and future-proofing potential, differs Paul. “For creative professionals, longer refresh cycles translate to reduced overhead costs and more time being creative. While this may impact PC market sales in the short term, as users upgrade less frequently, but as I said before it will also open up possibilities for innovative HW as a service solution which will, eventually, bring more revenue for the PC market, and it could lead to increased customer satisfaction and loyalty in the long run.”

As to being a greener option than erstwhile PCs, Atwal opines that may not be the case necessarily unless more significant cloud workloads move to the PC.

But Prof. Bhatt feels otherwise. “AI PC will be part of Green IT, due to lesser dependence on server/ cloud/ edge. Therefore, data retrieval will be faster. In addition, total cost of ownership (TCO) will be lower as there will be less need for recurring subscription fees associated with cloud access.”

AI PCs have the potential to be greener than current PCs, primarily due to their emphasis on efficiency and optimisation, seconds Paul. “AI technologies can optimise power usage by intelligently allocating resources based on workload demands, thereby reducing energy consumption. Additionally, advancements in hardware design, such as more energy-efficient processors and components, contribute to overall energy savings.”

The person in the driving seat picks the same lane. “By focusing on power efficiency, AMD’s latest processing solutions, including those equipped with AI capabilities, are poised to contribute to a greener computing environment- this comes from many factors including advanced manufacturing technology, efficient design and the inclusion of dedicated AI hardware which offers a more efficient way of completing AI tasks than a CPU or GPU. This not only applies to AI PCs but also to specialised industrial and telecommunications devices as well and AMD expects that AI will rapidly adopt in its full product lineup.” Cites Sinha.

What’s the last circuit then?

The big question is -Who would rule this market- traditional PC players or AI specialists like Intel, NVIDIA or smartphone/GPU experts like Apple, or High-end smartphone companies or Gaming big-shots?

Let’s look at the current device market trajectory first- to set the context.

Despite the modest 3.6 percent growth expected in the overall device market in 2024, the market will remain 18 percent smaller than its 2021 size - as seen in the total device market forecast by Gartner. The 3.6 percent rate, expected in 2024, is down from the 4.6 percent growth previously estimated. At the segment level for 2024, we can see that in PCs — Units reduced by 6.5 million to 250.4 million in 2024, and growth reduced further by 1.4 percentage points to 3.5 percent. As to Tablets — Units could decrease by 5.3 million to 133.3 million in 2024. After two years of decline, the PC market could grow from 2024 through 2027, with units reaching 281 million by 2027, but this is still some 50 million units lower than the peak in 2021, when Chromebooks drove growth.

In such a muted market, the AI PC opportunity can be a big game-changer. It does open a big door for non-Intel devices and non-PC players through ARM and Qualcomm, by creating a more dynamic ISV Eco-system that has been missing from PC industry for a long time, weighs in Atwal.

Traditional PCs often require frequent upgrades to keep up with evolving software demands and technological advancements. However, AI PCs, with their emphasis on AI-driven optimisation and hardware acceleration, are designed to remain relevant for longer periods - Arup J. Paul, Visual artist and creative technologist

Can such PCs give more power, more memory muscle, more speed, less latency etc. to specific user segments? Paul nods. “AI-powered PCs are anticipated to bring about a vast array of improvements in multiple performance metrics such as power, memory, speed, and latency. These advancements are crucial in meeting the demanding needs of creative professionals, who often work with resource-intensive applications and large datasets. I would add one more thing, which is portability, AI PCs should be highly portable if they empower a new generation of creative professionals regardless of their geo-location and studio structure or lack thereof.”

As Prof. Bhatt captures, “Their adoption is relatively limited at present; however, it will grow in the future as the use-cases grow and awareness increases. Traditional PC Players may find it difficult to compete with AI specialists like Intel, NVIDIA etc.”

As AI continues to become an integral part of our daily computing tasks, the adoption of AI PCs will become important for consumers and a strategic priority for enterprises focused on harnessing the full potential of AI for future growth and success, Sinha sums up.

If all that is happening, how far are we from a F1 that’s just about racing drones! If Hamilton and Verstappen have to wrestle their thumbs instead of steering wheels, who will win that race? Ralph Hogenbirk? Just a guess!

By Pratima H

pratimah@cybermedia.co.in