Qualcomm: Running large models on mobile chips, this will be the "Holy Grail"

Artificial intelligence's impact on chips is gradually extending from the cloud to mobile devices. Once smartphones equipped with high-end Qualcomm chips are launched next year, ordinary users will be able to use generative artificial intelligence anytime, anywhere.

According to a CNET article on Friday, Ziad Asghar, Senior Vice President of Product Management at Qualcomm, said that the solution of embedding generative artificial intelligence (AIGC) in smartphones is more personalized and useful than cloud-based AIGC.

"I think this will be the holy grail," Asghar said. "This is a real promise that makes us very excited about the prospects of this technology."

In mid-July, Qualcomm announced its collaboration with Meta to launch the "Mobile Llama 2" in 2024.

Currently, Qualcomm is embedding AIGC into the next generation of high-end chips, which will make its debut at the Qualcomm Snapdragon Summit in Hawaii at the end of October this year.

In Asghar's lab, Qualcomm chips can handle AI models with 7 billion parameters, although this is far less than the 175 billion parameters of the OpenAI GPT-3 model, it should be suitable for mobile search.

"We will actually be able to do device demos at the (Hawaii) summit," Asghar said.

The demonstration device at the summit may be equipped with Qualcomm's next-generation high-end chip, Snapdragon 8 Gen3. As early as February this year, Qualcomm released the first terminal demonstration of Stable Diffusion, an image generation software running on an Android phone, which was equipped with Snapdragon 8 Gen2 chip.

There are many benefits to using generative AI on devices. For example, search records and personal data are kept confidential and do not pass through remote servers. Local AI is also faster than the cloud and can work in areas with no mobile service, such as on airplanes.

More importantly, device-based solutions also have commercial and efficiency implications. In a white paper released last month, Qualcomm stated that as machine learning models become more complex, the cost of running servers is increasing.

As early as April, OpenAI was spending about $700,000 a day to have ChatGPT respond to prompts, and the cost prediction was based on the GPT-3 model, not the more complex and potentially higher maintenance cost GPT-4 model.

It is reported that Qualcomm's solution does not require a complete server cluster, but instead allows the device's existing chip brain to do all the necessary thinking, thus avoiding additional costs.

Avi Greengart, an analyst at market research firm Techsponential, told CNET via email: "Running artificial intelligence on a smartphone is actually free - you pay in advance for the computing power." And outside of the mobile phone sector, Qualcomm also plans to announce AIGC solutions for PCs and automobiles at the upcoming summit.

Asghar said, "For those of us who grew up watching Knight Rider, KITT (the talking car in Knight Rider) will now become a reality."