Target NVIDIA! These eight startups are challenging the "GPU dominator"

NVIDIA's Shocking Performance in the Second Quarter Shakes the Field, with data center revenue doubling to over $10 billion in three months, resulting in a four-fold increase in net profit to $6.74 billion YoY, firmly maintaining its dominant position in AI chips.

Behind NVIDIA, many young startups are striving hard, some continuously launching products claiming to surpass NVIDIA GPUs, while others are targeting NVIDIA's CUDA programming platform.

According to The Information, in 2017, at least a dozen startups attempted to challenge NVIDIA's position in the AI chip field. However, after a round of market reshuffling, those that should have closed down did, and those that needed a change in leadership did so.

At that time, two companies that received considerable attention, KnuEdge and Reduced Energy Microsystems, both closed down. Wave Computing filed for bankruptcy in 2020 and made a comeback the following year under a new name, MIPS. According to The Register, a former VP of Engineering at AI chip startup Mythic wrote on LinkedIn that the company ran out of cash last year. Earlier this year, Mythic announced the appointment of a new CEO and secured $13 million in funding.

Among the existing early-stage AI chip startups, Cerebras' large-scale AI chip integrates many processing capabilities of NVIDIA GPUs, while SambaNova Systems seems to be the most capable company to challenge NVIDIA in the early stages. In the previous round of financing frenzy, these two companies, along with Graphcore, raised several rounds of large-scale financing with valuations in the billions of dollars.

This market cleansing tells newcomers that startups are facing high costs and competitive risks.

More importantly, despite recent investor enthusiasm for AI, the funding environment for startups is more challenging than the previous generation. Data from PitchBook, a venture capital, private equity, and M&A database, shows that in the first half of this year, AI chip startups received just over $1 billion in venture capital, compared to a staggering $9.5 billion in funding in 2021.

In addition, these startups also face the challenges of technological complexity and high costs in the stagnation phase, and they need to complete the design and deliver it to manufacturers like TSMC for production as soon as possible. Nevertheless, founders and investors believe that generative AI will bring revolutionary changes to the computing field, which is enough to allow them to have a share of the pie from NVIDIA.

Here are the eight latest startups challenging NVIDIA:

D-Matrix and Rain Neuromorphics claim that their developed chips and software are more cost-effective for training and running machine learning models compared to NVIDIA products. Tiny Corp and Modular are developing an alternative to CUDA, a programming language developed by NVIDIA that speeds up application development but can only be used with NVIDIA GPUs.

Qyber, Modular, and MatX, three chip startups, were founded by former Google engineers and have gained the trust of investors.

Modular

Founded in 2022

Founders: Chris Lattner (CEO), Tim Davis (Chief Product Officer)

Investors: GV, Greylock Partners, Factory HQ, SV Angel

Equity Financing: $30 million

Modular is developing a development platform and programming language for training and running machine learning models. It allows users to choose from a range of AI tools, including the open-source software TensorFlow originally developed by Google and the open-source software PyTorch originally developed by Meta Platforms. Modular users can then run the models on server chips from NVIDIA, Intel, and others.

Building and running AI applications requires a significant amount of computing power, and developers say it's difficult to switch between different types of chips when trying to control costs. Modular launched a limited preview version of its product in May, aiming to help engineers run models on different types of hardware more easily to balance performance and cost.

Modular has not disclosed its pricing structure and generates limited revenue, but the founders' backgrounds have encouraged investors. CEO Lattner previously led the development of the Swift programming language at Apple and later worked at Google with Modular's co-founder and Chief Product Officer Tim Davis, overseeing the development of the company's AI products. According to The Information, Modular is currently in talks for a funding round that values the company at $600 million.

MatX

Founded in 2022

Founders: Reiner Pope (CEO), Mike Gunter (Chief Technology Officer)

Founder and CEO of MatX, Reiner Pope Investors: Outset Capital, SV Angel, Homebrew

Equity Financing: Unknown

MatX is developing a dedicated chip for large language models (LLM) used in text applications, with the aim of designing a chip that runs faster and more cost-effectively than hardware such as NVIDIA's GPU.

The startup was founded by former Google employees. MatX CEO Reiner Pope was involved in building Google's Pathways language model. MatX CTO Mike Gunter previously worked on Google's Tensor Processing Unit (TPU), which is Google's dedicated AI hardware division and competes with NVIDIA's GPU.

According to MatX's official website, the company has received support from several venture capital firms but has not announced its products or funding amount. According to the website, the California-based company, headquartered in Mountain View, has also received support from Amit Singh, Chief Business Officer of cybersecurity giant Palo Alto Networks, and researchers in large language models. It is currently unclear whether MatX has acquired customers, but the company states that it has received "strong support from a well-known large language model company."

Qyber

Founded in: 2022

Founders: Guillaume Verdon, Tom Hubregtsen

Investors: Hof Capital, Julian Capital

Equity Financing: Unknown

According to a well-informed source, Qyber is developing a chip specifically designed for running LLM.

The startup was founded by former researchers from Alphabet's secretive "Moonshot Factory" division, focusing on quantum computing. The company's official website remains mysterious about its plans, only stating, "The next computing era is not yet extinct, it is still alive." Stay tuned.

According to PitchBook, in November last year, Qyber raised funds from New York venture capital firm Hof Capital and San Francisco seed fund Julian Capital. Information cited an insider report that the startup has recently been discussing new funding with potential investors.

Rain Neuromorphics

Founded in: 2017

Founders: Gordon Wilson (CEO), Jack Kendall (CTO), Juan Nino (Chief Scientific Advisor)

Rain Neuromorphics Founder and CEO Gordon Wilson Investors: Airbus Ventures, Baidu Ventures, FoundersX Ventures, Sam Altman, Daniel Gross

Equity Financing: $33 million

Rain Neuromorphics has now been renamed Rain AI with the aim of solving the high cost problem of using traditional GPU for training and running machine learning models.

Generally, chips generate heat when transferring data between memory and processing components, which requires continuous cooling of the GPU, thereby increasing the power cost of data centers. Rain's chips and software combine memory and processing functions, allowing them to operate at lower temperatures and more energy-efficiently, said CEO Gordon Wilson.

Tiny Corp

Established in: 2022

Founder: George Hotz

Investors: undisclosed

Equity Financing: $5.1 million

It is said that Tiny Corp's products can help developers accelerate the process of training and running machine learning models.

George Hotz, the founder and former CEO of Comma AI, a magical kid and autonomous driving startup, announced the establishment of Tiny Corp in May this year, with the goal of completely changing the AI computing field.

Founder of Tiny Corp, George Hotz

Currently, Hotz is working on a set of open-source deep learning tools called tinygrad. In an article published on GitHub in May, Hotz stated that he believes tinygrad can be a "strong competitor" to Pytorch, a deep learning tool originated from Meta Platforms.

However, for now, Hotz seems to prefer to keep a low profile. He said, "We don't talk to two types of people, the police and journalists."

SiMa.ai

Established in: 2018

Founder: Krishna Rangasayee (CEO)

Investors: Fidelity, Amplify Partners, Dell Technologies Capital, VentureTech Alliance

Equity Financing: $200 million

Sima.ai is developing hardware and software that supports AI applications in the "edge" industries such as aviation, drones, automotive, and medical devices. Sima.ai was founded in 2018 by Krishna Rangasayee, who had worked at chip manufacturer Xilinx for nearly 20 years. In an interview with Information, he mentioned the growing demand in these industries for new hardware that supports AI, and he hopes to address this issue.

For example, autonomous vehicles require real-time decision-making, so they can benefit from running AI software locally. In the healthcare industry, companies may prefer to keep sensitive data on devices rather than uploading it to the cloud.

In June of this year, Sima.ai announced that its first-generation "edge" AI chips had entered mass production and that they were collaborating with over 50 clients in manufacturing, automotive, aerospace, and other industries.

Lightmatter

Founded in 2017

Founders: Nicholas Harris (CEO), Darius Bunandar (Chief Scientist), Thomas Graham (CEO)

Investors: Matrix Partners, Spark Capital, Viking Global, SIP Global Partners

Equity Financing: $266 million

Lightmatter uses lasers to transmit data between chips and server groups.

This startup emerged from technology that former MIT students patented. Co-founder and CEO Nicholas Harris stated that chips from traditional manufacturers like NVIDIA, AMD, and Intel transmit data through power-consuming wires, which is costly. In contrast, Lightmatter's products can reduce data center energy consumption costs by approximately 80% when training and running machine learning models.

Harris also expressed Lightmatter's desire to license its technology to companies like NVIDIA, AMD, and Intel for use in their own chips.

D-Matrix

Founded in 2019

Founders: Sid Sheth (CEO), Sudeep Bhoja

Investors: Microsoft, SK Hynix, Playground Global, Entrada Ventures

Equity Financing: $51 million

D-Matrix is developing a dedicated chip and software for running machine learning models that involve combined processing and memory, which are typically separate components on a chip. D-Matrix co-founder and CEO, Sid Sheth, stated that this approach allows the D-Matrix chip to generate less heat, requiring less cooling, making it a more cost-effective choice compared to mainstream GPU and CPU chips. Sheth emphasized the importance of this in an era where many companies are looking to build generative AI applications based on OpenAI's GPT-4 and other large language models.

Sheth mentioned that D-Matrix focuses on inference (running machine learning models) rather than training these models, as they believe that over time, these models will become larger and the operational costs will increase. D-Matrix has already had customers testing its chips and software and plans to deploy them for commercial use in the first half of next year.