The new champion of open-source large models: not Meta's Llama 2, but from this European company

Have you ever seen a language model that is 10 times smaller than Llama 2, performs better, and supports open source?

Mistral 7B has achieved that.

According to media reports, the "preview model" released by Mistral shows that Mistral 7B has successfully outperformed Llama 2 on MT Bench, with only one-tenth of the parameters of Llama 2, which is 70B.

MT Bench is a test set consisting of 80 high-quality multi-turn dialogue questions, designed to test the ability of multi-turn dialogue and instruction following.

This means that large language models (LLMs) have finally found a solution to the problem of parameter size and performance balance. According to official information, Mistral 7B outperforms LLMs with parameters as high as 13B in all standard English and code benchmark tests.

In September of this year, Mistral AI, a French AI company that has only been established for 6 months, officially released Mistral 7B. Last week, as the only European company, Mistral AI participated in the AI Engineer Summit held in the UK in October, sharing the stage with tech giants such as OpenAI, Google, and Meta.

"Technological Pioneer"

Mistral's professionalism has made it a new favorite in the AI investment community.

Arthur Mensch, the founder of Mistral, said in an interview with the media that although Mistral is a young startup, it competes with the entire AI industry, including Google and OpenAI:

"We have always been pioneers in this technology."

"We compete with everyone."

Mistral describes the growth rate of Mistral 7B as follows:

"In two years, it has gone from Gopher to Chinchilla, then to Llama 2, and now Mistral 7B."

Among them, Gopher was launched by DeepMind in 2021 with 280B parameters; Chinchilla was launched by DeepMind in 2022 with 70B parameters; Llama 2 was launched by Meta in July 2023 with 34B parameters.

Currently, there are reports that a16z is considering investing $250 million in Mistral. According to insiders familiar with the negotiations, heavyweight Silicon Valley companies including General Catalyst and Andreessen Horowitz are considering investing 400 million euros, which could push Mistral's valuation to 1.5-2 billion euros.Partner Antoine Moyroud, from Lightspeed Venture Partners, led the first round of financing for Mistral and stated, "They have exceeded our internal expectations," adding, "We are increasingly excited about this business."

Competing with Silicon Valley?

Currently, Silicon Valley AI companies, led by Google and OpenAI, are at the top of the pyramid and constantly seeking further development. These companies are also the focus of most investors.

It is reported that OpenAI is attempting to sell employee stock at a valuation of $86 billion. Anthropic recently received investment commitments from Google and Amazon, with a total investment amount potentially reaching $6 billion.

Mistral's rise has illuminated Europe's presence in the AI field.

Because companies with a market value exceeding 1 billion euros are rare in Europe, and French President Macron has repeatedly hinted at ambitious plans for the AI field, hoping to cultivate European AI companies.

Mistral's advantage is not only in technology. Mensch has stated that compared to larger and better-funded competitors, Mistral has an advantage in efficiency.

He mentioned that the company launched its first LLM model with only a 10-person team, with training costs of less than $500,000, while competitors spent tens of millions of dollars. He added, "We are pleased to be the most capital-efficient LLM company."

Another advantage is its open-source approach. Mistral publicly releases its AI models, supporting the Apache 2.0 open-source license. This allows enterprise customers to have better control over their data, increases visibility into the usage process, and attracts professionals in the development field.

However, despite raising a record-breaking 105 million euros in seed funding in June, Mistral is not yet profitable. Mensch stated that this situation will change "by the end of the year," and he expects to release a new platform for customers to access their LLM models.

Pia d'Iribarne, a partner at New Wave and one of Mistral's investors, stated that the "fundamentals for building a large-scale AI company are already in place."