OpenAI's most powerful "Enterprise Edition" explodes onto the scene, will the B-side large model market be dominated by a single winner?

Speed doubled, secure encryption, unlimited access to GPT-4 - in the early hours of today, OpenAI announced the release of the enterprise version of ChatGPT, which can be considered the "most powerful version" of ChatGPT!

Compared to the standard version of ChatGPT, the enterprise version offers even more powerful performance, including: unlimited access to GPT-4, a 2x speed increase, support for unlimited advanced data analysis, a 32k Tokens context window that can be used for 4x input and files, shared chat templates, free API access, and guaranteed data privacy and security for enterprises.

In addition to the current version suitable for large enterprises, OpenAI will soon launch a self-service ChatGPT Business product for various small teams, serving organizations of all sizes and types. In other words, starting today, OpenAI has sounded the horn for a comprehensive attack on the B2B market.

Naturally, this raises a question: Will the B2B AI large model market be dominated by a single winner?

Recently, Guru Chahal, a partner at Lightspeed Venture Partners, conducted an in-depth analysis of this question.

The author believes that the most likely path for the B2B market is for enterprises to use large models in the exploration stage and gradually transition to smaller, specialized (adjusted + refined) models as their understanding of large models deepens through practical use.

Chahal also mentioned the factors that enterprises need to consider when choosing models, as well as the development opportunities for AI infrastructure, including evaluation frameworks, model operation and maintenance, reinforcement systems, operation and maintenance tools, and data utilization.

The article provides substantial insights and will undoubtedly be beneficial to those interested in the B2B AI market, AI infrastructure, and future opportunities.

The following is the full content of the article. Enjoy~✌️

Table of Contents:

● Classification of the Large Model Ecosystem

● Matching Use Cases with Models

● Where Are the Future Opportunities?

Over the past decade, as a member of the Lightspeed team, I have witnessed astonishing innovations in the field of artificial intelligence and machine learning, thanks to our deep collaboration with exceptional entrepreneurs.

Now, we are further collaborating and communicating with their companies, the platforms they have built, and the customers they serve in order to gain a more systematic understanding of how enterprises think about generative AI.

Specifically, we have delved into the large model ecosystem, attempting to explore questions such as "Will the most powerful large model dominate the market?" and "Will enterprises blindly rely on OpenAI's API or choose more diverse real-world use cases?"

The answers to these questions will determine the growth direction of the future large model ecosystem, as well as the flow of computing power, talent, and capital.

Classification of Large Model Ecosystem

According to our research, we believe that the field of artificial intelligence is undergoing a "Cambrian explosion" of models. In the future, developers and businesses will choose the most suitable models based on their actual needs, although the use of models in the exploration phase may be more concentrated.

The most likely path for the B-side is for businesses to use large models in the exploration phase, gradually transitioning to the use of smaller, specialized (adjusted + refined) models in the production phase as their understanding of large models deepens through practical use.

The following diagram illustrates our view of the evolution of the foundational model ecosystem.

We believe that the field of artificial intelligence models can be divided into three main categories, which are somewhat overlapping:

Category 1: Mega-brain Models

These are the most outstanding models and pioneers in the field of models. They have produced stunning demonstration effects that have deeply attracted our attention. When developers try to explore the potential limits of artificial intelligence applications, these models are often the default starting point.

These models have high training costs and complex maintenance and expansion. However, the same model can handle tasks such as law school entrance exams (LSAT), medical school entrance exams (MCAT), writing high school essays, and interacting with you like a chatbot friend. Currently, developers are experimenting with these models and evaluating their use in enterprise applications.

It should be noted that these models have high usage costs, significant inference latency, and may be overly complex in clearly defined restricted use cases.

At the same time, these models are general models and may not be accurate enough for specialized tasks (e.g., see comprehensive research at Cornell University).

Moreover, they are also black boxes that may pose privacy and security challenges to businesses, and businesses are exploring how to utilize these models without leaking data.

OpenAI, Anthropic, and Cohere all belong to this category.

Category 2: Challenger Models

These models also have high capabilities, second only to leading models. Llama 2 and Falcon are the top performers in this category. They are often as excellent as the "N-1" or "N-2" models in Category 1.

According to certain benchmarks, Llama 2 is even comparable to GPT-3.5-turbo. By fine-tuning on enterprise data, these models can perform as well as models in Category 1 for specific tasks.

Many of these models are open source (or very close to it). Once released, they are often quickly improved and optimized by the open-source community.

Category 3: Long Tail Models

These are "expert" models. They are specifically designed for specific purposes, such as document classification, identification of specific attributes in images or videos, pattern recognition in business data, etc. These models are flexible, have low training and usage costs, and can run in data centers or at the edge. Just by browsing Hugging Face, you can get a glimpse of the vastness of this ecosystem, which will continue to expand as it serves various use cases!

Matching Use Cases with Models

Although still in its early stages, we have already seen some leading development teams and companies start thinking about this ecosystem in a sophisticated way. They are eager to match use cases with the most suitable models, and may even use multiple models for more complex scenarios.

When deciding which model(s) to use, the following factors are typically considered:

a. Data privacy and compliance requirements, which can influence whether the model runs on the company's infrastructure or if the data can be sent to externally hosted inference endpoints.

b. Whether fine-tuning the model is crucial for this use case or if there is a strong desire to perform fine-tuning.

c. Expected "performance" levels of inference (latency, accuracy, cost, etc.).

The actual list is often longer than the above, reflecting the diverse range of use cases developers want to leverage AI to solve.

Where the Opportunities Lie

This emerging ecosystem has several important implications:

① Evaluation Framework: Companies will need tools and expertise to assess which model is suitable for which use case.

Developers need to determine how best to evaluate whether a specific model is fit for the "desired job." Evaluation needs to consider multiple factors, including not only model performance but also cost, levels of control that can be exercised, and more.

② Running and Maintaining Models: Platforms are expected to emerge to assist companies in training, fine-tuning, and running models, especially for the third tail of models.

These platforms, previously often referred to as ML Ops platforms, are expected to expand their definition to include generative AI. Platforms like Databricks, Weights and Biases, Tecton, and others are rapidly moving in this direction.

③ Augmenting Systems: Models, especially hosted LLMs (retrieval-augmented models), need to deliver exceptional results through generative augmentation.

This involves making sub-decisions, including:

o Data and Metadata Ingestion: How to connect structured and unstructured enterprise data sources and then ingest data along with metadata about access policies, etc.

o Generation and Embedding Storage: Choosing models to generate embeddings for data. Then, how to store these embeddings: which vector database to choose based on desired performance, scale, and functionality?

Here, there is an opportunity to build enterprise-oriented RAG (retrieval-augmented generation) platforms to simplify the complexity of choosing and combining these platforms:

① Operational Tools: Enterprise IT departments need to establish governance measures and manage costs for engineering teams.

Just like all the work done for software development today, they need to extend these tasks to include the use of AI. Areas of interest for IT include:

o Observability: How do models perform in production environments? Do their performance improve/deteriorate over time? Are there usage patterns that may impact model selection in future application versions?

o Security: How to ensure the security of AI local applications. Are these applications vulnerable to new attack methods and require new platforms?

o Compliance: We expect that the use of AI local applications and LLM will need to comply with the frameworks already being developed by relevant regulatory authorities. This is in addition to existing compliance systems such as privacy, security, consumer protection, and fairness. Companies will need platforms to help them maintain compliance, conduct audits, generate compliance certificates, and perform related tasks.

② Data: It is expected that platforms will be rapidly adopted to help businesses understand their data assets and extract maximum value from these assets using new AI models.

As one of the largest software companies on Earth once told us, "Our data is our moat, our core IP, our competitive advantage."

It will be crucial to monetize this data by leveraging artificial intelligence in a way that drives "differentiation" without compromising defensive capabilities. Platforms like Snorkel play a critical role in this.

We believe that now is the perfect time to build AI infrastructure platforms.

While the application of artificial intelligence will continue to transform the entire industry, supporting infrastructure, middleware, security, observability, and operational platforms are necessary to enable every enterprise to adopt this powerful technology.

Source: Hard AI