OpenAI: Disrupting everything, also being disrupted by everything

What will the world look like in the next 50 years?

Kevin Kelly, the author of "Out of Control" and founding editor of "Wired" magazine, believes that from the next 5,000 days to the next 50 years, AI will become the most important keyword in the world, serving as an ubiquitous infrastructure just like the internet.

And OpenAI is, in his opinion, the most successful AI company, "an excellent case of disruption."

Since the hasty launch and unexpected popularity of ChatGPT at the end of last year, this company and its founding team have been thrust into the center stage of this global technological frenzy.

Where did OpenAI come from, and where is it heading? Recently, renowned tech journalist Steven Levy published a lengthy article in "Wired" magazine, delving into the growth history and vision of OpenAI.

Altman's Choice

Before leading OpenAI, Sam Altman had already become the CEO of Y Combinator, the world's most famous tech incubator. The significance of cashing out from all these unicorn companies was not about filling the partners' wallets, but about providing funding for species-level transformation.

He established a research department, hoping to fund ambitious projects that would solve the world's biggest problems. But in his view, AI is the disruptive innovation that changes everything: a superintelligence that can solve human problems better than humans themselves.

He even had thoughts of running for governor of California. But he realized that he had the ability to do something even greater - leading a company that would change humanity itself.

At a dinner party in California, he hit it off with Elon Musk.

At that time, Musk was in a heated argument with Google co-founder Larry Page. Because Musk believed that human consciousness was precious and unique, while Page believed that machines and humans were equal. If machines truly gained consciousness and eliminated humans, it would be the law of natural evolution. Page even accused Musk of being a "speciesist."

So, Musk decided to spend some money to make more efforts for the "human team."

Altman, who cared about both technological transformation and AI safety, naturally became his ideal partner.

What attracts top talent to a brand-new AI research institution? Altman's answer was the crazy vision of AGI.

AGI refers to the so-called general artificial intelligence, which can perform complex tasks like humans. During the days when Altman was still serving as YC CEO, computers had already achieved astonishing feats through deep learning and neural networks, such as labeling photos, translating text, and optimizing complex advertising networks.

These advancements made him believe that AGI was within reach for the first time. However, he felt concerned about handing it over to large corporations. He believed that these companies would be too focused on their own products and might miss the opportunity to develop AGI as quickly as possible. Moreover, if they did create AGI, they might release it to the public without taking necessary precautions, regardless of the consequences. So there needs to be someone else to check and balance them.

The most important principle for Altman in selecting candidates for recruitment is that they must be believers in AGI. With his own influence and Musk's persuasive language about exploring AGI, Altman recruited people like Stripe CTO Greg Brockman and Google Brain's core scientist Ilya Sutskever.

OpenAI was officially established in December 2015.

In 2021, he told reporters:

"AGI can only be built once. And there aren't many people who can operate OpenAI well. I'm lucky that a series of experiences in my life have truly prepared me for this."

Period of Confusion

Despite having a crazy and great vision, OpenAI had no clue about how to achieve its goals.

Altman recalled that when the initial small team had no office and gathered at Brockman's apartment, his mind was constantly thinking:

"What should we do?"

Even more than a year after the company was founded, there was little progress. At that time, the company did not have a clear direction, just randomly trying things out, investing a lot of effort in researching video game systems and publishing a few papers on robotics.

Altman remembered the state of the company at that time and said:

"We knew what we wanted to do. We knew why we wanted to do it. But we didn't know how to do it."

But they believed. The continuous improvement of artificial neural networks using deep learning techniques provided support for their optimism. Sutskever said that pursuing artificial intelligence "is not completely crazy. It's just moderately crazy."

It wasn't until 2016 that OpenAI welcomed legendary AI researcher Alec Radford. After accepting OpenAI's invitation, he told his high school magazine that taking on this new position was "a bit like joining a graduate program" - an open, low-pressure habitat for researching AI.

Radford is an introverted and low-key researcher who did not accept the author's invitation for a face-to-face interview but instead wrote a long email describing his work at OpenAI.

His biggest interest is to have neural networks engage in clear conversations with humans. This is different from the traditional scripted approach to creating chatbots, from the primitive ELIZA to the popular Siri and Alexa, all of which have been poor. He wrote, "Our goal is to see if there are any tasks, any environments, any domains, anything where language models can be useful." He explained that at the time, language models were seen as novel toys that could only occasionally generate a meaningful sentence, and only if you squinted your eyes. His first experiment was scanning 2 billion Reddit comments to train the language model.

Like many of OpenAI's early experiments, this one failed. But that was okay. This 23-year-old was given permission to keep going and fail again. Brockman said, "We just thought, Alec is great, let him do his thing."

Turning Point

In early 2017, a preprint of a research paper co-authored by eight Google researchers appeared, but it didn't attract much attention. The formal title of this paper was "Attention Is All You Need," but it later became known as the "Transformer paper," named both to reflect the game-changing nature of the idea and to pay homage to the toy that transformed from a truck into a giant robot.

The Transformer allowed neural networks to more efficiently understand and generate language. They analyzed the corpus in parallel to identify which elements were worth paying attention to. This greatly optimized the process of generating coherent text in response to prompts.

Eventually, people realized that the same technique could also generate images and even videos. Although the paper later became known as the catalyst for the current AI frenzy, at the time, Ilya Sutskever was one of the few who understood how powerful this breakthrough was.

Brockman recalled that when Ilya saw the emergence of the Transformer, he exclaimed, "This is what we've been waiting for."

This is OpenAI's strategy - to work hard to solve problems and then have faith that the missing pieces will be found by the team or someone in the field.

After that, Alec Radford began experimenting with the Transformer architecture. He said that the progress made in two weeks exceeded the progress of the past two years. He gradually realized that to maximize the potential of the new model, the key was to scale up - train on extremely large datasets. This idea was referred to as the "Big Transformer" by his colleague Rewon Child.

This approach required a change in OpenAI's previous scattered and individualistic corporate culture. It required pooling team resources and focusing on a breakthrough point. Adam D'Angelo, CEO of Quora and a member of OpenAI's board of directors, explained to the author:

"To leverage the advantages of the Transformer, you need to scale it up. You need to make it more like an engineering organization. You can't have every researcher doing their own thing, training their own models, and producing elegant things that can be published. You have to do the more mundane, less elegant work." Radford and his collaborators named the model they created "generatively pretrained transformer," abbreviated as GPT. Eventually, this model became known as "generative AI." To build this model, they collected 7,000 unpublished books, many of which were romance, fantasy, and adventure genres, and refined them using Quora answers and thousands of articles from middle and high school exams. The model consists of 117 million parameters or variables and outperforms all previous models in understanding language and generating answers.

But the most remarkable result is that, after processing such a large amount of data, the model is able to provide expertise in completely new domains beyond its training. These unplanned capabilities of the model are called "zero-shot." They still puzzle researchers, which is also why many people in the field are concerned about these so-called large language models.

Commercialization

Early funding for OpenAI mostly came from Musk. However, in 2018, Tesla began researching the use of AI technology for autonomous driving, and coincidentally, OpenAI had made significant technological breakthroughs.

Musk has always regarded OpenAI as his own property, so he proposed at the time to take over the entire company and merge it directly into Tesla. However, this proposal was firmly rejected by Altman and other executives, leading to a severed relationship. Musk withdrew all his investments and announced his departure at a company-wide meeting.

During the meeting, he predicted that OpenAI would end in failure and even called at least one researcher a "fool."

With no revenue, Musk's withdrawal undoubtedly posed a survival crisis for the company. Although OpenAI was doing cutting-edge AI research in Silicon Valley, it was a non-profit organization, which undoubtedly limited its attractiveness for fundraising.

In March 2019, OpenAI's executives came up with a peculiar solution. While remaining non-profit, they would create a for-profit entity. However, this for-profit division had a revenue cap, which was not publicly disclosed. Based on the company's bylaws, it could be as high as trillions of dollars (OpenAI also believes that if their revenue really reaches this number, they would have already created AGI that is practically usable). After reaching this cap, all profits earned by the for-profit entity would be returned to the non-profit lab.

With this new company structure, OpenAI successfully attracted investments from venture capital firms like Sequoia. But embarrassingly, **for OpenAI, the billions of dollars in venture capital are just a negligible amount. AI research is an exaggerated bottomless pit. The Big Transformer method for creating large language models requires large-scale hardware, and each iteration of the GPT series requires exponentially increasing computing power, which only a few companies can afford. **

So, OpenAI quickly locked in on Microsoft. Altman told reporters that this was because Microsoft CEO Satya Nadella and CTO Kevin Scott had enough courage: after spending over 20 years and billions of dollars building a so-called cutting-edge AI research department, they admitted that their work was a mess and then bet on a small company that had only been established for a few years.

Microsoft initially invested $1 billion and received computing time on its servers as a return. But as confidence grew on both sides, the scale of the deal expanded. Now, Microsoft has invested $13 billion in OpenAI.

Microsoft has also secured a significant benefit for itself. It not only owns a "non-controlling stake" in OpenAI's profitable division, reportedly 49%, but also obtained exclusive commercialization rights to OpenAI's technology. Moreover, it successfully made its cloud computing platform Azure the exclusive cloud service provider for OpenAI. In other words, Microsoft's massive investment not only secured a powerful partner but also locked in one of the world's most popular new customers for Azure cloud services.

Furthermore, according to the terms of the deal, some of OpenAI's initial ideals - providing equal access to everyone - seem to have been thrown in the trash.

During the course of the transaction, OpenAI gradually took on the characteristics of a for-profit organization, which made some employees feel uncomfortable. Several executives resigned, believing that OpenAI had become too commercialized and deviated from its original mission.

The Future of OpenAI

As the vision of a crazy AGI becomes increasingly real, Sam Altman and his team face growing pressure. They must achieve revolutions in each product cycle, meet the commercial demands of investors, and maintain a leading position in fierce competition. More importantly, they bear the mission of preventing the "almost savior" AI from eliminating humanity.

In the face of the changing times, OpenAI has undergone many changes, but the vision of building a safe AGI remains unchanged and continues to drive them forward. The leaders of OpenAI firmly believe that they will create AI systems that are intelligent enough and safe enough to lead humanity into an unimaginably prosperous era.