How OpenAI launched an API for ChatGPT and Dedicated Capacity

OpenAI launches an API for ChatGPT this year. For us to call ChatGPT a free text-generating AI developed by San Francisco-based startup OpenAI.com, a big win is a serious understatement.

Looking back to December last year, ChatGPT had an estimated more than 100 million monthly active users on different apps. ChatGPT attracted major media attention and brought in countless memes on social media platforms. The program is currently being a tool to write hundreds of e-books in Amazon’s Kindle store. Also, it connects with co-authoring at least one scientific paper online.

As we know that OpenAI as a business — albeit a capped-profit one — has a reason to monetize ChatGPT, so that investors will not become anxious. To achieve better options, they went on to launch a premium service, ChatGPT Plus, last February 2023. And in March 1 2023, OpenAI is introducing an API that’ll allow any business to build ChatGPT tech into their apps, websites, products and services.

ChatGPT API

Looking at the previous outcome of ChatGPT, an API was always the plan. This is an information from Greg Brockman, the co-founder, president and chairman of OpenAI. He said he held video call yesterday afternoon ahead of the launch of the ChatGPT API. The conversation is below;

“It takes us a while to get these APIs to a certain quality level,” Brockman said. “I think it’s kind of this, like, just being able to meet the demand and the scale.”

According to Brockman, the ChatGPT API is powered by the same AI model behind OpenAI’s wildly popular ChatGPT, dubbed “gpt-3.5-turbo.” For now, GPT-3.5 is the most powerful text-generating model OpenAI offers today. The achievement is through its API suite. So, the “turbo” moniker refers to an optimized, more responsive version of GPT-3.5 that OpenAI’s been quietly testing for ChatGPT.

ChatGPT Pricing

ChatGPT pricing starts at $0.002 per 1,000 tokens, or about 750 words. Greg claims that the ChatGPT API can drive a range of experiences. It also includes “non-chat” applications. Companies such as Snap, Quizlet, Instacart and Shopify are among the first startups that is implementing it.

Actually, the main motivation leading to developing gpt-3.5-turbo is to cut down on ChatGPT’s gargantuan compute costs. Sam Altman who’s the CEO of OpenAI once called ChatGPT’s expenses “eye-watering,”. He is also estimating them at a few cents as fees per chat in compute costs. The cost might look small but with over a million users, that presumably adds to up to billions of dollars.

But Greg B. says that gpt-3.5-turbo is improved in other ways;

“If you’re building an AI-powered tutor, you never want the tutor to just give an answer to the student. You want it to always explain it and help them learn. But that’s an example of the kind of system you should be able to develop [with the API],” says Greg. “We think this is going to be something that will just, like, make the API much more usable as well as accessible.”

ChatGPT in SnapChat

The ChatGPT API underpins My AI, as Snap’s recently announces chatbot in their app. It works for Snapchat+ subscribers, and Quizlet’s new Q-Chat virtual tutor feature. The popular Shopify is using the ChatGPT API to build a personalized assistant for shopping recommendations. On the other side, Instacart is leveraging it to create Ask Instacart toll. This upcoming toll will allow Instacart customers to ask about food and get “more accurate” answers from product data in the company’s retail partners.

“Grocery shopping can require a big mental load. It comes with a lot of factors at play, like budget, health and nutrition. It also brings in personal tastes, seasonality, culinary skills, prep time, and recipe inspiration,”. Greg says that Mr. JJ Zhuang (Instacart chief architect) told him via email.

“What if AI could take on that mental load and actually make grocery shopping truly fun? What if people could help the household leaders who are commonly responsible for grocery shopping, meal planning, and putting food on the table? Instacart’s AI system, when integrated with OpenAI’s ChatGPT, will enable us to do exactly that. Secondly, we’re thrilled to start experimenting with what’s possible in the Instacart app.”

Those who’ve been closely following the ChatGPT saga, though, might be wondering if it is ripe for release — rightly so.

ChatGPT to answer questions

Early on, users were able to prompt ChatGPT to answer questions in racist and sexist ways, a reflection of the biased data on which ChatGPT was initially trained. (ChatGPT’s training data includes a broad swath of internet content, namely e-books, Reddit posts and Wikipedia articles.) ChatGPT also invents facts without disclosing that it’s doing so, a phenomenon in AI known as hallucination.

ChatGPT — and systems like it — are susceptible to prompt-based attacks as well, or malicious adversarial prompts that get them to perform tasks that weren’t a part of their original objectives. Entire communities on Reddit have formed around finding ways to “jailbreak” ChatGPT and bypass any safeguards that OpenAI put in place. In one of the less offensive examples, a staffer at startup Scale AI was able to get ChatGPT to divulge information about its inner technical workings.

ChatGPT and Brands

Company Brands, no doubt, wouldn’t want to be caught in the crossropes. Greg is adamant they won’t be. Why so? One reason, he says, is continued improvements on the back end. However, it is in some cases at the expense of Kenyan contract workers. Furthermore, Greg is talking about a new (and decidedly less controversial) approach. He says OpenAI is calling Chat Markup Language, or ChatML. Presently, ChatML feeds text to the ChatGPT API as a sequence of messages together with metadata. This is as opposed to the standard ChatGPT. In addition, this will consumes] raw text represented as a series of tokens. (The word “presidential” would be split into the tokens “pre,” “si,” “den” and “tial,” for example.)

For example, given the prompt “What are some interesting party ideas for my 30th birthday?”. A developer can choose to input that prompt with an additional prompt like; “You are a fun conversational chatbot designed to help users with the questions they ask. See ChatGPT Artificial Intelligence Chatbot Developed by OpenAI, for more details.

You have to answer truthfully and in a fun way!” or “You are a bot” before having the ChatGPT API process it. These instructions help to better tailor — and filter — the ChatGPT model’s responses, according to Brockman.

ChatGPT API and Developer Growth

Greg is saying; “we’re moving to a higher-level API. If you have a more structured way of representing input to the system, where you say, ‘this is from the developer’ or ‘this is from the user’. I should expect that, as a developer, you actually can be more robust [using ChatML] against these kinds of prompt attacks.”

Another change that’ll (hopefully) prevent unintended ChatGPT behavior is more frequent model updates. With the release of gpt-3.5-turbo, developers will by default be automatically undergo upgrades to OpenAI’s latest stable model, Greg says. All this will begin with gpt-3.5-turbo-0301. Developers will have the option to remain with an older model if they so choose. Although, which might somewhat negate the benefit.

ChatGPT Updates

Whether they opt to update to the newest model or not, Brockman notes that some customers — mainly large enterprises with correspondingly large budgets — will have deeper control over system performance with the introduction of dedicated capacity plans. First detailed in documentation leaked earlier this month, OpenAI’s dedicated capacity plans, launched today, let customers pay for an allocation of compute infrastructure to run an OpenAI model — for example, gpt-3.5-turbo. (It’s Azure on the back end, by the way.)

In addition to “full control” over the instance’s load — normally, calls to the OpenAI API happen on shared compute resources — dedicated capacity gives customers the ability to enable features such as longer context limits. Context limits refer to the text that the model considers before generating additional text; longer context limits allow the model to “remember” more text essentially. While higher context limits might not solve all the bias and toxicity issues, they could lead models like gpt-3.5-turbo to hallucinate less.

Brockman says that dedicated capacity customers can expect gpt-3.5-turbo models with up to a 16k context window, meaning they can take in four times as many tokens as the standard ChatGPT model. That might let someone paste in pages and pages of tax code and get reasonable answers from the model, say — a feat that’s not possible today.

Brockman alluded to a general release in the future, but not anytime soon.

Greg has tells us; “The context windows are starting to creep up. Now, as part of the reason that we’re continually dedicating-capacity-customers-only right now is because there’s a lot of performance tradeoffs on our side”. “We might eventually be able to offer an on-demand version of the same thing.”

Microsoft and ChatGPT

Given OpenAI’s increasing pressure to turn a profit after a multibillion-dollar investment from Microsoft, that wouldn’t be terribly surprising.

Sometime ago, Microsoft made an announcement that it is bringing ChatGPT into Bing. In their statement, they will also implement it to their Edge browser. It’s even available to try out now. It generally uses an enhanced version of the AI model present in ChatGPT dubbed the Prometheus model. Unlike the ChatGPT app, this version upgrade can take into account recent information. While ChatGPT can only summarize information dated back a few years.

Although ChatGPT is coming to Bing and Edge, it isn’t here yet. Microsoft has a Bing ChatGPT waitlist available, which reportedly topped more than one million sign-ups in the first 48 hours. Reports say Microsoft is rolling out the feature to select users now, so we expect a full release in the coming months.

It’s pushing the limits of ChatGPT and its usefulness, though. As we uncovered, Bing Chat is capable of some unhinged responses, and Microsoft is quickly working to bring the AI into line.

What’s the Future of ChatGPT?

From what we see online, there’s no doubt that the tech world has already become obsessed with ChatGPT. And trust me, it’s not slowing down anytime soon. We know that ChatGPT-4 is the next iteration of the model. As well, it is supposedly going to significantly improve the accuracy and capability of ChatGPT. There’s no firm release date for it yet, but the New York Times reported that it would launch sometime in the first quarter of 2023.

Personally, I think the bigger development will be how ChatGPT is integrated into other applications. Microsoft reportedly made a multibillion-dollar investment in ChatGPT. This alone is already starting to pay off for them. The first integration was in Teams Premium, with some of OpenAI’s features showing up to automate tasks and provide transcripts. With ChatGPT now available in Bing, it’s only a matter of time before we see more options. We might even see ChatGPT & OpenAI’s other technologies wrapped into applications like PowerPoint and Word.

Similar Posts

• Artificial Intelligence AI – Meaning, Benefits, Types, Movie, Future
• Fraud Detection Algorithms Using Machine Learning and AI
• 7 Steps To Begin Machine Learning & Data Science
• How AI is Building the Robo-Advisor Landscape

All that to say, if you think AI is a big deal now, just wait until it’s built into the most common applications regularly for employee as well as student purposes. We don’t know how or when this will start to roll out, but it’s certainly an important part of the future of ChatGPT.