Developing an AI-based product using GPT-3

Warning: Some parts of this article were written with the assistance of AI.

What would you do if you could go back in time and be one of the first people to use the Internet?

As for me, I would start a tech company. However that isn't what I did and the reason why is because I did not have absolute knowledge at the time that the Internet was going to be as important as it is now.

Much like the advent of the Internet, deep learning and AI are going to be one of the big technology revolutions of the 2020's and we're only just starting to see this technology become available to the public as a useful tool. Siri, Alexa and other AI's have already become commonplace, but they are still quite weak in their abilities. For example. if you ask a hard question of Siri, you will most likely receive a list of hyperlinks to review instead of a useful answer.

Enter GPT-3

This is where GPT-3 comes in. I was having a hard time making it easy for the layman to understand so I went ahead and asked GPT-3 to explain it for us.

"GPT-3 is a computer program that can “read” text and transform it into new text. The GPT-3 is trained to write sentences that are similar to other sentences, in an attempt to make the text sound more human."

That's not a bad explanation, and it gives just a hint as to the power and capability of GPT-3. Put simply, GPT-3 can take an input prompt and generate text that sounds human. People have so far written books, blogs, articles, advertisements, jokes, stories, and much more using the power of GPT-3.

GPT-3 was created by OpenAI, a tech company in California and is currently only available to beta testers. I was one of the lucky people who was granted access to GPT-3 in beta testing and this is the story of my process of creating a product that uses GPT-3.

This article is going to be about making products using GPT-3. For more information about the technical aspects of GPT-3 I encourage you to check out the OpenAI website.

Using the GPT-3 API and First Use Cases

OpenAI provides developers with a playground to explore and test their prompt ideas, as well as a RESTful API so that developers can create applications that use GPT-3.

The interface to GPT-3 is an API that uses a request/response pattern in which users pass a "prompt" along with some configuration information, and they receive a "completion" from GPT-3, which is just text. Much like communicating with people, the quality of the message that you send to GPT-3 will dictate to some degree what kind of response you will get.

I work in the finance industry, and so logically the first thing that I tried to do was to solve an age-old technical problem in finance. Create human-readable paragraphs of info about a stock, bond, currency, commodity or derivative using just data. This has been solved in the industry, but requires the development of complex systems to analyze the data and create the text. So my idea was the create relatively accurate financial "smart text" about a currency.

It worked! Using GPT-3's "instruct" models I was able to quickly create a prompt that will take in some data about bitcoin and return a human-readable paragraph using the context of the data provided.

Example Prompt:

Here is some data about the current bitcoin price:

{"time":{"updated":"Dec 30, 2020 20:17:00 UTC","updatedISO":"2020-12-30T20:17:00+00:00","updateduk":"Dec 30, 2020 at 20:17 GMT"},"bpi":{"USD":{"code":"USD","symbol":"$","rate":"28,787.2383","description":"United States Dollar","rate_float":28787.2383},"GBP":{"code":"GBP","symbol":"£","rate":"21,148.0553","description":"British Pound Sterling","rate_float":21148.0553},"EUR":{"code":"EUR","symbol":"€","rate":"23,426.3061","description":"Euro","rate_float":23426.3061}}} 

Write a paragraph about the current bitcoin price:

Example Completion

Here is what GPT-3 returned.

The current bitcoin price is approximately $28,787.23 per US dollar and $21148.05 per British pound sterling.

GPT-3 was able to take a blob of data and transform the information into new content. Not only was it able to figure out what to write, but it also detected the presence of currencies and even converted the currency amounts to the appropriate decimal place. It made a small mistake by dding a dollar sign forthe British Pound, but overall the output was really good!

I rapidly built a prototype for my financial app just to test out the capability. In the end however I decided that the financial product was a little too risky for me to take on at this time. There are legal ramifications to presenting financial information incorrectly to consumers. So I felt like I needed to look for a better option.

This is what led me to look for a new idea.

The Cure for Writer's Block

Writing content can be really hard, especially when you know what you want to write about, but just can't seem to get started with an idea.

One of the things that I noticed was most useful when I was using the GPT-3 playground was that it was really useful in helping me break through writer's block. I found that I could enter in a little background info to the prompt and ask it to rephrase the content or create ideas. While not all of the ideas and content were used verbatim. it did give me a lot of ideas and helped me to break free of writer's block.

So I decided to test out a use case for writers. I created a sample prompt that was made up of four data elements. Topic - the general topic of the story or content.
Background - some background information about the story or content.
Title - the title of the story Story - the story or content idea. I found that by just providing a single example of the above prompt to GPT-3, I could create a vast array of new ideas for stories and content by just passing "topic" and "background" to the API. GPT-3 would return a title and the story in return. This where I got the idea for my product, StormyNight.ai.

It was a dark and StormyNight.ai

There is a classic Peanuts (tm) cartoon in which Snoopy is trying to write a novel and he cannot seem to get past the first sentence, "It was a dark and stormy night." So I decided to help Snoopy out and created my first AI product for the world. StormyNight AI would start out as a way to generate and share story ideas and content that are created by GPT-3.

While this first use case is not an absolute cure for the writer's block problem, I hope that the stories generate really do help authors get past that first sentence. Here is an example of StormyNight in action creating a story idea using just a little bit of context.

As you can see, this product has a lot of potential to help authors write and create stories.

Let's get Social

Stormy Night is not only a tool for creating story ideas, but also the intent is to create a social network for authors to share their ideas, content, and even become a platform for content delivery using GPT-3.
Stormy night is still in a closed beta test. However in the next few releases we will be creating the concept of Author Portals, where an author can post their GPT-3 generated content along with their own edits and create shareable content.
Users will be able to "follow" other authors on the platform, creating their own personalized social feed of content. We will eventually add the ability to comment on and discuss the content as well.

In January 2021, we will open up the site for more users and soon we hope to deliver on the social network features in later parts of 2021.

Addressing AI Risk in Content

With as much hype as it has gotten, GPT-3 is not human and it has been trained from text written by normal people on the public Internet. As a result, GPT-3 is not unbiased and can be used to create risky content such as political misinformation. For this reason we cannot trust GPT-3 alone to generate books and articles. They must be curated by humans, for now.

In order to reduce the risk, OpenAI has also provided a content filter API to help identify the potential risk of content that was generated by it.

StormyNight uses the content filter to score the inputs and outputs of a user's content. We used this to create a content risk score of Low, Medium and High for each piece of content. For now only low risk content will be made available on public feeds. This eliminates most curse words, racist words, and potential political content.

Also, users have the ability to "report" content that is offensive or political in nature. Reported stories are removed from public feeds.

Once the social network has been established we plan to create a new feature called an Author Content Risk score, that will use the GPT-3 risk API to score all of their content and then form an aggregation of their content across multiple posts.

This will allow for the network to develop a level of trust for an author over time. If they post risky content, it will bring their score down and vice versa. This will make it possible for well-meaning authors to create content that can be shared publicly and provide a degree of freedom about what potentially risky content can be shared.

Architecture and Productionalization

StormyNight.ai is a single-page application for web browsers written in React and Node.js using the MERN stack. Node was used to create an "experience" API back-end that connects various microservices together. The application uses the following microservice technologies.

• OpenAI's GPT-3 API
• Auth0 for Identity Authentication and Authorization
• MongoDB Atlas - Data storage

We host the API and client app on Ubuntu using Digital Ocean droplets. We use SSL and OAuth2 scopes to control and rate limit the API.

In order to go to production with a GPT-3 product you must address AI risk with the tool. For this reason users are limited to a certain number of content items, and the input and output parameters must be limited somewhat. This is both for financial and risk reasons because the API calls do have a cost associated with them. One story idea costs around 2 cents to generate.

Conclusion

Right now StormyNight.ai is in a beta testing phase. The number of users who can create content is being limited. However I plan to open it up to a greater degree very soon. Beta testers will be granted the ability to create up to 25 pieces of content without cost.

This is my first "Minimum Viable Product" release, meaning that it provides a real working product, with authentication and everything but some of the key features are not yet developed. This will help me to test the business model behind my product and see if I should continue with it or try another strategy. I hope that the world enjoys my product and uses it to create amazing new content for people to enjoy.