ChatGPT to Write GPT-3 and Python: Duh!

Building Better Search - Using ChatGPT

I've been playing with OpenAI's new Embeddings Models (more to come, wow are they good and cheap now.) building some search stuff, and working on layering GPT logic to build more complex workflows. Part of that is using GPT to clarify and expand search - Of course, right?

My thinking was have GPT parse the query, give me three related searches, then hand each one of those to PineconeDB to get results automatically. Easy, right? A couple lines of prompt, parse the response, split on commas, send it.

There are a few problems with that approach that are obvious now - I'm building MVP's, they don't have to be efficient yet! I managed to avoid the pitfalls of my overthinking by doing something I hadn't much yet - giving ChatGPT space.

Be an LLM's Mentor, not Micromanager

Or: It's smarter than you, stupid.

Inspired by a remarkably good talk on CSS I watched recently I decided to try something new - Just give this thing space with the general idea. Thinking higher level is a challenge for me after spending so long in the weeds. I want to tell LLM's exactly how to solve my problems, but I'm not great at telling them what I actually need solved.

So I tried this:

Try it yourself and see if you get different results:

I want to use gpt3 to generate semantic search queries for it. If someone makes a query I want three helpful semantic search queries that will help answer the question

Query Summarization - The Surprise

It shouldn't have been but I was shocked - It split the task into two parts. Duh, right? Summarize, related queries.

The thing is - it also writes shockingly succinct (but generic) prompts. But they're good! Splitting tasks up is usually a good thing in GPT-land. It lets us use cheaper models to do simpler things, which is faster, cheaper, and more reliable to boot!

It used two prompts, one to summarize the query, and one to generate the queries.

import openai_secret_manager

# Get API key
secrets = openai_secret_manager.get_secrets("openai")

# Use the API key to create a session object
import openai
openai.api_key = secrets["api_key"]

# Define the prompt for generating a summary
prompt = (f"summarize: {input_query}")

# Send the prompt to GPT-3
response = openai.Completion.create(
    engine="text-davinci-002",
    prompt=prompt,
    max_tokens=1024,
    n=1,
    stop=None,
    temperature=0.5,
)

# Get the generated summary
generated_summary = response["choices"][0]["text"]

The important thing there is the prompt - summarize: <whatever>. Duh, right? Why would I overcomplicate it? I don't have to do any fancy parsing either - since I'm sending it straight into GPT.

Note too though that it wrote python for me! OpenAI does this on purpose - Python is what it does best. JS/Html/Cloudformation work pretty well too though.

Related Queries

Likewise, ChatGPT is like "Hey bud, just say related_queries:"

# Define the prompt for generating related queries
prompt = (f"related_queries: {generated_summary}")

# Send the prompt to GPT-3
response = openai.Completion.create(
    engine="text-davinci-002",
    prompt=prompt,
    max_tokens=1024,
    n=3,
    stop=None,
    temperature=0.5,
)

# Get the generated related queries
generated_related_queries = [choice["text"] for choice in response["choices"]]

With the n set to 3, we get our three queries back! It also wrote the boilerplate to parse the text out of the responses - Something I don't want to even think about when I'm shipping an MVP.

Caution

The biggest problem with this approach is ChatGPT isn't aware of recent changes to OpenAI's API (or... any of them). Your best bet there is to paste in the example from the OpenAI Cookbook and ask it to tailor it to your usecase - It's really good at that.

Pinecone is the same way - you'll HAVE TO paste in examples, or you'll get very confident bullshit back.

Improving Results Quickly

The obvious thing is first - Watch what models it recommends. ChatGPT doesn't know about the newest iteration of models either. You should try the GPT-3 Model Comparison for an idea of what model you should use.

The big thing is shoot lower than you think - Curie and Babbage both are probably fine for this task. Just don't use an old model, that's even worse! text-davinci-002 is just old now - and we just don't need the power of 003 to answer simple questions.

Don't overthink it - If you have five questions to ask of data, just do it 5x with a cheaper model. It'll probably do fine (and be half or less the cost of trying to do it in one call with Davinci!)

Use Davinci when you need well written text.

Conclusion

Just use ChatGPT to write GPT-3 code. It's good at it. You can have it refine prompts too, like adding examples, which is especially helpful if you need a particular output format. The biggest thing is to use ChatGPT to get out of your own way - Pitch it an idea and let it work on it for a while. It works really well.

Be careful when working with newer tech - Today it doesn't know what it doesn't know.