How I reduced OpenAI Costs and Token Usage by 40% with Python

Just wanted to show a simple tip to reduce your OpenAI API costs (I know it’s already cheap, but with bulk content creation, anything you can reduce is good).

How the prompt reducer works

The first thing you need to know is that our prompt is going to pass in two cleaning phases:

• First, we will remove the stop words from the text. Stop words refer to a collection of frequently employed words within a given language. For instance, in the English language, words like “the,” “is,” and “and” are prime examples of stop words.

• Then, we will lemmatize all the verbs. The lemmatizer is a process that reduces words to their base or root form, which helps in simplifying and standardizing verb forms and makes it easier for language models like GPT to understand and generate text.

How to Use It

Read the Blog Post: Explore a detailed guide on how I reduced OpenAI costs and token usage by 40% with Python. Read the Blog Post
Streamlit App: Easily reduce your prompts using the Streamlit web application. Try the Streamlit App
Manual Integration: If you prefer manual integration, you can copy the preprocess_text function from the prompt-reducer.py github file and use it in your own projects.