Are you tired of ChatGPT’s outdated responses when it comes to your favorite open-source Python library, Streamlit? In this tutorial, we’re going to show you how to build a custom chatbot using LlamaIndex and GPT-3.5 for accurate documentation answers that are up-to-date with the latest releases.
To set the stage, what LlamaIndex is all about. It’s an open-source framework designed specifically for building RAG (Retrieval, Analysis, Generation) pipelines using LLMs (Large Language Models). In simpler terms, it helps us augment our models with custom data sources to provide more accurate and relevant responses based on the context of the question being asked.
Now Let’s begin exploring with how we can use LlamaIndex and GPT-3.5 for a Streamlit documentation chatbot. Here are the steps:
1. Load your data In this case, we’re using the markdown files that make up Streamlit’s documentation as our custom data source. You can sub in your own data if you want. To load and index it, use LlamaIndex to create a vector store index. Here’s an example script:
# Import necessary libraries
from llama_index import SimpleDirectoryReader, ServiceContext, VectorStoreIndex
import openai
# Define input directory and OpenAI API key
input_dir = "./data" # replace with your own data directory path
openai_key = "YOUR_OPENAI_KEY"
# Create an instance of the OpenAI API client using the provided key
llm = openai.APIClient(openai.Configuration({"api_key": openai_key}))
# Create a service context using the OpenAI API client
service_context = ServiceContext.from_defaults(llm=llm)
# Create a SimpleDirectoryReader object to read data from the specified input directory
reader = SimpleDirectoryReader(input_dir="./data", recursive=True)
# Load data from the input directory using the SimpleDirectoryReader object
docs = reader.load_data()
# Create a VectorStoreIndex using the loaded data and the service context
index = VectorStoreIndex.from_documents(docs, service_context=service_context)
# The above script loads data from a specified directory and creates a vector store index using the OpenAI API.
2. Create the chat engine LlamaIndex offers several different modes of chat engines to choose from based on your use case and knowledge base. For our Streamlit documentation chatbot, we’re going to use the Condense question engine which always queries the knowledge base for an answer. Here’s how you can create it:
# Import necessary libraries
from llama_index import SimpleDirectoryReader, ServiceContext, VectorStoreIndex, PromptSource, Assistance
import openai
# Define input directory and create OpenAI client
input_dir = "./data" # replace with your own data directory path
llm = openai.APIClient(openai.Configuration({"api_key": "YOUR_OPENAI_KEY"}))
# Create service context using default settings
service_context = ServiceContext.from_defaults(llm=llm)
# Create a simple directory reader to load data from input directory
reader = SimpleDirectoryReader(input_dir="./data", recursive=True)
# Load data from input directory using the reader
docs = reader.load_data()
# Create a vector store index using the loaded documents and service context
index = VectorStoreIndex.from_documents(docs, service_context=service_context)
# Create a prompt source using a query function that takes in a question and returns it
prompt_source = PromptSource.from_query_function(lambda q: q)
# Create an assistant using the index and prompt source
assistant = Assistance(index, prompt_source=prompt_source)
# The above script imports necessary libraries and creates an assistant using the Condense question engine from LlamaIndex. It also sets up a prompt source and creates a vector store index using the loaded documents and service context.
3. Build the chatbot app Now that we have our data loaded and indexed, let’s build a Streamlit chat UI using st.chat_input() and st.chat_message(). We can also store and update the chatbot’s message history using session state. Here’s an example script:
# Import necessary libraries
import streamlit as st # Importing the streamlit library and aliasing it as "st"
from llama_index import SimpleDirectoryReader, ServiceContext, VectorStoreIndex, PromptSource, Assistance # Importing necessary classes from llama_index module
import openai # Importing the openai library
# Set input directory and initialize OpenAI client
input_dir = "./data" # Setting the input directory path
llm = openai.APIClient(openai.Configuration({"api_key": "YOUR_OPENAI_KEY"})) # Initializing the OpenAI client with the API key
# Create service context, reader, and load data
service_context = ServiceContext.from_defaults(llm=llm) # Creating a service context with default settings and passing in the OpenAI client
reader = SimpleDirectoryReader(input_dir="./data", recursive=True) # Creating a SimpleDirectoryReader object with the input directory path and recursive flag set to True
docs = reader.load_data() # Loading data from the input directory using the reader
# Create index and prompt source
index = VectorStoreIndex.from_documents(docs, service_context=service_context) # Creating a VectorStoreIndex object from the loaded documents and service context
prompt_source = PromptSource.from_query_function(lambda q: q) # Creating a PromptSource object using a lambda function that returns the query
# Create assistant
assistant = Assistance(index, prompt_source=prompt_source) # Creating an Assistance object using the index and prompt source
# Set up Streamlit chat UI
st.title("Streamlit Documentation Chatbot") # Setting the title of the chatbot app
st.header("Ask me anything about Streamlit!") # Setting the header of the chatbot app
if st.session_state.messages[-1]["role"] != "assistant": # Checking if the last message in the session state was from the assistant
with st.chat_message("assistant"): # Using the chat_message function to display a message from the assistant
with st.spinner("Thinking..."): # Using the spinner function to display a loading indicator
response = assistant.query(st.session_state.messages[-1]["content"]) # Querying the assistant with the last message from the user
st.write(response) # Displaying the response from the assistant
message = {"role": "assistant", "content": response} # Creating a message dictionary with the role set to "assistant" and the content set to the response
st.session_state.messages.append(message) # Appending the message to the session state messages list
4. Deploy the app! After building your chatbot, deploy it on Streamlit Community Cloud by creating a GitHub repository and following these steps: https://www.streamlit.io/cloud-deployment#create-a-new-app
That’s it! You now have an accurate Streamlit documentation chatbot that uses LlamaIndex to augment GPT-3.5 with your custom data sources for more relevant and up-to-date answers based on the context of the question being asked.