Medical Chatbot with LLMs

Overview

Implemented a retrieval-augmented chatbot as an educational project: GPT-4o answers questions using passages retrieved from a medical reference book indexed in Pinecone.

The project covers the path from document ingestion to a containerized Flask app, with a GitHub Actions workflow that builds the Docker image, pushes it to Amazon ECR and runs it on an EC2 instance.

Approach

The system prompt tells the model to answer from the retrieved context, to say when it doesn’t know, and to keep answers to three sentences.

Architecture

1. Ingestion

   DirectoryLoader and PyPDFLoader read the source PDF, and each page keeps only its source as metadata.

2. Chunking

   RecursiveCharacterTextSplitter with 500-character chunks and a 20-character overlap.

3. Embeddings and vector store

   HuggingFace all-MiniLM-L6-v2 embeddings (384 dimensions) stored in a Pinecone serverless index with cosine similarity.

4. Retrieval and generation

   Similarity search returns the top 3 chunks, and a LangChain retrieval chain passes them to GPT-4o with the system prompt.

5. App and deployment

   Flask serves the chat page and a message endpoint. GitHub Actions builds the Docker image, pushes it to Amazon ECR and runs it on EC2 through a self-hosted runner.

Technology

• Python
• LangChain
• HuggingFace
• Pinecone
• GPT-4o
• Flask
• Docker
• AWS ECR
• AWS EC2
• GitHub Actions

Limitations

• There is no evaluation of retrieval quality or answer quality.
• Answers rely on a single reference source, which isn’t redistributed with the repository.
• Retrieved passages aren’t shown to the user as citations.
• There is no clinical validation. It isn’t intended for diagnosis or treatment recommendations and isn’t production healthcare software.