Overview
I built a Python application that combined the fine-tuned model with retrieval of product and policy information. We then commissioned the developer of the company’s live chat software to integrate my application into the existing system, allowing sales staff to use AI suggestions within their usual chat interface.
The problem
The fine-tuned model and vector database needed an application around them before they could be used in customer live chats. Each incoming message had to be combined with the conversation history, relevant product and policy information had to be retrieved, and the resulting answer had to be returned to the company’s live chat software. This required coordinating the locally stored vector database with the external embedding and language-model APIs.
I also needed a practical way to test different model versions and response settings.
The application needed a consistent runtime environment, with its Python dependencies, embedding dataset and configuration available wherever it was hosted.
What was achieved?
I built a testing application for trained models to evaluate training and variations on the RAG (vector DB).
Sales staff also used the application to evaluate the model and report incorrect responses.
We commissioned the developer of the company’s live chat software to integrate my Python application into the existing operator interface. Employees could review AI-generated suggestions, send them directly, edit them before sending, or write their own replies.
I also requested a report button so employees could flag poor responses. The integration recorded the model’s input and suggested reply, together with the associated chat and employee, for later review.
The model was trained to use an <employee> placeholder instead of an employee’s name. The integration replaced this placeholder with the name of the employee handling the chat, so the same model could generate personalised greetings for different members of staff.