by Lazy Sloth
Open Source LLM based Web Chat Interface
import os
import requests
import json
from fastapi import FastAPI
from fastapi.responses import HTMLResponse
from pydantic import BaseModel
import uvicorn
app = FastAPI()
#this is the list of free models in openrouter.ai as of Dec 7, 2023.
MODELS = [
"mistralai/mistral-7b-instruct",
"huggingfaceh4/zephyr-7b-beta",
"openchat/openchat-7b",
"undi95/toppy-m-7b",
"gryphe/mythomist-7b",
]
class Prompt(BaseModel):
text: str
model: str
Open Source LLM based Web Chat Interface
Created: | Last Updated:
Introduction to Open Source LLM based Web Chat Interface
Welcome to the step-by-step guide on how to set up and use the Open Source LLM based Web Chat Interface. This template allows you to create a web chat interface that connects with various open-source language models provided by openrouter.ai. You'll be able to send prompts to these models and receive responses, simulating a conversation. The chat history will be displayed on the web page, with user messages in green and model responses in grey.
To begin using this template, click on "Start with this Template" on the Lazy platform.
Setting Environment Secrets
Before you can interact with the openrouter.ai API, you need to set up an environment secret for your API key. Follow these steps to configure your environment secret:
- Visit openrouter.ai and register for an account to obtain your free API key.
- Once you have your API key, go to the Environment Secrets tab within the Lazy Builder interface.
- Create a new secret with the key `OPENROUTER_API_KEY` and paste your API key as the value.
This API key will be used to authenticate your requests to the openrouter.ai API.
Using the Test Button
After setting up your environment secret, you can use the Test button to deploy your app. The Lazy CLI will handle the deployment process, and you will not need to provide any additional input at this stage.
Interacting with the Web Chat Interface
Once your app is deployed, Lazy will provide you with a dedicated server link to access your web chat interface. If you're using FastAPI, you will also receive a link to the API documentation.
To interact with the chat interface:
- Open the provided server link in your web browser.
- You will see a web page with a chat interface and a dropdown menu to select one of the available language models.
- Type your message into the input field at the bottom of the page.
- Choose the model you wish to converse with from the dropdown menu.
- Click the "Send" button or press "Enter" to submit your prompt.
- The conversation will update with your message in green and the model's response in grey.
- While the model is generating a response, the send button will show a spinner, indicating that the process is ongoing.
Enjoy your conversation with the open-source language models! Remember, you can always switch between different models to explore various responses and capabilities.
Conclusion
By following these steps, you should now have a fully functional Open Source LLM based Web Chat Interface. This guide has walked you through setting up your environment secret, deploying the app with the Test button, and interacting with the web chat interface. If you encounter any issues or have further questions, please refer to the documentation provided by openrouter.ai or reach out for support.
Technologies
![Python](https://i.pinimg.com/originals/2f/9c/11/2f9c11f9e55efbf1791f12c06d60729b.jpg)
![Javascript](https://upload.wikimedia.org/wikipedia/commons/thumb/6/6a/JavaScript-logo.png/600px-JavaScript-logo.png)