Lazy Sloth
by Lazy Sloth

Open Source LLM based Web Chat Interface

Start with this template
428
Start with this template
428 
import os
import requests
import json
from fastapi import FastAPI
from fastapi.responses import HTMLResponse
from pydantic import BaseModel
import uvicorn

app = FastAPI()

#this is the list of free models in openrouter.ai as of Dec 7, 2023.
MODELS = [
    "mistralai/mistral-7b-instruct",
    "huggingfaceh4/zephyr-7b-beta",
    "openchat/openchat-7b",
    "undi95/toppy-m-7b",
    "gryphe/mythomist-7b",
]

class Prompt(BaseModel):
    text: str
    model: str
Get full code

Open Source LLM based Web Chat Interface

Created: | Last Updated:

Introduction to Open Source LLM based Web Chat Interface

Welcome to the step-by-step guide on how to set up and use the Open Source LLM based Web Chat Interface. This template allows you to create a web chat interface that connects with various open-source language models provided by openrouter.ai. You'll be able to send prompts to these models and receive responses, simulating a conversation. The chat history will be displayed on the web page, with user messages in green and model responses in grey.

To begin using this template, click on "Start with this Template" on the Lazy platform.

Setting Environment Secrets

Before you can interact with the openrouter.ai API, you need to set up an environment secret for your API key. Follow these steps to configure your environment secret:

  • Visit openrouter.ai and register for an account to obtain your free API key.
  • Once you have your API key, go to the Environment Secrets tab within the Lazy Builder interface.
  • Create a new secret with the key `OPENROUTER_API_KEY` and paste your API key as the value.

This API key will be used to authenticate your requests to the openrouter.ai API.

Using the Test Button

After setting up your environment secret, you can use the Test button to deploy your app. The Lazy CLI will handle the deployment process, and you will not need to provide any additional input at this stage.

Interacting with the Web Chat Interface

Once your app is deployed, Lazy will provide you with a dedicated server link to access your web chat interface. If you're using FastAPI, you will also receive a link to the API documentation.

To interact with the chat interface:

  • Open the provided server link in your web browser.
  • You will see a web page with a chat interface and a dropdown menu to select one of the available language models.
  • Type your message into the input field at the bottom of the page.
  • Choose the model you wish to converse with from the dropdown menu.
  • Click the "Send" button or press "Enter" to submit your prompt.
  • The conversation will update with your message in green and the model's response in grey.
  • While the model is generating a response, the send button will show a spinner, indicating that the process is ongoing.

Enjoy your conversation with the open-source language models! Remember, you can always switch between different models to explore various responses and capabilities.

Conclusion

By following these steps, you should now have a fully functional Open Source LLM based Web Chat Interface. This guide has walked you through setting up your environment secret, deploying the app with the Test button, and interacting with the web chat interface. If you encounter any issues or have further questions, please refer to the documentation provided by openrouter.ai or reach out for support.

Technologies

OpenAI OpenAI
Python Python
Javascript Javascript