by moonboy

AI Scraper with Google Gemini Flash

Test this app for free
32
Test this app for free
32 
import json
from abilities import llm
from fastapi import Request
import logging

import uvicorn
from fastapi import FastAPI
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from fastapi.staticfiles import StaticFiles
from selenium_utils import SeleniumUtility

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

app = FastAPI()
templates = Jinja2Templates(directory="templates")
app.mount("/static", StaticFiles(directory="static"), name="static")

@app.get("/", response_class=HTMLResponse)
async def form_page(request: Request):
    return templates.TemplateResponse("form.html", {"request": request})

@app.post("/process-page-info", response_class=HTMLResponse)
Get full code

AI Scraper with Google Gemini Flash

Created: | Last Updated:

Web application to help scraping specific webpages. This selenium app sends the code of the page to Gemini flash 1.5 and asks it to retrieve the answer and points out the element on the page to do so.

Here's a step-by-step guide for using the AI Scraper with Google Gemini Flash template:

Introduction

The AI Scraper with Google Gemini Flash template is a powerful web application that helps you scrape specific webpages using Selenium and Google's Gemini 1.5 Flash AI model. This app allows you to input a URL and a question, then retrieves the relevant information from the webpage and provides you with a detailed answer, the HTML element containing the answer, and Python code to extract the answer using Selenium WebDriver.

Getting Started

To begin using this template, follow these steps:

  1. Click "Start with this Template" in the Lazy Builder interface.
  2. Press the "Test" button to deploy the application.

Using the App

Once the app is deployed, you'll receive a server link to access the web interface. Follow these steps to use the AI Scraper:

  1. Open the provided server link in your web browser.
  2. You'll see a form with two input fields:
  3. URL: Enter the webpage URL you want to scrape.
  4. Question: Enter the question you want to answer based on the webpage content.
  5. Click the "Submit" button to process your request.
  6. Wait for the AI to analyze the webpage and generate a response.

Understanding the Results

After processing your request, the app will display the following information:

  1. Answer: A detailed response to your question based on the webpage content.
  2. HTML Element: The specific HTML tag containing the answer on the webpage.
  3. Selenium Code: Python code snippet to extract the relevant content using Selenium WebDriver.

Integrating the App

This AI Scraper can be integrated into other applications or workflows. Here's a sample code snippet to make a request to the API endpoint:

```python import requests

url = "https://your-lazy-app-url.com/process-page-info" data = { "url": "https://example.com", "question": "What is the main topic of this webpage?" }

response = requests.post(url, data=data) result = response.json()

print("Answer:", result["answer"]) print("HTML Element:", result["html_element"]) print("Selenium Code:", result["selenium_code"]) ```

Replace "https://your-lazy-app-url.com" with the actual URL provided by Lazy for your deployed app.

By following these steps, you can easily use the AI Scraper with Google Gemini Flash to extract and analyze information from webpages efficiently.

Technologies

CSS CSS
HTML HTML
Python Python
Selenium Selenium
Gemini Gemini