Chat with a Website

Learn how to extract data from websites and create a basic AI-driven chat system that integrates with those websites.

Segregating the Backend and the Frontend for the App

While building apps using Databutton, one cruicial step to keep in mind is the clear separation and definition of the app's architecture.

A. Backend (Functional Endpoints)

Defining the backend involves identifying the necessary tasks that app needs to perform. We can implement the "Chat with Website" app with two main functional endpoints-

• / scrape-website : This endpoint takes a URL as input and returns the scraped content of the website as output.

• / ai-chat : This endpoint receives a context and prompt, generates a conversational AI response using OpenAI's API, and returns the response.

Here's few example prompts while generating the `scrape-website` endpoint.

Prompting Tip : When communicating with AI agents, it's beneficial to be explicit about both the information you're providing (input) and what you expect back (output). If possible, providing an example, such as a URL, can enhance the AI's comprehension.

The next step involves talking to the capability agent — this is where it is about creating a FastAPI router. This router is a crucial piece because it’s what the front end of our application will communicate with. Again, the capabability agents builds it seamlessly!

B. Building the front-end and connecting all the pieces

In this application, we have opted for a minimal user interface (UI) for the chat with website app. Therefore, we can easily consolidate the entire UI into a single application (or page) instead of dividing it into smaller, distinct building blocks known as “components” within Databutton.

Quick note: Using a component-based approach for UI construction is generally considered a best practice within the Databutton environment.

Let’s walk through where the UI integrates with the endpoints within the code:

1. Importing the Capability-related module: The part import brain from “brain” , is where Databutton agents ensure that the UI interacts with the backend services for tasks like fetching website data and creating AI chat responses.

2. API Interaction for Website Data Fetching: Once, the brain module is imported, we can easily call the rest of its functionalities. For instance, the code brain.scrape_website({ url }) initiates an API call to retrieve content from a specified URL. As mentioned earlier, the purpose of this endpoint is to dynamically obtain and display web content within the UI.

See below for the relevant code snippet…

...

  const handleFetchData = async () => {
    console.log("Fetching data for URL:", url);
    try {
      const response = await brain.scrape_website({ url });
      if (response.ok) {
        const data = await response.json();
        console.log("Scraped data:", data);
        setScrapedData(data.content);
      } else {
        console.error("Failed to fetch scraped data:", response);
        setScrapedData(
          "Failed to fetch data. Please check the console for more details.",
        );
      }
    } catch (error) {
      console.error("Error fetching scraped data:", error);
      setScrapedData(
        "Failed to fetch data. Please check the console for more details.",
      );
    }
  };
...

3. API Interaction for AI-Generated Conversations: This code snippet below shows best how the endpoint brain.ai_chat({ context, prompt }) is utilized to send user input and context for generating AI responses. Again, the purpose is to facilitate interactive Q&A sessions within the UI, enhancing user engagement.

...

const handleSendMessage = async () => {
    const prompt = userInput;
    const context = scrapedData;
    try {
      const response = await brain.ai_chat({ context, prompt });
      if (response.ok) {
        const data = await response.json();
        setChatMessages([...chatMessages, { prompt, response: data.response }]);
        setUserInput("");
        if (inputRef.current) inputRef.current.focus();
      }
    } catch (error) {
      console.error("Error sending message:", error);
    }
  };

...

In all above, we strictly focused on code segments crucial for API interactions which was then carried out by the Databutton AI agents. We skipped over other aspects like UI management and error handling (which you will encounter) to focus on these key points.