📑Data storage

How to work with Databutton's in-built storage.

Databutton includes built-in storage compatible with various popular file formats, such as text files, CSV files, JSON files, binary files, and even pandas DataFrames.

Databutton Agent can read and write files using the Databutton Python SDK.

Databutton storage is suitable for testing and small-scale production apps. For larger file sizes and scaling up, it's advisable to use external databases like Firebase or Supabase.

Firebase Integration

Read the common usage examples of the storage SDK below.

#storage-endpoints-usage-examples

Alternatively, manually any files can be uploaded as well,

1. Navigate to the "Storage" tab on the Databutton platform.

2. Click on "Upload my data" to initiate the upload process.

3. Select the data file you wish to upload from your device.

Examples using the Databutton's Storage endpoint.

Creating a new dataframe

To create a new or replace an existing dataframe use the following API from any view or job:

import databutton as db
import pandas as pd

# This can be data scraped from API, input from a form,
# or anything else that fits in a dataframe.
data_to_store = []
df = pd.DataFrame(data=data_to_store)

# Save the dataframe 'df' to the key 'my-dataframe',
# making it accessible in your views, jobs, and web UI
db.storage.dataframes.put("my-dataframe", df)

Reading from a dataframe

To access a dataframes in a job or view, Databutton provides the following API:

import databutton as db

# Returns the dataframe stored under 'my-dataframe' or
# an empty dataframe if it does not exist.
df = db.storage.dataframes.get(key="my-dataframe")

By using db.storage.dataframes.get(key), you can access your dataframes anywhere.

Deleting a dataframe

import databutton as db

# deletes the dataframe with the given key
df = db.storage.dataframes.delete(key="my-dataframe")

This will delete the whole dataframe, also its entry in the Data Storage.

Adding records to a dataframe

In order to add new records to a dataframe, e.g. in a scraping job, you could do the following:

import databutton as db
import pandas as pd

# Fetch existing dataframe 'my-dataframe', or if it does
# not exist call the 'default' argument to produce the default value.
# Here we default to returning an empty dataframe.

existing_df = db.storage.dataframes.get(
    key="my-dataframe", default=lambda: pd.DataFrame()
)

# Data to add to the dataframe. Could be scraped from an API of your choice.
data = [{"x": 10, "y": 5}]
new_df = pd.DataFrame(data)

# Concatinate the two dataframes
df_to_store = pd.concat([existing_df, new_df], ignore_index=True)

# Store the dataframe safely in Databutton
db.storage.dataframes.put(key="my-dataframe", value=df_to_store)

Working with json files

Use json files to work with nested dict-like structures for smaller and unstructured data. An example use case is taking basic form data input in a view:

import databutton as db
import streamlit as st

config = db.storage.json.get(
    "config",
    default=lambda: {"email":"[e-mail]"}
)

# Use streamlit form
with st.form("my-form"):
    email = st.text_input("Email", placeholder=config.get("email", ""))

    if st.form_submit_button("Submit"):
        # Define a dict with configuration to store as json
        config = {"email": email}

        # Replace existing config
        db.storage.json.put("config", config)

        # Show in view that config has been safely stored
        st.write("Notification email has been changed!")

Working with text files

Text files are simply fetched as Python strings. If you need to pass it to a library expecting a text file you can wrap the str in io.TextIO like this:

import databutton as db
# Get some html page from somewhere
scraped_html = """
Some <b>very</b> interesting content.
"""

# Store it as text in your data app
db.storage.text.put("webpage.html", scraped_html)

# Later retrieve it from databutton storage
html = db.storage.text.get("webpage.html")

Working with pickle

Pickle files works out of the box with db.storage.binary, here's an example

import databutton as db
import pickle

random_object = {1: 2, 3: 4, "foo": "bar"}
st.write(random_object)

pickled = pickle.dumps(random_object)
db.storage.binary.put("pickled-obj", pickled)

stored_obj = db.storage.binary.get("pickled-obj")
unpickled = pickle.loads(stored_obj)

Working with any other type of file

A file of a type unknown to Databutton can still be stored as a binary file, and fetched as a Python bytes object. If you need to pass it to a library expecting a file as input or output you can wrap the bytes in io.BytesIO like in this basic image processing example using pillow:

import io

import databutton as db
import requests
from PIL import Image

# Get image from URL
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

# Apply some image processing
image = image.rotate(45)

# Save image to in-memory bytes buffer
image_buffer = io.BytesIO()
image.save(image_buffer, format="jpeg")

# Put it to databutton storage
image_buffer.seek(0)
db.storage.binary.put("img.jpeg", image_buffer.read())

# Read it back from databutton storage
image_bytes = db.storage.binary.get("img.jpeg")
image = Image.open(io.BytesIO(image_bytes))