How To Share A Local LLM App With Gradio
· 7 min read · Views --
Last updated on

How To Share A Local LLM App With Gradio

Author: Alex Xiang

Artificial Intelligence

How To Share A Local LLM App With Gradio

Gradio can be used to share a local LLM application quickly. It is a simple and powerful Python library for creating interactive machine-learning and data applications. It lets you build a Web UI for models, visualizations, and other features with very little code.

By setting share=True in launch(), a Gradio demo can be exposed through a temporary public link:

import gradio as gr

def greet(name):
    return "Hello " + name + "!"

demo = gr.Interface(fn=greet, inputs="textbox", outputs="textbox")

demo.launch(share=True)  # Share your demo with just 1 extra parameter

Running this example produces output similar to:

python demo/test.py
Running on local URL:  http://127.0.0.1:7860
IMPORTANT: You are using gradio version 3.48.0, however version 4.44.1 is available, please upgrade.
--------
Running on public URL: https://547eabbc00701b541c.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)

This generates a public share link such as:

https://547eabbc00701b541c.gradio.live

You can send the link to other people, and they can use the model directly in their browser. This is very useful for quick demos.

Opening the link in a browser shows a simple page with an input box and an output box:

The share link expires after 72 hours. Also remember that a shared link is publicly accessible. Anyone who has the link can use your demo. Do not expose sensitive data, sensitive functions, or local operations that can modify important files.

If you want a permanent public link for an application, use Hugging Face Spaces. Spaces provides free permanent hosting for machine-learning demos.

After creating a free Hugging Face account, there are two common ways to deploy a Gradio app to Spaces:

  • Terminal: run gradio deploy in the application directory. The CLI collects basic metadata and starts the app. To update the Space, rerun the command or enable GitHub Actions to update on git push.
  • Browser: upload the folder that contains the Gradio app and related files.

For details, refer to the official Gradio and Hugging Face Spaces documentation.