Clients

When you create a deployment, it exposes multiple endpoints for you to interact with. There are multiple ways to interact with the deployment, including:

• Python Client
• Raw cURL calls
• Web UI

For the following example, we assume that you have already logged into your workspace. If not, refer to the quickstart for login instructions. We'll also assume that you have created, pushed and run photon like follows:

lep photon create -n myphoton -m hf:gpt2
lep photon push -n myphoton
lep photon run -n myphoton 

and you have a deployment named myphoton running the gpt2 model.

Python client

Python client is the most common way to connect to a running deployment, especially if you are building AI application logic in Python.

To construct a client, you will need three pieces of information:

• Your workspace id
• Your workspace token
• Your deployment name For the workspace id and token, if you are logged in in the CLI, you can obtain them via lep workspace id and lep workspace token. Otherwise, you can find them in the setting page of the web UI. After obtaining the information, you can create a client in python. The client will automatically query the deployment, and convert all endpoints into python methods. To see a list of all available methods, you can use client.paths(). To see the specified documentation for a method, you can use help(client.methodname). For example, for the myphoton deployment, you can do:

from leptonai.client import Client
WORKSPACE_ID = "YOUR_WORKSPACE_ID"
DEPLOYMENT_NAME = "myphoton"
TOKEN = "YOUR_API_TOKEN"

client = Client(WORKSPACE_ID, DEPLOYMENT_NAME, token=TOKEN)
# Print all available methods
print(f"Avaliable methods: {['\n' + path for path in client.paths()]}")
# Inspect the doc for `run`
print(f"\nDocumentation for client.run:\n{client.run.__doc__}")

Now, you can call the run method as if it is a regular python function:

print(client.run(inputs="I enjoy walking with my cute dog"))

Raw cURL calls

Lepton's deployment uses standard RESTful API format. You can use any HTTP client to interact with the deployment. For example, in the above example, you will be able to interact with the deployment using the following cURL command (remember to replace the workspace id and token with your own):

export WORKSPACE_ID=`lep workspace id`
export WORKSPACE_TOKEN=`lep workspace token`
curl -X 'POST' \
  "https://${WORKSPACE_ID}-myphoton.cloud.lepton.ai/run" \
  -H 'Content-Type: application/json' \
  -H 'accept: application/json' \
  -H "Authorization: Bearer $WORKSPACE_TOKEN" \
  -d '{
  "do_sample": true,
  "inputs": "I enjoy walking with my cute dog",
  "max_length": 50,
  "top_k": 50,
  "top_p": 0.95
  }'

To retrive the full list of openapi endpoints, you can use the following command:

curl -X 'GET' \
  'https://${DEPLOYMENT_ENDPOINT}/openapi.json' \
  -H 'accept: application/json' \
  -H "Authorization: Bearer $WORKSPACE_TOKEN"

This will return a json file with all the endpoints. You can also use the web UI to view the openapi spec. Go to the deployment page, and click on the "API" tab. You will see a list of all the endpoints, and you can click on each endpoint to see the documentation and example input and output.

Web UI

In addition to the python client and raw cURL command, the Web UI provides interfaces for you to interact with the model from the web. The web UI also gives you example prefilled code in python and with a curl command which you can use to interact with the model - you can copy and paste the code into your own application. Look for the "API" tab under the deployment page, and choose "Python" or "HTTP" from the option on the right side of the page.

For your convenience - the entry for the web UI is at https://dashboard.lepton.ai/.

More clients?

We are continuing to add more client types, such as JavaScript. If you would like us to prioritize a client in a specific language, please let us know.