Observability
Lepton provides a comprehensive observability platform to help you monitor and troubleshoot your applications, including logs, audit logs, and health checks.
Logs
Learn how to use real-time logs and advanced logs.
Log storage and persistency features are only available for enterprise customers.
What are logs?
Logs are generated by the replica in endpoint, job and pod, including all standard output and error output (print
or echo
).
With real-time logs:
- Logs are real-time and can be viewed as soon as the replica in the deployment, job and pod are running. Real-time logs will no longer be available once they are deleted or terminated.
- Real-time logs can only be read from the last 10,000 rows
With advanced logs:
- Support search and filter
- View log context
- Query at the deployment, job, pod and Workspace levels
- Advanced logs can keep logs generated within 7 days at most
Real-time Logs
To view real-time logs:
- From the dashboard, navigate to the deployment, job or pod detail page.
- Select the replicas tab.
- Click the Logs button to view the real-time logs.
You can also click on the Live button on the right top corner to switch the live mode.
Advanced Logs
Advanced logs are only available under Enterprise plan.
To view advanced logs:
- From the dashboard, select the Observability tab and select Logs.
- Or, select the Logs tab from the deployment, job or pod detail page.
Filters
You can use the following filters from the left sidebar to get a refined search experience.
Filter | Description |
---|---|
Timeline | Query logs by specific time, including the maximum (7 days) or a custom time range, default is most recent 24 hours. |
Scope | Select or enter a deployment, job, or pod name to query the logs of all replicas within that scope. |
replica | Select or enter a specific replica name to query the logs of that replica. |
Log Context
It is helpful to see the log context when the filter matches multiple time periods or scopes.
Mouse over the line whose context you want to view, and click the show context button to view the log context. In the log context dialog, the current line will also be highlighted, along with its context
Audit Logs
Learn how to view audit logs in Lepton.
Audit logs are available under Enterprise plan.
What are audit logs?
Audit log records actions and API calls performed by members of your workspace. It helps you to track the changes made by members in your workspace.
View audit logs
From the dashboard, select the Observability tab and select Audit Logs.
Health Checks
Learn how to create health checks in Lepton AI.
Health checks are only available under Enterprise plan.
What is health checks?
Health checks involve observing and checking the status of your resources and services. It helps you understand their performance and health.
You can also configure alerts to get notified when a specific condition is met.
Create health checks
From the Dashboard, select the Observability tab and select Health Checks. Then click the Create button to create health checks.
- Name: The name of the monitor.
- Alert Destination: The alert destination to send the alert to.
- URL: The URL and method to monitor, uou can select the API URL from deployment.
- Body: The body of the request.
- Headers: The headers of the request (use JSON format), you may also to add
Authorization
header if needed. - Expected Status: The HTTP status codes to expect in the response. default is
200
and you can add multiple status codes. - Expected Regex Match: The regex pattern to match in the response body, e.g.
.*success":true.*
. - Check Interval: How often to check the health of the target, in seconds.
- Maximum Latency: The maximum latency allowed for the target, in milliseconds.
- Consecutive Success Count: The number of consecutive successful checks required to mark the target as healthy.
- Consecutive Failure Count: The number of consecutive failed checks required to mark the target as unhealthy.
Configure alerts
From the Dashboard, select the Observability tab and select Health Checks. Then click the Alert Configuration button to create alerts.
- Name: The name of the alert.
- Webhook: The URL to send the alert to, we send webhooks using an Slack webhooks format.
Mute alerts
You can also mute the alerts by click the mute button on the right side of the list or on the details page.
Telemetry
Lepton also supports to export prometheus metrics that are used to collect and forward metrics to third-party metric providers. Navigate to the Telemetry tab under the Observability page, and you can configure for exporting Prometheus metric here.
Now we only support integration with Google Cloud, press the configure button and paste your Google Cloud Accees Control then the telemetry will be set up in a minute.
You need to make sure you have open the required permissions.