Tracking and budgeting LLM costs
G8KEPR meters every LLM request that passes through the AI Gateway, so you always know what you're spending and by whom. You can also set daily, monthly, and total budget limits that stop or alert on overspend before your bill surprises you.
How cost metering works
Every inference request routed through the gateway is metered automatically — there's nothing to instrument in your application. For each request, G8KEPR:
- ▸Counts input and output tokens
- ▸Looks up current provider pricing for the exact model used
- ▸Records the cost against your organization, the user or key that made the call, the provider, and the model
These records roll up into live totals and a monthly summary, so you can break spend down by any of those dimensions.
Metering covers both G8KEPR-managed provider keys and your own bring-your-own-key (BYOK) credentials. BYOK keys can also carry their own per-key monthly cost limit.
Viewing your costs
- 1.Open the AI Gateway section of your dashboard.
- 2.Select Cost & Usage to see current spend.
- 3.Filter by user, provider, or model, or switch to the monthly view for a full billing-period rollup.
Setting budget limits
Budgets are enforced per organization (and can be scoped to individual users). You can set three limit types:
- ▸Daily — resets every 24 hours
- ▸Monthly — resets each billing period
- ▸Total — a hard lifetime cap
To configure a limit:
- 1.Go to AI Gateway → Budgets.
- 2.Choose the scope (organization or a specific user).
- 3.Enter your daily, monthly, and/or total amounts and save.
New organizations start with sensible default limits, which you can raise or lower at any time.
Alerts
- ▸Threshold alerts notify you as you approach or cross a budget limit, so you can react before traffic is affected.
- ▸Reviewing your Cost & Usage view regularly helps you catch spend that's trending higher than expected, even while you're still under budget.
A hit budget limit blocks further requests for that scope until the period resets or you raise the limit. Set alert thresholds below your hard cap so you're warned before traffic is cut off.