Performance
Total cost
Performance
Total cost
Definition
The total cost test ensures that the total request cost (in USD) for the data is within a given range.
To enable this test, the data published/streamed to Openlayer must have a
column with the cost for each row. In the data
config, the name of this column is
specified in the costColumnName field.
Taxonomy
- Category: Performance.
- Task types: LLM.
- Availability: and .
Why it matters
- If you use an external API to get the model predictions, you might be charged for each request. This is the case for most third-party LLMs, such as OpenAI GPTs. Setting up tests for the cost of the requests is important to avoid unexpected costs.
Related
Was this page helpful?