Cached queries

Below are some example SQL queries that might be helpful when querying cached logs. You can run these queries in the Velvet AI SQL editor, or any other tool you're comfortable with.

Cached logs

Caching unlocks some additional metadata stored with each warehoused log. If the velvet-cache-enabled header is set, the gateway will respond with a velvet-cache-status header.

velvet-cache-status will be one of HIT, MISS, NONE/UNKNOWN

Example SQL queries

Show the difference in price between cached and not cached requests.

SELECT 
  (metadata->'cache'->>'enabled')::boolean AS cache_enabled,
  SUM((metadata->'cost'->>'input_cost')::numeric) AS total_input_cost,
  SUM((metadata->'cost'->>'output_cost')::numeric) AS total_output_cost,
  SUM((metadata->'cost'->>'total_cost')::numeric) AS total_cost
FROM llm_logs
GROUP BY cache_enabled
ORDER BY cache_enabled DESC;

Break down expected vs. actual token costs.

SELECT
  COALESCE(metadata->'usage'->>'model', 'unknown') AS model,
  SUM((metadata->'cost'->>'total_cost')::numeric) AS actual_total_cost,
  SUM((metadata->'expected_cost'->>'total_cost')::numeric) AS expected_total_cost
FROM llm_logs
WHERE metadata->'usage'->>'model' IS NOT NULL
GROUP BY model
ORDER BY model ASC;