Data retention

Velvet has data retention and archival features to improve performance, reduce storage costs, and provide easy access to historical data for analysis. Features vary based on pricing tier.

See plan limits and pricing pages for more.

Data types

We think about data in two ways; (1) short-term "working memory" and (2) long-term "historical" data.

Working memory data is the dataset within the current retention window. The logs in working memory reside in the Postgres log database. Use logs for Velvet features like observability, evaluations, and analytics. Connect to the Postgres database directly to access raw request logs.

Historical data is the dataset that Velvet can archive after the retention window has elapsed. Store this to your data warehouse (like S3), and access it using an analytics database (like ClickHouse).

Data retention

Our data retention policies keep your log database fast and performant. We maintain a large enough working memory dataset to enable observability, evaluations, and caching.

Data archive

Velvet can archive data beyond our retention window to long-term storage. This drastically reduces Postgres storage costs by writing archival data to S3 via the Parquet file format. The Parquet format enables easy access to the historical data via tools like ClickHouse and DuckDB for analytics use cases.