ChatGPT database connector caching: define freshness, scope, and invalidation
A ChatGPT database connector can return a correct answer and still create the wrong decision.
The query was valid when it ran. The customer changed status five minutes later, an access grant was revoked, or the finance cutoff moved. The next user receives the cached result without knowing that its data or policy context has expired.
Caching is not only a performance feature. For AI database workflows, it is a data contract.
Start with the decision, not the cache hit rate
Different questions tolerate different ages. A product-catalog summary may remain useful for an hour. An incident dashboard, entitlement check, or payment status may be unsafe after seconds.
Define a freshness class for every approved operation:
- reference: slow-changing definitions and catalogs;
- operational: queues, inventory, and service state;
- transactional: balances, access, orders, and approvals;
- uncacheable: sensitive detail or data whose authorization must be evaluated live.
The model should not choose that class. It belongs to the tool definition and policy.
For a broader framework, see freshness windows for AI database agents.
Cache a governed operation, not arbitrary SQL
A raw SQL string is a weak cache identity. Equivalent queries can have different text, and a small prompt-driven change can alter scope.
Build the cache key from normalized, trusted inputs:
operation_version
tenant_id
environment
role_or_policy_hash
normalized_parameters
schema_version
freshness_classTenant, role, and environment must come from authenticated context. Never let the conversation provide them as free-form cache dimensions.
This is easier when the connector exposes approved tools or views rather than a generic query endpoint. Related: approved views for AI database agents.
Prevent cross-tenant cache leakage
The most serious cache bug is not stale data. It is a valid result returned to the wrong principal.
A cache key must include every dimension that can change the visible result: organization, user or role, region, environment, approved filters, policy version, and sometimes locale or reporting timezone.
Test missing dimensions deliberately. Run the same operation for two tenants with identical parameters and confirm that neither the result nor existence metadata crosses the boundary. Repeat after role changes and permission revocation.
Recheck authorization on every read
A cached payload does not carry permanent authority.
Before returning it, the server should revalidate the current identity and policy context. A user removed from a role must not retain access until the cache entry expires.
There are two useful patterns:
- include a policy or entitlement version in the cache key so changes create a miss;
- perform a lightweight live authorization check before serving the cached result.
High-risk operations may require both. Revocation must win over performance.
Choose invalidation signals explicitly
Time-to-live alone is simple, but it leaves a known stale window. Combine it with signals that match the business workflow:
- change-data-capture or event-driven invalidation;
- table or entity version counters;
- schema and semantic-layer version changes;
- role, policy, and entitlement changes;
- manual invalidation for incident response;
- a short hard expiry as the final bound.
Invalidate by affected scope. A customer update should not flush every tenant, but it must invalidate every cached operation that derived from that customer state.
Return cache evidence to the model
The connector should return more than rows or aggregates. Include:
generated_atandsource_fresh_through;- cache status and age;
- freshness class and maximum age;
- operation, schema, and policy versions;
- scope summary and trace ID;
- whether the result is partial or truncated.
The final answer can then say “data current through 01:05 UTC” instead of presenting cached data as live.
For evidence design, see query provenance for AI database agents.
Do not cache sensitive detail by default
Aggregate counts and approved reference data are safer cache candidates than customer records, private notes, credentials, or unrestricted free text.
If a workflow requires record detail, minimize fields before the result enters model context and before it enters a cache. Encryption at rest does not fix an overbroad cache key or a stale authorization decision.
See ChatGPT database query data minimization.
Treat stale-while-revalidate as an explicit promise
Serving a slightly old result while a refresh runs can improve latency, but only for operations that tolerate it. Return the stale state visibly and define a hard maximum age.
For transactional or access-control questions, fail closed or query live. “Fast but possibly stale” is not a neutral trade-off when the answer triggers a payment, entitlement, or incident decision.
A concrete result envelope
{
"data": {...},
"generated_at": "2026-07-31T01:05:00Z",
"source_fresh_through": "2026-07-31T01:04:42Z",
"cache": {
"status": "hit",
"age_seconds": 18,
"max_age_seconds": 60
},
"operation_version": "support_queue_summary.v3",
"policy_version": "tenant-support.v12",
"trace_id": "..."
}The envelope makes latency, freshness, policy, and evidence part of the same contract.
Test the failure modes
- Use identical parameters for two tenants.
- Revoke a role after a cache entry is created.
- Change a business definition without changing the schema.
- Update a source row during stale-while-revalidate.
- Simulate an invalidation event that arrives late or twice.
- Deploy a new operation version beside old cache entries.
- Ask the model to hide or ignore the freshness metadata.
- Verify that logs identify the cache decision without copying sensitive payloads.
Where Conexor fits
Conexor provides MCP infrastructure for connecting AI clients to databases and APIs through governed tools. A clear cache contract helps those tools stay fast without separating an answer from the identity, freshness, policy, and evidence that make it safe to use.