Section 7 · Learn · Security
Partitioning by tenant
Splitting a table so each tenant's rows live in their own partition with their own index, removing the post-filtering problem rather than mitigating it.
Splitting a table so each tenant's rows live in their own partition with their own index, removing the post-filtering problem rather than mitigating it.
What it is
A LIST partition on tenant keeps each tenant's rows physically separate, and each partition carries its own vector index.
Why it is needed
If an index contains only rows this role may see, there is nothing to post-filter away. The recall cliff disappears because the cause disappears.
How it works
The planner prunes partitions the query cannot match, so only permitted partitions are scanned. Passing the permitted tenant list as an explicit predicate enables that pruning at plan time.
Compared
Monolithic index
- one index over every tenant
- index chooses first
- policy filters after
- permitted rows may never be reached
Partitioned by tenant
- one index per tenant
- only permitted partitions scanned
- nothing to post-filter away
- pruning happens at plan time
In this project
Recommended for Phase 2 after the post-filtering measurements. Two things do different jobs here and it is worth being precise: row-level security is the correctness boundary, and the explicit tenant predicate is purely a performance hint. In the earlier synthetic measurement recall was identical with and without the predicate — the predicate never widened access — but plan-time pruning cut latency from 38.5 ms to 1.03 ms. The security test suite must therefore drop the predicate and still pass; that is what proves it is an optimisation and not the boundary.
Trade-offs
Advantages
- Removes post-filtering recall loss
- Smaller indexes build faster
- Plan-time pruning cuts latency sharply
Limitations
- Many partitions add planning overhead
- Small tenants make per-partition indexes meaningless
- Cross-tenant queries must touch many partitions
Common mistakes
- Treating the explicit predicate as the security boundary
- Partitioning so finely that a partition holds fewer rows than ef_search
When to use it
Reach for it when
- Tenant isolation matters and tenants are reasonably sized
Avoid it when
- Hundreds of tiny tenants; merge them into semantic groups first
How it interacts with the rest of the system
PostgreSQL Row-Level SecurityANN post-filtering under access controlMulti-tenancy and tenant assignmentHNSW and approximate nearest neighbour search