Community

PRISM Index Features and Performance in pg_vectorsearch

The PRISM index in pg_vectorsearch introduces performance-enhancing features for scalable vector search, using IVF-style hierarchical routing, RaBitQ quantization, and SIMD-optimized distance computation for improved accuracy and speed.

AI-generated editorial illustration of the article topic; not a photograph, product screenshot or measured result.

PRISM Index Features and Performance

The PRISM index in pg_vectorsearch introduces several performance-enhancing features designed for scalable vector search. It employs an IVF-style approach with hierarchical routing, enabling efficient querying over large datasets, such as those containing 100 million or more vectors. The index leverages RaBitQ quantization, which provides theoretical error bounds and supports two-stage search for improved accuracy and speed. Additionally, SIMD-optimized distance computation using AVX2, AVX512, and NEON instructions accelerates query processing. Dynamic updates are supported through posting-list splits via the prism_rebalance function, ensuring the index remains efficient even as data evolves. The PRISM index is also compatible with existing pgvector tables, allowing users to index vector and halfvec columns without significant changes. According to pg_vectorsearch, PRISM outperforms graph-based HNSW in speed on a 100-million-vector dataset, achieving equivalent recall with only a quarter of the memory.

Vector Storage and Compatibility

pg_vectorsearch introduces specialized vec32 (float32) and vec16 (float16) columns for storing vectors, offering flexibility in precision and storage efficiency. These types are binary-compatible with pgvector's vector and halfvec types, allowing both extensions to coexist within the same schema without conflicts. This compatibility enables developers to leverage existing pgvector infrastructure while using pg_vectorsearch for enhanced scalability. The example demonstrates creating a table with a vector column and building an HNSW index using vector_l2_ops, showing how pg_vectorsearch can be integrated alongside pgvector for advanced vector search capabilities.

Index Creation and Query Execution

The pg_vectorsearch extension introduces scalable vector search capabilities in PostgreSQL through the prism index type. To create an index, users execute a command like CREATE INDEX ON items USING prism (embedding vec32_l2_ops);, which establishes a vector search index on the embedding column using the vec32_l2_ops operator class. Adjusting parameters such as prism.nprobe = 40 influences query precision and performance. Queries leverage the <-> operator for distance calculation, as demonstrated in SELECT * FROM items ORDER BY embedding <-> '[3,1,2]' LIMIT 5;. For space optimization, vec16 can be used instead of vec32, with corresponding operator classes. The EXPLAIN command provides insight into query execution plans, revealing how PostgreSQL utilizes the prism index for efficient vector similarity searches.

Distance Operators and Metric Mapping

pg_vectorsearch introduces distance operators and metric mapping to enable efficient vector search in PostgreSQL. Specific operators like <- for L2 distance, <#> for inner product, and <=> for cosine similarity are mapped to corresponding operator classes such as vec32_l2_ops and vec16_cosine_ops, depending on the vector type. When replacing an HNSW index, vector__ops maps to vec32__ops or vec16__ops, ensuring compatibility and performance. According to Hacker News Show, these mappings are essential for leveraging the correct metric during queries. Hacker News Show notes that defaults are tuned from large-scale benchmarks, and most deployments only adjust prism.nprobe for fine-tuning. Hacker News Show explains that inserts append to posting lists, which grow as data is added and never split automatically, affecting maintenance and performance considerations.

Building and Requirements

To build pg_vectorsearch, developers need PostgreSQL 18+ with development headers, a GNU C11 compiler (such as GCC 11+ or Clang 14+), and the Meson build system. Optional dependencies, such as CBLAS, can be included for faster RaBitQ encoding. For a release build, the command meson setup builddir --buildtype=release is used, followed by meson install -C builddir to install the extension to PostgreSQL. Additional build options are detailed in the development guide. Users are encouraged to consult the maintenance guide for information on how the extension works and its current limitations.