pgvector
cargo add octopux --features sqlx,pgvector # sqlx with `postgres`; add graphql for the GraphQL scalars
The vector, halfvec and sparsevec columns of pgvector become the Vector, HalfVector and SparseVector fields of the sqlx models.
The vectors stay on the backend: clients send text and receive rows. The backend embeds the text before a save or a search.
Vector fields
The model sent to the clients leaves the vector out:
#[derive(Serialize, Deserialize, sqlx::FromRow, HttpFindListDelete, SqlxModel)]
#[http_find_list_delete(Id, FindQuery, ListQuery, DeleteQuery, AppState)]
#[sqlx_model(database = "postgres", filter)]
#[octopux_info(path = "article")]
pub struct Article {
pub id: Id,
pub title: String,
pub body: String,
// no `embedding`: the vector(384) column is never sent
}
The insert and update models hold it, skipped from the payload, and before_save computes it:
use octopux::pgvector::Vector;
#[derive(Serialize, Deserialize, InputObject, HttpCreate, SqlxNewModel)]
#[http_create(SaveQuery, AppState)]
#[sqlx_model(database = "postgres", model = "Article", before_save)]
pub struct NewArticle {
pub title: String,
pub body: String,
#[serde(skip)]
#[graphql(skip)]
pub embedding: Option<Vector>,
}
#[async_trait]
impl BeforeSave<AppState> for NewArticle {
async fn before_save(mut self: Self, state: &AppState) -> Result<Self> {
self.embedding = state.embedder.embed(vec![format!("{}\n{}", self.title, self.body)]).await?.pop();
Ok(self)
}
}
Similarity search
A SqlxFilter field with op = "nearest" orders the list by distance, nearest first; limit keeps the k nearest rows. Skip it from the query string so that only the backend sets it:
#[derive(Default, Deserialize, JsonSchema, ApiComponent, SqlxFilter)]
#[sqlx_filter(database = "postgres")]
pub struct ListQuery {
pub limit: Option<usize>,
pub category: Option<String>,
#[sqlx_filter(column = "embedding", op = "nearest")]
#[serde(skip)]
#[schemars(skip)]
pub near: Option<pgvector::Vector>,
}
The search embeds the text and lists the rows:
pub async fn search_articles(query: SearchQuery, state: &AppState) -> anyhow::Result<Vec<Article>> {
let near = Some(state.embedder.embed_query(&query.q).await?);
let list = ListQuery { limit: query.limit, category: query.category, near };
Ok(Article::list(&list, state).await?)
}
Mount /article/search before the routes of the model, so that search is not taken for an {id}. The other filters still apply.
distance |
Order | HNSW operator class |
|---|---|---|
cosine (default) |
<=> |
vector_cosine_ops |
l2 |
<-> |
vector_l2_ops |
inner_product |
<#>, largest first |
vector_ip_ops |
l1 |
<+> |
vector_l1_ops |
Index the column with the operator class of the distance (halfvec_*_ops, sparsevec_*_ops for the other types):
CREATE INDEX article_embedding_idx ON article USING hnsw (embedding vector_cosine_ops);
Similar rows
The rows similar to a row reuse its stored vector, nothing is embedded: a HasMany relation reads it, then orders the other rows by distance.
SELECT id, title, 1 - (embedding <=> $2) AS similarity FROM article
WHERE id <> $1 ORDER BY embedding <=> $2 LIMIT $3
GraphQL and OpenAPI
With graphql and openapi, the vector types are scalars and documented schemas. Keep them out of the API anyway: resolvers take text, inputs skip the vector with #[graphql(skip)].
{ searchArticles(q: "ramen in tokyo", limit: 2) { id title } }
mutation { createArticle(input: { title: "Pasta al pesto", body: "Basil, pine nuts..." }) { id } }
CLI and reverse engineering
With --postgres, the CLI offers pgvector::Vector field types: the migration creates the extension, the vector(n) column and its HNSW cosine index. octopux-reverse maps the pgvector columns to these types; hide the fields as above.
Semantic search generated by the CLI
With --sqlx --postgres, generate-model and add-field offer to compute a pgvector::Vector field from the text of the model with fastembed, locally on the CPU:
? `embedding` is a vector: compute it from the text of the model with fastembed, and search the model by meaning? (y/N) y
? Embedding model › [MultilingualE5Small]
? Fields `embedding` is computed from › [title, summary]
| Model | Dimensions | |
|---|---|---|
MultilingualE5Small |
384 | multilingual |
MultilingualE5Base |
768 | multilingual, more precise, slower |
BGESmallENV15 |
384 | English |
BGEBaseENV15 |
768 | English, more precise, slower |
AllMiniLML6V2 |
384 | English, fastest |
The CLI writes:
src/embedder.rs: the fastembed model, shared by all the searches. Your code depends on fastembed, octopux does not: edit it freely.<table>_search.rs: theBeforeSaveimplementations,GET /<model>/search?q=..., thesearch<Models>GraphQL query, andembed_missingfor the rows saved before.- with
--bootstrap, the embedder inAppState.
It then offers GET /<model>/{id}/similar, see generate-relation --nearest.
Mount the search first: .configure(book_search::configure).configure(book::configure). The model is downloaded on the first run into .fastembed_cache.
The search always answers the nearest rows, even unrelated ones: filter on similarity. Its scale depends on the model (unrelated rows stay below ~0.8 with E5, ~0.55 with BGE, ~0.3 with AllMiniLML6V2); check a few searches on your data. A vector computed from more text separates better than a title alone.
See the recipes example: a semantic search over 340 recipes.