Why switching embedding models breaks your vector database, and why you can't just resize the column
ERROR: different vector dimensions 1536 and 768
Also appears as
- ValueError: shapes (768,) and (1536,) not aligned
- vector must have at least 1 dimension error after model change
- IndexError when comparing embeddings of different sizes
Short answer
Different embedding models produce vectors of different fixed dimensions, so swapping models without re-embedding the entire corpus produces a hard dimension mismatch error or, worse, silently meaningless similarity scores if the column is resized without re-indexing. There is no shortcut: changing the embedding model requires re-embedding every document and rebuilding the vector index from scratch.
Affects: Any vector database with a fixed-dimension column, triggered specifically by upgrading, downgrading, or swapping the embedding model.
Fastest path through an embedding model migration
- 1Confirm the new model's output dimension with a quick test embedding call and compare it against your vector column's declared dimension.
- 2Do not attempt to pad, truncate, or reinterpret vectors between different dimensions or models; treat this as a full migration, not a schema patch.
- 3Create a new vector column or table sized for the new model's dimension.
- 4Re-embed every document in the corpus with the new model and write into the new column or table.
- 5Rebuild the HNSW or IVFFlat index on the new column, then cut over queries only after validating retrieval quality on the new embeddings.
- 6Drop the old column or table only after the new one is validated in production.
How to confirm this is your problem
- A database error explicitly reporting mismatched vector dimensions
- Retrieval quality collapses immediately after upgrading or swapping the embedding model with no other code changes
- Some rows in the vector table were embedded with the old model and others with the new one, producing inconsistent similarity scores
- Application code throws a shape mismatch error when comparing query and document vectors
- Everything runs without error but similarity scores are uniformly low and rankings look random
Root causes and fixes
The vector column was declared for the old model's dimension and the new model outputs a different size
pgvector and most vector databases enforce a fixed dimension per column or index at creation time. Every embedding model has a fixed, non-negotiable output size determined by its architecture, and there is no valid way to compare or store vectors of different dimensions in the same column.
Fix: Create a new column or table sized for the new model's exact output dimension, and treat model swaps as a full re-embedding migration rather than an in-place update.
ALTER TABLE items ADD COLUMN embedding_new vector(1024); SELECT vector_dims(embedding) FROM items LIMIT 1;
Some rows were re-embedded with the new model while others still hold vectors from the old model
If a migration is done incrementally in place without a full cutover, the table can end up with a mix of old-dimension and new-dimension vectors, or same-dimension vectors from two different models with incompatible geometry, silently corrupting similarity search for the entire table.
Fix: Never mix embeddings from two different models or versions in the same column. Migrate to a new column or table, backfill it completely, validate it, and only then switch queries over and drop the old data.
Application code assumes a fixed dimension constant that was not updated after the model change
Many RAG codebases hardcode the embedding dimension in config or as a magic number used for validation, padding, or truncation logic. If that constant is not updated alongside the model swap, code paths that check or reshape vectors will throw shape errors even after the database schema is correctly updated.
Fix: Search the codebase for the old dimension value as a literal and replace it with a single config-driven constant read from the embedding model's own reported output size, not hardcoded twice.
grep -rn '768' --include='*.py' .
Truncated or Matryoshka-style embeddings were sliced to the wrong length
Some newer embedding models support Matryoshka representation learning, where a shorter prefix of the full vector is still a valid, lower-fidelity embedding. If code slices to an arbitrary length that does not match either the model's native full size or one of its officially supported truncation points, similarity scores become unreliable in ways that don't always throw an outright dimension error.
Fix: Only truncate embeddings to lengths the model's documentation explicitly supports, and re-index consistently at that exact length across the entire corpus and all queries.
Diagnostic commands
Check the actual dimension of stored vectors
SELECT vector_dims(embedding) FROM items LIMIT 5;
If this returns more than one distinct value across rows, the table has a mixed-dimension corruption problem that must be fully re-embedded and cannot be patched with a schema change alone.
Check the declared column dimension
SELECT atttypmod FROM pg_attribute WHERE attrelid = 'items'::regclass AND attname = 'embedding';
Compare this against the output of a fresh test call to your embedding model; if they differ, either the schema needs to be migrated to the new dimension or the wrong model is being called.
Test-embed a sample string with the current model and print its length
python -c "from your_embedder import embed; print(len(embed('test')))"This confirms exactly what dimension the currently configured model produces, which should be the single source of truth for both the schema and any hardcoded config values.
Stopping it from happening again
- Store the embedding model name and dimension as metadata alongside the vector table, and validate it programmatically before any write.
- Never hardcode the embedding dimension as a magic number in more than one place; derive it from the model at startup.
- Treat any embedding model change as a full corpus migration project with a validation gate, not a config toggle.
- Keep the old embedding table available, read-only, until the new one is validated in production, so rollback is possible.
- Run a retrieval quality regression test immediately after any embedding model change before removing the old data.
When this becomes an architecture problem
If you are re-embedding a very large corpus, or need to support multiple embedding models simultaneously for different tenants or use cases, this becomes an infrastructure and pipeline design question (parallel embedding throughput, storage cost, zero-downtime cutover) rather than a one-off migration script, and is worth planning explicitly before starting the re-embed.
Frequently asked questions
Can I convert a 768-dimension embedding to 1536 dimensions with padding or interpolation?
No. Padding, truncating, or interpolating vectors between models does not produce a meaningful embedding; the dimensions of two different models encode entirely different learned representations, so mathematical operations to force compatibility only produce vectors that look valid but carry no real semantic relationship to either model's actual embedding space.
How long does re-embedding a large corpus take?
It depends primarily on embedding model throughput and corpus size; a mid-sized embedding model on a single GPU can typically process several hundred to a few thousand chunks per second in batches, so a corpus of a few million chunks is usually a multi-hour to one-day job if batched properly.
Do I need to rebuild the vector index after re-embedding, or just the data?
Both. The HNSW or IVFFlat index is built directly from the vector data at creation time, so simply updating the underlying vectors in place without rebuilding the index leaves the index referencing stale or dimensionally invalid data. Always rebuild the index as the final migration step.
Is it safe to run two embedding models side by side during migration?
Yes, as long as they write to separate columns or tables and queries are explicitly routed to the matching one. The danger is only in mixing outputs from two models within a single column or index, which silently corrupts similarity search.
Size it properly next time
Free calculators that prevent this class of failure before you provision hardware.
Vector Database Sizing Calculator
Convert vector count, embedding dimensions, and precision into a real storage footprint, including index overhead and replica factor, before you pick a vector database.
Free ToolEmbedding Model Serving Cost Calculator
Estimate the GPU hours and dollar cost to embed your daily document volume, factoring in embedding model size, batching efficiency, and re-indexing overhead.
Free ToolDocument Ingestion Pipeline Estimator
Estimate total pipeline time from document count, OCR share, and embedding throughput, so ingestion timelines stop being a guess in the project plan.
Related problems
pgvector similarity queries are slow
pgvector queries are almost always slow because of an index and operator class mismatch (an index built for one distance function while queries use a different operator), a missing index entirely so Postgres falls back to a sequential scan, or search-time parameters (ef_search, probes) set too low. Confirm EXPLAIN ANALYZE shows an index scan, match the operator class to your distance function, and tune maintenance_work_mem before building large HNSW indexes.
Vector index memory usage is too high (HNSW blowing up RAM)
Vector index memory usage exceeds raw embedding size because HNSW stores a graph of neighbor connections on top of every vector, typically adding 1.5 to 3 times the raw vector size in overhead depending on the M parameter, while IVFFlat has lower overhead but worse recall at the same speed. Estimate memory as dimension times row count times 4 bytes for raw vectors, then add graph overhead, and consider quantization if the total does not fit in available RAM.
RAG retrieves irrelevant or wrong documents
RAG retrieves the wrong documents most often because the embedding model used to index the corpus differs from the one used at query time, or because chunks are large enough that a single embedding averages away the passage that actually answers the question. Fix embedding consistency and chunk granularity first, then add a reranker and metadata filters before touching the LLM prompt.
Guidepgvector vs Dedicated Vector Databases: An Honest Comparison
pgvector vs Milvus, Qdrant, and Weaviate for enterprise RAG: real tradeoffs on scale, latency, operational overhead, and when Postgres is genuinely enough.
GuideEnterprise RAG Architecture: The Full 2026 Blueprint
A practitioner's blueprint for enterprise RAG in 2026: ingestion, chunking, embedding, retrieval, rerank, generation, and the eval loop that keeps it honest.
GuideRAG Chunking Strategies: Fixed, Semantic, Structural, and Late
Compare RAG chunking strategies, fixed-size, semantic, structural, and late chunking, with concrete guidance on chunk size, overlap, and when each wins.
Still stuck, or tired of fighting your own infrastructure?
Netray deploys and operates on-prem AI for regulated manufacturers and defense suppliers. We have debugged this stack in production, on air-gapped networks, at scale.