Which embedding model should I use?

Whichever wins on your benchmark of your data. Default to a 1024-dim model from OpenAI/Voyage/Cohere if you have no benchmark yet.

How often do we re-embed the corpus?

Whenever you change embedding models. Otherwise: every time a source document changes. Treat embeddings as derived data, like a search index.

Should we host embeddings ourselves or use an API?

API until your monthly embedding bill exceeds your engineer's time-cost of hosting. Roughly: API at $2k/mo.

How do we handle PII in the embedded text?

Embeddings are not reversible to plaintext directly, but a determined attacker with the model can recover information. Treat them like the original text. Encrypt at rest. Apply the same access controls.

What is the maximum input size?

Each model has a max input. OpenAI's text-embedding-3 is 8,191 tokens. Voyage's is 32k. If your chunks exceed it, you have to re-chunk.

Embeddings Deep Dive

$ ai-engineer-path

AI Engineering Support

AI Engineering · Beginner guide · 13 min read

← Back to AI Engineering

If this helped, buy me a coffee

Everything is free. Tips keep me writing the rest.

Buy me a coffee →

$ ai-engineer-path

Practical, hands-on guides for backend engineers learning to build with AI.

Topics

AI Engineering

Site

Support FAQ About Contact Glossary

Legal

Privacy Terms RSS feed