When a team decides its agents need a knowledge base, the first question is usually which SaaS product to pay for. None. Running in the same Docker you already have: a markdown folder for the notes, pgvector (free Postgres extension) for semantic search, and a local embedding model that downloads once and never makes another external call. Zero hosted API, zero key, zero per-request cost. The editor, if you want one, is Obsidian, which is also local and free to use, at home and at work. VS Code opens the same files.
This is the setup I run against my own vault, and the one I would put in front of a team: retrieval-augmented access to conventions, runbooks, and architecture decisions, with no SaaS, no per-seat cost, and no vendor lock-in. The agent reads from a vault that humans edit. Humans edit it because the editing experience is pleasant. That last part is the entire trick.
What RAG Actually Is
Retrieval-Augmented Generation. You do not put your knowledge base inside the LLM. You put it next to the LLM and, at inference time, retrieve the relevant snippets and inject them into the prompt. That is how agents know about things that were not in the training data: team conventions, runbooks, architecture decisions from last quarter. Pure in-context, no fine-tuning, no vendor.
The pieces you need: somewhere to store the text (markdown files), somewhere to store the embeddings (pgvector), and a pipeline that keeps the two in sync. That is it. The complexity comes from operating this in production, not from building it.
What an Embedding Is and Why You Need One
Keyword search (SQL LIKE '%...%') does not work for RAG. If a note talks about 'production deploy' and the agent asks 'how to push code to the server', you want the note to surface even though the words do not match. That is search by meaning, not by string. Embeddings solve this: each text chunk becomes a vector of numbers (say, 384 or 1024 floats) that represents the meaning of the text. Texts with similar meaning produce similar vectors. Search becomes geometry. Give me the vectors closest to this one. That is what pgvector does.
Someone has to generate those vectors. That someone is the embedding model, and this is the decision where most tutorials bury you in a paid API with no justification. There is a 100% local option that runs in your own Docker stack next to Postgres, no external API, no key, no cost. Use that as the default. Only consider a hosted API if you have a concrete reason (re-embedding volume, measured recall problems, notes the local model cannot read), and the post covers all three at the end.
Why Obsidian Specifically
I use Obsidian by default because the vault is just a folder of markdown files. Put Obsidian on top of that folder and you get a great editor with backlinks, tags, graph view, and search. Remove Obsidian and you still have a folder of markdown files. No lock-in. Git versions the vault. The agent reads the files directly.
Obsidian is a desktop app (Windows, Mac, Linux, mobile) that runs 100% on your machine. The vault never leaves your disk. Their paid services (Sync and Publish) are optional add-ons you ignore completely: cross-device sync in this setup is git. Using it at work is free too. Since February 2025, Obsidian is free for commercial use, and the Commercial license ($50 per user per year) is optional support rather than a requirement. That changed a month after this post first went up, when companies of two or more still had to buy the licence.
If the team prefers not to use Obsidian, the setup works identically. The agent reads .md files. It does not care whether you edited them in VS Code, Obsidian, or vim.
The Architecture
The Indexer
The indexer is a small Python service. It watches the vault directory, chunks notes when they change, generates an embedding for each chunk with a model that runs inside the container itself, and upserts into pgvector. No external calls. Runs in Docker alongside the rest of the stack.
# libs/memory/indexer.py
import os
from pathlib import Path
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
from sentence_transformers import SentenceTransformer
import psycopg
VAULT_PATH = Path(os.environ["VAULT_PATH"])
DB_URL = os.environ["DATABASE_URL"]
# Local model. Downloads once (~90MB) to the container disk.
# Zero API, zero key, zero per-call cost.
# Resulting vector: 384 dimensions.
model = SentenceTransformer("all-MiniLM-L6-v2")
def chunk(text: str, max_chars: int = 1500) -> list[str]:
paragraphs = text.split("\n\n")
chunks, buf = [], ""
for p in paragraphs:
if len(buf) + len(p) > max_chars:
chunks.append(buf)
buf = p
else:
buf = f"{buf}\n\n{p}" if buf else p
if buf:
chunks.append(buf)
return chunks
def embed(text: str) -> list[float]:
return model.encode(text).tolist()
def reindex(file_path: Path):
text = file_path.read_text()
source = str(file_path.relative_to(VAULT_PATH))
chunks = chunk(text)
with psycopg.connect(DB_URL) as conn:
for i, c in enumerate(chunks):
emb = embed(c)
# The %s::vector cast is not optional. psycopg3 sends a Python
# list as float8[] and Postgres refuses to store that in a
# VECTOR column. The alternative is register_vector(conn) from
# pgvector.psycopg, which teaches psycopg the type once.
conn.execute(
"INSERT INTO knowledge (source, chunk_index, content, embedding) "
"VALUES (%s, %s, %s, %s::vector) "
"ON CONFLICT (source, chunk_index) DO UPDATE SET "
"content = EXCLUDED.content, "
"embedding = EXCLUDED.embedding, "
"updated_at = NOW()",
(source, i, c, emb)
)
# A note that got shorter leaves orphan chunks behind.
conn.execute(
"DELETE FROM knowledge WHERE source = %s AND chunk_index >= %s",
(source, len(chunks))
)
conn.commit()
class Handler(FileSystemEventHandler):
def on_modified(self, event):
if event.src_path.endswith(".md"):
reindex(Path(event.src_path))
if __name__ == "__main__":
# Full initial indexing
for md in VAULT_PATH.rglob("*.md"):
reindex(md)
# Watch for changes
observer = Observer()
observer.schedule(Handler(), str(VAULT_PATH), recursive=True)
observer.start()
observer.join()That is the whole indexer. It is not minimal pseudo-code. It is real Python that runs. It handles initial indexing, file change detection, and embedding regeneration on edit. The 1500-character chunk size is a middle ground: small enough to retrieve precise snippets, large enough to preserve paragraph context. One caveat: all-MiniLM-L6-v2 truncates input longer than 256 word pieces, roughly 1,000 characters of English, so the tail of a full 1500-character chunk never reaches the embedding. If your notes are dense, lower max_chars or pick a model with a longer input window.
The Schema
CREATE EXTENSION IF NOT EXISTS vector;
-- VECTOR(384) matches all-MiniLM-L6-v2.
-- If you swap the embedding model, adjust the dimension and reindex.
CREATE TABLE knowledge (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
source TEXT NOT NULL,
chunk_index INTEGER NOT NULL,
content TEXT NOT NULL,
embedding VECTOR(384) NOT NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (source, chunk_index)
);
CREATE INDEX idx_knowledge_embedding
ON knowledge USING hnsw (embedding vector_cosine_ops);
CREATE INDEX idx_knowledge_source ON knowledge (source);One table. Chunk text, embedding, source path. The source path is what matters when the agent shows the retrieved context to a human: 'this came from conventions/testing.md', not 'this came from embedding ID 8d3f'.
How the Agent Uses This
# libs/memory/retrieval.py
# Reuses the same model loaded in the indexer (one instance, CPU).
def retrieve(query: str, top_k: int = 5) -> list[dict]:
q_emb = model.encode(query).tolist()
with psycopg.connect(DB_URL) as conn:
rows = conn.execute(
# Inner query stays on the HNSW index. Outer query re-ranks
# the candidates and demotes anything older than 6 months.
"WITH candidates AS ("
" SELECT source, chunk_index, content, updated_at, "
" 1 - (embedding <=> %s::vector) AS similarity "
" FROM knowledge "
" ORDER BY embedding <=> %s::vector "
" LIMIT 20"
") "
"SELECT source, chunk_index, content, "
" similarity - CASE "
" WHEN updated_at < NOW() - INTERVAL '6 months' THEN 0.05 "
" ELSE 0 END AS score "
"FROM candidates "
"ORDER BY score DESC "
"LIMIT %s",
(q_emb, q_emb, top_k)
).fetchall()
return [
{"source": r[0], "chunk": r[1], "content": r[2], "score": r[3]}
for r in rows
]The agent builds a query from the trigger payload plus its current reasoning, calls retrieve, and gets back the top 5 chunks. Those chunks go into the system prompt with their source paths. If the agent cites the knowledge base in the output, it cites the source path, not an opaque ID. A human can open exactly that file and verify. The two-step query is deliberate: pull a wide candidate set with the vector index, then re-rank in SQL. Re-ranking in the outer query keeps the freshness penalty from throwing away the index.
What to Put in the Vault
Not everything. Be deliberate about what belongs in a team knowledge base that agents read.
The rule I use: if the note will still be useful 12 months from now, it belongs in the vault. If it is a snapshot of a moment, it goes to Slack or a ticket.
Cost
By default, this setup costs zero. The embedding model downloads once to the container disk (around 90MB) and runs on the CPU of the server you already have. pgvector is a free Postgres extension. The only paid infra is the server itself where everything runs, and if you already run Postgres for anything else, this is zero additional infrastructure.
For a vault of 500 notes averaging 2 chunks each, initial indexing with all-MiniLM-L6-v2 on a modern CPU finishes in well under a minute. Continuous re-embedding on edits is instant from the editor's point of view. If you scale to tens of thousands of notes and start feeling indexing latency, then it makes sense to revisit the decision. Until then, resist the urge to 'improve' by switching to a paid API before you have a number that justifies it.
Mistakes I Have Already Made
First mistake: I built a complicated chunking strategy with overlap and semantic splitting in the first version. It performed worse than paragraph-based chunking because the boundaries were less predictable. Start simple. Paragraph chunks with a max size. Only tune when retrieval quality tells you to tune.
Second mistake: I treated the knowledge base as append-only and let stale content pile up. Left alone, that ends with the agent citing a deprecated convention from a runbook nobody has touched in a year. The retrieval query above is the fix: it pulls a wider candidate set, then subtracts a small penalty from anything whose updated_at is more than 6 months old. A stale note can still win if nothing fresher is close, which is the behavior you want.
Third mistake: I gave the agent write access to the vault so it could 'learn' by updating notes. That is the wrong instinct. A low-confidence run can rewrite a note into something subtly wrong, nothing in the pipeline catches it, and the next retrieval treats the agent's own text as source material. The vault is edited by humans. Agent output goes to a separate agent_learnings/ folder that a human reads before anything gets promoted into the vault.
The knowledge base your agents read from and the knowledge base your team edits should be the same folder. Any architecture that separates the two makes the two diverge, and divergent memory is worse than no memory at all.
Optional Upgrades
Three situations where the default local model stops being the right answer. Only these three.
Massive volume. If the vault grows past tens of thousands of notes and local re-embedding starts taking hours, a hosted API finishes in minutes. Voyage AI voyage-3 is the most direct upgrade here because it was trained specifically for retrieval, and it supports input_type='query' vs 'document' to improve recall.
Recall quality validated by benchmark. If you measured local retrieval and it is pulling irrelevant chunks in specific scenarios, larger hosted models sometimes fix it. But measure before swapping. The problem is often chunking or ranking, not the model.
Input the local model cannot represent. all-MiniLM-L6-v2 truncates at 256 word pieces and was trained on English. If your notes are long-form or multilingual, retrieval quality drops for a reason no amount of tuning fixes. Try the bigger local option first: nomic-embed-text through Ollama takes a far longer input, handles more languages, and is still free. A hosted model with a long context window is the step after that, not before it.
When the numbers add up, the swap is a few lines:
# --- Local default (zero cost) ---
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2") # 384 dim
def embed(text: str) -> list[float]:
return model.encode(text).tolist()
# --- Upgrade to Voyage (pay per call) ---
import voyageai
vo = voyageai.Client() # reads VOYAGE_API_KEY from env
def embed_doc(text: str) -> list[float]:
result = vo.embed([text], model="voyage-3", input_type="document")
return result.embeddings[0]
def embed_query(text: str) -> list[float]:
result = vo.embed([text], model="voyage-3", input_type="query")
return result.embeddings[0]
# WARNING: voyage-3 returns 1024 dim. Adjust VECTOR(1024) in the schema
# and reindex the entire vault. The migration is not transparent.References
- sentence-transformers (local embedding, default for this post): https://www.sbert.net
- Ollama (local embedding models, e.g. nomic-embed-text): https://ollama.com
- Obsidian: https://obsidian.md
- Obsidian, free for work (February 2025): https://obsidian.md/blog/free-for-work/
- pgvector: https://github.com/pgvector/pgvector
- Voyage AI (optional hosted upgrade): https://www.voyageai.com