768 dimensions to catch 'phone call avoidance' and 'can't initiate phone calls' as the same friction
The graph extraction engine kept creating duplicate nodes. "Can't initiate phone calls" and "Phone call avoidance" are the same friction, but string matching can't see that. The graph fills up with near-identical nodes and the whole dependency structure becomes noise.
768-dimensional embedding vectors via gemini-embedding-001. Every new node gets embedded. Every existing node has a cached vector. Cosine similarity determines if you've said this before. Three zones: ≥0.85 is automatic merge, 0.80–0.85 triggers an LLM arbitration call to decide if they're genuinely the same or subtly different, below 0.80 it's a new node. The EmbeddingCache class is 71 lines — pure math on the similarity side, no API calls for the comparison itself.
Specifically chose gemini-embedding-001 over the legacy text-embedding-004 — "Lever your Gemini documentation skills to verify you're doing this well with the latest and greatest." The newer model supports MRL dimensionality scaling, so 768 dims matches the original spec while getting better quality vectors. Quality and accuracy over latency — a fair trade when someone's mapping their invisible barriers and duplicate nodes would blur the picture.