I ported Hierarchical Navigable Small World algorithm (HNSW) for efficiently finding nearest neighbors to TypeScript (with a little LLM help).
I fucking did it.
I already have cosine similarity in a flat index. Are you saying I should port hierarchical small navigable worlds to pure TS?
— 🎭 (@deepfates) May 25, 2023
This is for doing semantic search over your notes in Obsidian. A flat index worked fine with smaller vaults, but on a vault with 10,000 notes it was taking 30–45 seconds to run a search, which is unacceptable. Now it takes 30 to 45 milliseconds.
The speedup is more like 100x on a vault with less than 1K notes, but still feels great. I was trying all these libraries with C++ bindings or WASM compiled from Rust or whatever. Somebody suggested this datalog thing that literally looks like a shitpost. I’m doing it in the browser.