Embeddings

What similarity threshold should I use to group things by meaning?

Five support cases about the identical problem scored between 0.49 and 0.82 against each other, while unrelated pairs reached 0.66. The distributions overlap, so no single threshold separates them — and the system that got it wrong reported no error at all.

4 min

How do I tune a threshold when I only logged the matches?

A similarity score was recorded every time two support cases were grouped and discarded every time they were not, so every number available for tuning was, by construction, above the threshold. The three near-misses that moved it were invisible until one column kept them.

7 min