kernelsJun 17, 2026 · 18 min
One cuBLAS flag makes GPU retrieval scoring 1.59× faster — and nothing for sparse search
The full path to 1.59× on a consumer RTX 3080: every retrieval and recsys pattern collapses to one dense matmul that cuBLAS runs at half rate, one flag flips it, and recall@10 holds at 0.987 against the fp32 ranking. Two numbers reported, 1.59× on the matmul and 1.25× end to end, plus the sparse follow-on that was measured and refuted on purpose.
retrievalcuBLASsearch
1.59×faster retrieval scoringfp32-acc ▸ fp16-acc
