Writing
How we make AI efficient.
Each post is one result, whether a model built from scratch or a kernel made faster, measured against a real production baseline.
Accelerating Wan 2.2 video generation by 16.8× via 4-step diffusion distillation
Reducing Wan 2.2 A14B video generation latency from 809 seconds to 48 seconds per 5-second clip on an NVIDIA A100 GPU. Evaluates 4-step Phased-DMD distillation LoRA adapters across 13 test scenes without quality loss or custom CUDA kernels.
2.34× faster LLM decoding: FP8 weights and native speculative decoding on Gemma-4
Optimizing single-stream inference for Gemma-4-26B on an A100 GPU using FP8 quantization and native Multi-Token-Prediction speculative decoding.
Optimizing block-sparse INT8 attention and multi-expert step caching for Wan 2.2
Analyzing hardware execution limits and block selector heuristics in Wan 2.2 video generation. Demonstrates 1.30–1.76× speedups in block-sparse INT8 attention using a mean-pool selector and 1.78× speedups via per-expert diffusion step caching.
Fixing ollama's MoE memory estimate — and a 3× kernel speedup that wasn't
Auditing Ollama and llama.cpp GPU memory estimation and quantized kernel execution. Shipped PR #17201 fixes a -20.6% VRAM under-estimation in MoE models. Includes analysis of MMQ kernel tile configurations and DeepSeek MLA execution paths.
Custom on-device voice commands from zero recordings
Designing a zero-shot synthetic data pipeline and INT8 quantized speech encoder to train custom on-device voice command classifiers for Qualcomm edge NPUs in under 60 seconds.
6.1× smaller, 5.77× faster, same FID: compressing a coupled-oscillator image model
Compressing Un-0's oscillator coupling matrix via Monarch factorization and Tensor Core dispatch to reduce model size from 19.4M to 3.2M parameters while cutting A100 batch latency from 102 ms to 17.7 ms at matched clean-FID.
Accelerating dense physics solves via FP16 Tensor Core accumulation
Dense matrix inversions, Gaussian process regression, and PDE Green's function applications reduce to large matrix multiplications that execute at half-rate on consumer Ampere GPUs. Enabling FP16 accumulation recovers 1.84–1.91× speedups at ~1e-3 relative error against FP64 reference solves.
Accelerating GPU vector retrieval scoring via FP16 accumulation
Evaluating PyTorch FP16 Tensor Core accumulation flags across dense retrieval scoring workloads. Achieves 1.59× speedups in matrix scoring operations and 1.25× end-to-end pipeline latency improvements while preserving Recall@10 = 0.987 ranking accuracy.