Company Description

Smallest AI is one of the world’s fastest-growing speech AI research companies. We have some of the world’s best speech AI models and we are powering over 30,000 customers across 12 different countries to automate their calling operations. At heart, we are engineers, researchers, and a bunch of nerds who love diving deep into AI.

Role

We’re hiring a GPU Optimization Engineer who understands GPUs at a deep, architectural level — someone who knows exactly how to squeeze every last millisecond out of a model, what GPU constraints matter, and how to restructure models for real-world inference performance. You’ll work across CUDA kernels, model graph optimizations, hardware-specific tuning, and porting models across GPU architectures. Your work directly impacts the latency, throughput, and reliability of smallest’s real-time speech models.

What You’ll Do

Optimize model architectures (ASR, TTS, SLMs) for maximum performance on specific GPU hardware
Profile models end-to-end to identify GPU bottlenecks — memory bandwidth, kernel launch overhead, fusion opportunities, quantization constraints
Design and implement custom kernels (CUDA/Triton/Tinygrad) for performance-critical model sections
Perform operator fusion, graph optimization, and kernel-level scheduling improvements
Tune models to fit GPU memory limits while maintaining quality
Benchmark and calibrate inference across NVIDIA, AMD, and potentially emerging accelerators
Port models across GPU chipsets (NVIDIA → AMD / edge GPUs / new compute backends)
Work with TensorRT, ONNX Runtime, and custom runtimes for deployment
Partner with the research and infra teams to ensure the entire stack is optimized for real-time workloads

Requirements

Strong understanding of GPU architecture — SMs, warps, memory hierarchy, occupancy tuning
Hands-on experience with CUDA, kernel writing, and kernel-level debugging
Experience with kernel fusion and model graph optimizations
Familiarity with TensorRT, ONNX, Triton, tinygrad, or similar inference engines
Strong proficiency in PyTorch and Python
Deep understanding of model architectures (transformers, convs, RNNs, attention, diffusion blocks)
Experience profiling GPU workloads using Nsight, nvprof, or similar tools
Strong problem-solving abilities with a performance-first mindset

Great to Have

Experience with quantization (INT8, FP8, hybrid formats)
Experience with audio/speech models (ASR, TTS, SSL, vocoders)
Contributions to open-source GPU stacks or inference runtimes
Published work related to systems-level model optimization

Who Will Succeed in This Role

Someone who:

thinks in kernels, not just layers
knows which optimizations are theoretical vs practically impactful
understands GPU boundaries (memory, bandwidth, latency) and how to work around them
is excited by the challenge of ultra-low latency and large-scale real-time inference
loves debugging at the CUDA + model level

GPU Optimisation Engineer