Release Notes#

Unreleased#

Packaging#

  • The third-party attribution now includes CUTLASS, whose headers ship in the wheel and whose templates the extension is built from; the PyTorch, NumPy and Triton entries were removed, as no code from them is bundled.

  • The wheel no longer ships GoogleTest/GMock headers and static libraries or the subquadratic_ops_torch/tests suite. Run the tests and benchmarks from a source checkout.

  • scikit-build-core is no longer a runtime dependency; it is only needed to build the wheel.

v0.3.1#

New Features#

  • Explicit Conv1D (experimental): explicit_conv1d(x, weight), imported from subquadratic_ops_torch.experimental.explicit_conv1d, computes the depthwise “same” convolution of fft_conv1d as a dense GEMM against the Toeplitz matrix of the filter on fifth-generation tensor cores (compute capability 10.x: B200, B300 and later 10.x GPUs).

    • Sequence lengths: multiples of 128 in [128, 2048]; filters up to 2 * seq_len taps; bfloat16 and float16 inputs, float32 accumulation.

    • Differentiable w.r.t. both x and weight; the filter gradient is accumulated in float32.

    • Kernels ship precompiled (no runtime compilation, no compiler dependency at runtime); GPUs outside compute capability 10.x raise NotImplementedError.

    • 2-3x faster than fft_conv1d at these sequence lengths on the GB300.

    • The kernel reads the filters directly and builds its Toeplitz taps on the fly, so weights that change between calls cost nothing extra; float32 weights are cast to the input dtype on every call.

Performance Improvements#

  • Lower host overhead in every wrapper: kernel modules are resolved once and cached, the current CUDA stream is read through the raw getter (about 3 us to under 0.1 us per launch), and buffers the kernels fully overwrite are no longer zero-filled. This matters most at small batch, where the host side dominates.

  • The long-FFT route of fft_conv1d no longer keeps its stage scratch or zero-fills gradients for it, and fft_conv2d no longer saves the padded weight for backward (34 MB per layer at X=512, H=32).

  • causal_conv1d with channel_last=False no longer forces a contiguous copy of the input, so cuDNN frontend 1.29 can take its layout-preserving SM100 path for transposed views.

Bug Fixes#

  • causal_conv1d (channel_last=False) and b2b_causal_conv1d failed with TypeError: 'module' object is not callable on nvidia-cudnn-frontend 1.28 and newer, depending on which op was called first in the process. The cuDNN frontend op is now resolved through a guard that handles the frontend’s lazy exports, and it is resolved before tracing so torch.compile(fullgraph=True) no longer breaks on it.

  • fft_conv2d now rejects kernels larger than the FFT tile up front; previously they were silently cropped.

  • The fake (meta) implementations of the fft_conv2d and fused_fft_conv2d primitives now return contiguous outputs, matching the real kernels, so torch.compile and opcheck work with transposed inputs.

  • fft_causal_conv1d_packed: tensor-valued max_seqlen / min_seqlen now raise TypeError instead of silently synchronizing the device, and the fake implementation rejects float64 and one-entry cu_seqlens like the real path.

  • fft_conv1d checks the filter length against the device’s limit before allocating, raising ValueError instead of a generic C++ error.

Dependencies#

  • warp-lang minimum raised to 1.14.0.

v0.3.0#

New Features#

  • Packed (variable-length) FFT Causal Conv1D: New engine consuming the flash-attention-style packed layout, x: (hidden_dim, total_seq_len) plus a cu_seqlens offset vector, instead of a dense padded batch.

    • Sequences are bucketed by the FFT size each one needs, so short sequences do not pay for the longest sequence in the pack.

    • Differentiable w.r.t. both x and weight. Both gradients are conjugate multiplies, run as two kernels rather than one fused pass so each holds the same two spectra the forward does.

    • dweight accumulates across every sequence and bucket with a fixed reduction order, so two identical runs give bit-identical gradients.

    • Supports float32, bfloat16 and float16.

    • Sequence lengths up to 16384. The bucket key is N = max(128, 2*next_pow2(L)) – the filter length does not enter it, and there is no bound on the filter length, jointly with the sequence length or otherwise.

    • Nine instantiated FFT sizes, the powers of two from 128 to 32768. The top bucket, and with it sequence lengths above 8192, requires SM90 or newer; on older GPUs the engine raises rather than computing a wrong answer.

    • max_seqlen is required and min_seqlen defaults to 1; both prune the launch list and are load-bearing for performance.

Performance Improvements#

  • Faster channel-last Causal Conv1D: the NWH path of causal_conv1d (channel_last=True) is served by a new generation of native kernels, forward and backward. Nothing changes at the API – same signature, layouts, dtypes and activations – and the previous implementation stays in the wheel as an automatic fallback, so every configuration that ran before still runs; the fastest available implementation is selected per problem and device.

    • Measured on GB300 against the previous kernels (B=16, D=12288, L=8192): forward 1.2x-1.8x in float32 and up to 1.6x in bfloat16 across kernel widths, with float16 – newly admitted to the fast path – around 2x. Backward up to 3.6x in bfloat16 and up to 2x in float32. Small-batch (B=1) shapes gain up to 2.1x forward and 3.4x backward.

    • The forward pass now serves any feature dimension without dropping to a penalized path. The backward fast path expects 16-byte-aligned feature dimensions and falls back safely otherwise.

Bug Fixes#

  • Fixed the channel-last (NWH) float64 path for large kernel widths: silu silently produced wrong results for widths 66-126 (with an illegal memory access at 127), while identity refused widths >= 64 the hardware can serve. float64 now serves widths 2-65, bounded by the device’s shared-memory limit, and anything beyond raises a descriptive error instead of computing.

  • GPU architecture dispatch now maps new compute capabilities to their family tier instead of rejecting them: GB300 / B300 (CC 10.3) previously failed every fft_conv1d variant with invalid argument (CUDA error 1). Filters up to 16384 now route to the short-FFT path on all 227 KB shared-memory parts (SM 9.0-11.x), while 12.x parts cap at 8192 by design.

Breaking Changes#

  • B2B Causal Conv1D is now served by the cuDNN frontend instead of a kernel built into this wheel. b2b_causal_conv1d keeps its signature, its argument names and its return value, and the numerics are unchanged – the two implementations were measured bit-identical, forward and for all four gradients, across the whole of the previously supported kernel-size grid. What changes is the envelope and the environment:

    • float64 is no longer supported and raises ValueError. Earlier releases accepted it. The other operations in this package are unaffected.

    • Kernel widths are now contiguous ranges rather than a discrete allowlist: projection 2–32 and mixer 2–256, so widths such as 5, 9 or 31 are served where they previously raised. Nothing that worked before stops working; the fp32 backward pass, which previously refused a mixer width of 256, now accepts it. A width outside these ranges raises ValueError before any kernel runs.

    • Requires ``nvidia-cudnn-frontend`` >= 1.27.0 and a cuDNN runtime >= 9.24, both now declared as dependencies. There is no fallback: an environment that cannot serve the operation raises a diagnostic naming the GPU, its device index and compute capability, and the frontend and backend versions actually resolved. Note that torch pins cuDNN 9.20 exactly, so pip check reports a conflict; the fix is to install 9.24 and the frontend ahead of torch’s pinned copy in the environment or container image.

    • CUDA-only, as before, and observed on SM90. The published cuDNN documentation for this operation advertises SM100 and newer; there is no hard architecture gate, and an unsupported device surfaces through the diagnostic above.

    The native CUDA sources, their binding and their C++ tests remain in the tree.

  • Causal Conv1D NHW (channel-first) is now served by the cuDNN frontend; the NWH (channel-last) layout keeps this package’s native kernels. causal_conv1d keeps its signature and layouts, and the NHW numerics are unchanged – native and frontend were measured bit-identical, forward and all three gradients, across dtypes, activations and kernel sizes. What changes on the NHW path:

    • float64 is no longer supported on NHW and raises ValueError. The NWH layout still serves it.

    • Kernel sizes are 2–256, matching the native JIT envelope; out-of-range sizes raise ValueError before any kernel runs.

    • Requires the same frontend/runtime as B2B above, with the same no-fallback diagnostic. The NWH layout has no such requirement.

    Wheel builds no longer compile the native NHW kernels; the sources, their binding and their C++ tests remain in the tree.

v0.2.0#

New Features#

  • Python version support: Added Python 3.14 support.

  • FFT Conv1D: New non-causal 1D convolution using real FFT.

    • Forward and backward pass with full autograd support.

    • Supports FFT sizes from 32 to 8192.

    • Automatic input/filter padding and FFT size validation.

    • Supports float16, bfloat16, float32, and float64 data types.

  • Causal Conv1D NHC layout: Added support for channel-last memory format in causal convolution.

  • Causal Conv1D NCH layout: Improved performance for workloads with 128-byte aligned sequences and 16-byte data types.

  • FFT Conv2D optimizations: Performance improvements to 2D FFT convolution kernels. Supports 2D depthwise convolution with ‘same’ padding using FFT.

Bug Fixes#

  • Fixed int32 index overflow for large tensors when batch_size * hidden_dim * seq_dim exceeds 2^31 elements. All kernel layout computations now use 64-bit indexing.

  • Fixed causal conv1d backward kernel producing incorrect weight gradients for certain sequence lengths due to uninitialized values.

  • Fixed padding calculation in fft_conv2d.py for correct same-padding semantics.

v0.1.1#

First official release of subquadratic_ops_torch, providing GPU-accelerated CUDA kernels for subquadratic operations with PyTorch bindings.

New Features#

  • B2B Causal Conv1D: Back-to-back depthwise causal 1D convolution fused kernel for the Striped Hyena 2 architecture used in the Evo2 model.

    • Fuses projection convolution, element-wise gating, mixer convolution, and skip connection into a single kernel launch.

    • Forward and backward pass with full autograd support.

    • Supports float16, bfloat16, float32, and float64 data types.

    Note

    Describes v0.1.1 as released. The float64 support and the discrete kernel-size allowlist recorded here no longer apply – see the v0.3.0 breaking changes above.

  • Causal Conv1D: Depthwise causal 1D convolution with optional bias and activation.

    • Forward and backward pass with full autograd support.

    • Built-in SiLU activation support.

    • Supports float16, bfloat16, float32, and float64 data types.

    • NCH layout support.

  • FFT Causal Conv1D: Causal 1D convolution using real FFT and IFFT for long filter sizes.

    • Short-FFT path for filters up to 16384 (SM 9.0+) or 8192 (older architectures).

    • Long-FFT path for arbitrary filter lengths.

    • Forward and backward pass with full autograd support.

    • Supports float16, bfloat16, float32, and float64 data types.

  • FFT Conv2D: FFT-based 2D depthwise separable convolution with same-padding semantics.

    • Forward and backward pass with full autograd support.

    • Supports float32, and float64 data types.

  • Implicit Filter: Implicit modal filter generation for the Hyena architecture.

    • Memory-efficient implementation of implicit modal filter.

    • Forward and backward pass with full autograd support.

    • Built on NVIDIA Warp for tiled kernel launches.

  • Rearrange: CUDA-accelerated tensor layout transposition between (B, H, L) and (L, B, H) formats with autograd support.

Platform Support#

  • CUDA-compatible NVIDIA GPUs (Ampere and newer).

  • CUDA Toolkit 12.0+.

  • Python 3.11–3.13.