Release Notes#

v0.3.0#

New Features#

  • Packed (variable-length) FFT Causal Conv1D: New engine consuming the flash-attention-style packed layout, x: (hidden_dim, total_seq_len) plus a cu_seqlens offset vector, instead of a dense padded batch.

    • Sequences are bucketed by the FFT size each one needs, so short sequences do not pay for the longest sequence in the pack.

    • Differentiable w.r.t. both x and weight. Both gradients are conjugate multiplies, run as two kernels rather than one fused pass so each holds the same two spectra the forward does.

    • dweight accumulates across every sequence and bucket with a fixed reduction order, so two identical runs give bit-identical gradients.

    • Supports float32, bfloat16 and float16.

    • Sequence lengths up to 16384. The bucket key is N = max(128, 2*next_pow2(L)) – the filter length does not enter it, and there is no bound on the filter length, jointly with the sequence length or otherwise.

    • Nine instantiated FFT sizes, the powers of two from 128 to 32768. The top bucket, and with it sequence lengths above 8192, requires SM90 or newer; on older GPUs the engine raises rather than computing a wrong answer.

    • max_seqlen is required and min_seqlen defaults to 1; both prune the launch list and are load-bearing for performance.

Performance Improvements#

  • Faster channel-last Causal Conv1D: the NWH path of causal_conv1d (channel_last=True) is served by a new generation of native kernels, forward and backward. Nothing changes at the API – same signature, layouts, dtypes and activations – and the previous implementation stays in the wheel as an automatic fallback, so every configuration that ran before still runs; the fastest available implementation is selected per problem and device.

    • Measured on GB300 against the previous kernels (B=16, D=12288, L=8192): forward 1.2x-1.8x in float32 and up to 1.6x in bfloat16 across kernel widths, with float16 – newly admitted to the fast path – around 2x. Backward up to 3.6x in bfloat16 and up to 2x in float32. Small-batch (B=1) shapes gain up to 2.1x forward and 3.4x backward.

    • The forward pass now serves any feature dimension without dropping to a penalized path. The backward fast path expects 16-byte-aligned feature dimensions and falls back safely otherwise.

Bug Fixes#

  • Fixed the channel-last (NWH) float64 path for large kernel widths: silu silently produced wrong results for widths 66-126 (with an illegal memory access at 127), while identity refused widths >= 64 the hardware can serve. float64 now serves widths 2-65, bounded by the device’s shared-memory limit, and anything beyond raises a descriptive error instead of computing.

  • GPU architecture dispatch now maps new compute capabilities to their family tier instead of rejecting them: GB300 / B300 (CC 10.3) previously failed every fft_conv1d variant with invalid argument (CUDA error 1). Filters up to 16384 now route to the short-FFT path on all 227 KB shared-memory parts (SM 9.0-11.x), while 12.x parts cap at 8192 by design.

Breaking Changes#

  • B2B Causal Conv1D is now served by the cuDNN frontend instead of a kernel built into this wheel. b2b_causal_conv1d keeps its signature, its argument names and its return value, and the numerics are unchanged – the two implementations were measured bit-identical, forward and for all four gradients, across the whole of the previously supported kernel-size grid. What changes is the envelope and the environment:

    • float64 is no longer supported and raises ValueError. Earlier releases accepted it. The other operations in this package are unaffected.

    • Kernel widths are now contiguous ranges rather than a discrete allowlist: projection 2–32 and mixer 2–256, so widths such as 5, 9 or 31 are served where they previously raised. Nothing that worked before stops working; the fp32 backward pass, which previously refused a mixer width of 256, now accepts it. A width outside these ranges raises ValueError before any kernel runs.

    • Requires ``nvidia-cudnn-frontend`` >= 1.27.0 and a cuDNN runtime >= 9.24, both now declared as dependencies. There is no fallback: an environment that cannot serve the operation raises a diagnostic naming the GPU, its device index and compute capability, and the frontend and backend versions actually resolved. Note that torch pins cuDNN 9.20 exactly, so pip check reports a conflict; the fix is to install 9.24 and the frontend ahead of torch’s pinned copy in the environment or container image.

    • CUDA-only, as before, and observed on SM90. The published cuDNN documentation for this operation advertises SM100 and newer; there is no hard architecture gate, and an unsupported device surfaces through the diagnostic above.

    The native CUDA sources, their binding and their C++ tests remain in the tree.

  • Causal Conv1D NHW (channel-first) is now served by the cuDNN frontend; the NWH (channel-last) layout keeps this package’s native kernels. causal_conv1d keeps its signature and layouts, and the NHW numerics are unchanged – native and frontend were measured bit-identical, forward and all three gradients, across dtypes, activations and kernel sizes. What changes on the NHW path:

    • float64 is no longer supported on NHW and raises ValueError. The NWH layout still serves it.

    • Kernel sizes are 2–256, matching the native JIT envelope; out-of-range sizes raise ValueError before any kernel runs.

    • Requires the same frontend/runtime as B2B above, with the same no-fallback diagnostic. The NWH layout has no such requirement.

    Wheel builds no longer compile the native NHW kernels; the sources, their binding and their C++ tests remain in the tree.

v0.2.0#

New Features#

  • Python version support: Added Python 3.14 support.

  • FFT Conv1D: New non-causal 1D convolution using real FFT.

    • Forward and backward pass with full autograd support.

    • Supports FFT sizes from 32 to 8192.

    • Automatic input/filter padding and FFT size validation.

    • Supports float16, bfloat16, float32, and float64 data types.

  • Causal Conv1D NHC layout: Added support for channel-last memory format in causal convolution.

  • Causal Conv1D NCH layout: Improved performance for workloads with 128-byte aligned sequences and 16-byte data types.

  • FFT Conv2D optimizations: Performance improvements to 2D FFT convolution kernels. Supports 2D depthwise convolution with ‘same’ padding using FFT.

Bug Fixes#

  • Fixed int32 index overflow for large tensors when batch_size * hidden_dim * seq_dim exceeds 2^31 elements. All kernel layout computations now use 64-bit indexing.

  • Fixed causal conv1d backward kernel producing incorrect weight gradients for certain sequence lengths due to uninitialized values.

  • Fixed padding calculation in fft_conv2d.py for correct same-padding semantics.

v0.1.1#

First official release of subquadratic_ops_torch, providing GPU-accelerated CUDA kernels for subquadratic operations with PyTorch bindings.

New Features#

  • B2B Causal Conv1D: Back-to-back depthwise causal 1D convolution fused kernel for the Striped Hyena 2 architecture used in the Evo2 model.

    • Fuses projection convolution, element-wise gating, mixer convolution, and skip connection into a single kernel launch.

    • Forward and backward pass with full autograd support.

    • Supports float16, bfloat16, float32, and float64 data types.

    Note

    Describes v0.1.1 as released. The float64 support and the discrete kernel-size allowlist recorded here no longer apply – see the v0.3.0 breaking changes above.

  • Causal Conv1D: Depthwise causal 1D convolution with optional bias and activation.

    • Forward and backward pass with full autograd support.

    • Built-in SiLU activation support.

    • Supports float16, bfloat16, float32, and float64 data types.

    • NCH layout support.

  • FFT Causal Conv1D: Causal 1D convolution using real FFT and IFFT for long filter sizes.

    • Short-FFT path for filters up to 16384 (SM 9.0+) or 8192 (older architectures).

    • Long-FFT path for arbitrary filter lengths.

    • Forward and backward pass with full autograd support.

    • Supports float16, bfloat16, float32, and float64 data types.

  • FFT Conv2D: FFT-based 2D depthwise separable convolution with same-padding semantics.

    • Forward and backward pass with full autograd support.

    • Supports float32, and float64 data types.

  • Implicit Filter: Implicit modal filter generation for the Hyena architecture.

    • Memory-efficient implementation of implicit modal filter.

    • Forward and backward pass with full autograd support.

    • Built on NVIDIA Warp for tiled kernel launches.

  • Rearrange: CUDA-accelerated tensor layout transposition between (B, H, L) and (L, B, H) formats with autograd support.

Platform Support#

  • CUDA-compatible NVIDIA GPUs (Ampere and newer).

  • CUDA Toolkit 12.0+.

  • Python 3.11–3.13.