Research CommonsResearch Commons
cpptensor/Numerics & dtypes

Numerics & dtypes

cpptensor's numerical stability policy for reductions and dot products, and its dtype model.

cpptensor favors accuracy on cancellation-heavy inputs for the operations where it matters most, and tracks dtype metadata across the API.

Numerical stability policy

sum(), mean(), and dot() prioritize accumulation accuracy over raw throughput:

  • CPU reductions use widened, compensated accumulation before casting back to float.
  • AVX runtime dispatch still uses optimized paths for pointwise and matmul-style kernels, but sum / mean route through the stable reduction implementation.
  • dot() accumulates products in widened precision before casting the scalar result back to float.
Why it matters

Naive single-precision summation loses accuracy when adding many values of differing magnitude. Compensated, widened accumulation keeps reductions reliable without forcing every op onto a slower path.

Dtypes

Tensor tracks element dtype metadata across the API:

  • Supported dtypes: bool, int32, float32, float64.
  • Comparison operators produce bool tensors.
  • Dtype is preserved across views, clone / contiguous, and factory creation (zeros, ones, full, randn).

Next