Autograd engine

cppgrad implements reverse-mode automatic differentiation. As you compute with tensors, the library records a graph of operations; calling .backward() walks that graph in reverse to compute gradients via the chain rule.

Building the graph#

You don't build the graph explicitly. Every differentiable operation on a tensor with requires_grad=true appends a node that remembers:

• the inputs it consumed, and
• the gradient function (GradFn) that knows how to differentiate it.

Tensor a = Tensor::full({2, 2}, 3.0f, /*requires_grad=*/true);
Tensor b = Tensor::full({2, 2}, 2.0f, /*requires_grad=*/true);
 
Tensor c = a + b;   // records an add node
Tensor d = c * b;   // records a multiply node

The backward pass#

Calling .backward() on a scalar (or seeded) output traverses the recorded nodes in reverse topological order, applying each GradFn to accumulate gradients into the leaf tensors.

d.backward();
 
a.grad();   // ∂d/∂a
b.grad();   // ∂d/∂b

Gradient functions#

Each op has a matching gradient function. GradFn is the base class, and subclasses implement the backward rule for a specific operation — for example SumFunction and MeanFunction. Prebuilt functions cover Neg, Exp, Log, Pow, Sum, Mean, Max, and more, so common models differentiate out of the box.

Next#

• See which tensor operations are available in Tensors.
• Run on a GPU by configuring the backend.