Dual numbers are 2-dimensional numbers that can be regarded as an extension of the real numbers. Along with the complex and split-complex numbers, they are one of the three 2-dimensional unital algebras.

Each dual number can be described as a linear combination of the real unit and the dual unit . Whereas the complex numbers feature an imaginary unit that squares to , the dual unit squares to zero, . This leads to the following algebraic operations.

We will write a dual number as the pair .

Addition and subtraction are element-wise,

with identity .

Multiplication is

which is commutative and has identity .

Division is therefore

From this we can see that

## Extending Real Functions to the Dual Plane

We can use this algebraic structure to extend any real analytic function to the dual plane, by taking its Taylor series and substituting a dual argument.

We can use this rule to uniquely extend any differentiable function to the dual plane, or indeed any suitable subset of the real line to the corresponding subset of the dual plane. This extension is compatible with the composition of functions, i.e. the extension of a composition is the composition of the extensions,

This is useful as it means we can extend complicated functions by composing elementary functions.

For example, to extend we extend to give , then square the result to give .

In other words, we don’t need to know the derivative of the complicated function in order to extend it, we only need to know the derivatives of the elementary functions of which it is composed.

## Automatic Differentiation

A real function’s derivative at a given point can be obtained by applying its extension to the dual number and then taking the dual part of the result,

This is called automatic differentiation. This is particularly useful when we don’t necessarily need to know as a function, we just want to obtain its value at a particular point . If the function is a complicated combination of many elementary functions, then computing the derivative on paper can be both time-consuming and error-prone. Instead of finding in full and then evaluating it, we can just evaluate the original function with a dual number.

This technique can be applied to multivariable functions, such as vector fields, , giving

The dual part is the directional derivative of the vector field in the direction , evaluated at . By choosing the direction to be the th basis vector, , and then taking the th component of the result, we can automatically compute the partial derivatives of the vector field,

If we denote the dual part as then the Jacobian is given by

This means we can compute the derivatives of an -dimensional vector field by evaluating the vector field times with dual numbers.

In a practical implementation it is more convenient to think of as a vector of dual numbers, rather than a pair of real vectors. This allows for a single implementation of the vector field to be evaluated with a vector of either real or dual elements, for example, by templating the scalar type in C++:

template<typename Scalar> Vector<Scalar> MyVectorField( const Vector<Scalar>& x ) { // evaluate the vector field }

The version of automatic differentation discussed here (using dual numbers) is called forward automatic differentiation. There is an alternate version called reverse automatic differentiation, and also extensions to the higher derivatives. Wikipedia has a decent article on the topic.

Fascinating . I’ve got to go try this

LikeLike