## Recurrence relations

# Warm-up: The Fibonacci sequence

The #Fibonacci sequence# is the sequence $f_n$ satisfying
:   $f_0 = 0, f_1=1$
:   $f_{n+2} = f_n + f_{n+1}$
so
:   0,1,1,2,3,5,8,13,21,34,55,...

Such an expression for a term in a sequence as a function of previous terms 
is called a #recurrence relation#.

Other examples:
    :	$h_0=1$
    :	$h_{n+1}=h_n+3$

    :	$h_0=1$
    :	$h_{n+1}=3h_n$

In these cases, we can easily find an expression for $h_n$ in terms of $n$.

Can we do this for $f_n$?

To do so, we should consider the more general problem where we vary the
initial values $f_0$ and $f_1$, and just consider sequences $f_n$ satisfying
the recurrence relation $f_{n+2}=f_n+f_{n+1}$.

If $f_n$ and $f'_n$ are two such sequences, then so is $c_1f_n+c_2f'_n$ for
any $c_1,c_2$ (i.e. the solutions form a vector space).

So if we can find #some# solutions to the Fibonacci recurrence relation, we
can easily generate more - perhaps including the Fibonacci sequence itself.

(In fact, if you recall your linear algebra, you should be able to see that we
only need to find two linearly independent sequences to generate all of them)

Let's look for solutions of the particularly simple form
:   $f_n = q^n$
with $q != 0$. Then the recurrence relation becomes
:   $q^{n+2}=q^n+q^{n+1}$
:   $<=> q^n(q^2-q-1)=0$
:   $<=> (q^2-q-1)=0$ (since $q!=0$)

This is a quadratic equation, so it has two solutions.

They are
:   $\phi = \frac{1+\sqrt 5}2, \phi' = \frac{1-\sqrt 5}2$.

($\phi$ is known as the Golden Ratio; it is the unique positive real
satisfying $\frac{1+\phi}\phi = \phi$)

So for any $c_1$ and $c_2$,
:   $c_1\phi^n + c_2\phi'^n$
satisfies the Fibonacci recurrence relation.

Let's find such a sequence which satisfies the initial conditions of the
Fibonacci sequence, $f_0=f_1=1$; then it must be the Fibonacci sequence.

:   $c_1 + c_2 = 0$
:   $c_1\phi + c_2\phi' = 1$

We can solve this system of simultaneous equations

\[ \begin{pmatrix}
	1 & 1 \\\phi & \phi'
    \end{pmatrix}
    \begin{pmatrix}
	c_1 \\c_2
    \end{pmatrix}
=
    \begin{pmatrix}
	0 \\1
    \end{pmatrix}
\]
\[
    \begin{pmatrix}
	c_1 \\c_2
    \end{pmatrix}
=
    \frac 1{\phi'-\phi}
    \begin{pmatrix}
	\phi' & -1 \\-\phi & 1
    \end{pmatrix}
    \begin{pmatrix}
	0 \\1
    \end{pmatrix}
=
    \frac 1{\sqrt 5}
    \begin{pmatrix}
	-1 \\1
    \end{pmatrix}
\]

So we obtain

Theorem:
    The Fibonacci numbers are
    :	$f_n = \frac{\phi^n - \phi'^n}{\sqrt 5}$
    where
    :   $\phi = \frac{1+\sqrt 5}2, \phi' = \frac{1-\sqrt 5}2$.

Note that $\phi'=1-\phi$, so we could also write this as
:   $f_n = \frac{\phi^n - (1-\phi)^n}{\sqrt 5}$.


# Homogeneous Linear Recurrence Relations with Constant Coefficients

A _homogeneous linear recurrence relation with constant coefficients_ is an
equation
:   $h_{n+k} = a_0 h_n + a_1 h_{n+1} + ... + a_{k-1} h_{n+k-1}$,
with $a_i$ complex numbers.

$k$ is the _order_ of the recurrence relation.
A number sequence $h_n$ satisfying the recurrence relation is called a
_solution_ to the recurrence relation.

If we add _initial conditions_
:   $h_0= c_0, ..., h_{k-1} = c_{k-1}$,
this clearly uniquely determines a solution.

Examples:
    **(i) The Fibonacci sequence.
    ** Geometric sequences, $h_{n+1} = a h_n$.
    ** The life-cycle of /inventioni exemplicus/ is as follows:
	a new hatchling remains in the larval stage until its first summer,
	then spends a year maturing, and in the subsequent summer lays
	a clutch of 7 eggs (which quickly hatch into larvae),
	then in the summer after lays a second clutch of 6 eggs, then dies.

	All /excemplicus/ are female (they reproduce parthenogenetically).

	Suppose no /exemplicus/ die except at the end of their life cycle.

	If 100 exemplicus hatchlings are introduced one summer, how many
	exemplicus larvae will there be at the end of the $n$th summer
	thereafter?

	Solution:
	    At the end of the $n$th summer, there are 7 larvae born from each
	    2-year-old /exemplicus/, and 6 from each 3-year-old.

	    So $h_n = 6h_{n-3} + 7h_{n-2}$ for $n>=3$,
	    i.e. $h_{n+3} = 6h_n + 7h_{n+1}$ for $n>=0$.

	    We also have the initial conditions
	    :   $h_0 = 100, h_1=0, h_2=700$.

	    So we get
	    :   $h_3=600, h_4=4900, h_5=8400, h_6=37900,...$
    *ee*

We proceed to generalise the solution to the Fibonacci recurrence relation to
solve general homogeneous linear recurrence relation with constant
coefficients.

Given a recurrence relation
:   $h_{n+k} = a_0 h_n + a_1 h_{n+1} + ... + a_{k-1} h_{n+k-1}$,
:   i.e. $h_{n+k} = \sum_{j=0}^{k-1}a_jh_{n+j}$,
we again look for solutions $h_n = q^n$.

Clearly $h_n = q^n$ is a solution iff
:   $q^k = a_0 + a_1 q + ... + a_{k-1} q^{k-1}$,
:   i.e. $q^k - a_{k-1}q^{k-1} - ... - a_1 q - a_0 = 0$.

The polynomial $x^k - a_{k-1}x^{k-1} - ... - a_1 x - a_0$ is called the
_characteristic polynomial_ of the recurrence relation.
It is a degree $k$ polynomial, so has $k$ roots in the complex numbers
(counting multiplicities).

Suppose that it has $k$ #distinct# roots, $q_1,...,q_k$.
(See the "Bonus" section for what happens when we have repeated roots.)

Claim:
    the $k$ vectors $((q_1^0,...,q_1^{k-1}),...,(q_k^0,...,q_k^{k-1}))$ are
    linearly independent in $\|C^k$,
Proof:
    Otherwise, considering the columns of the $k$-by-$k$ matrix whose rows are
    these vectors,
    the $k$ vectors $((q_1^0,...,q_k^0),...,(q_1^{k-1},...,q_k^{k-1}))$ are
    linearly dependent.
    i.e. there are $b_0,...,b_{k-1} \in \|C$ not all 0, such that the
    polynomial
    :	$b_0 + b_1 x + ... + b_{k-1}x^{k-1}$
    has roots $q_1,...,q_k$.

    But a degree $k-1$ polynomial can't have $k$ distinct roots;
    contradiction.
    \qed

So any given initial conditions $h_0=c_0,...,h_{k-1}=c_{k-1}$ can be written
as a linear combination
:   $h_n = b_1q_1^n + ... + b_kq_k^n = \sum_{i=1}^kb_iq_i^n$.

Taking this as a definition of $h_n$ for all $n$,
we see that not only does it satisfy the initial conditions by choice of
$b_i$,
but it satisfies the recurrence relation; indeed
:   $h_{n+k} = \sum_{i=1}^k b_iq_i^{n+k}$
:       $= \sum_{i=1}^k b_iq_i^nq_i^k$
:       $= \sum_{i=1}^k b_iq_i^n(\sum_{j=0}^{k-1}a_jq_i^j)$
:       $= \sum_{j=0}^{k-1}\sum_{i=1}^k b_iq_i^na_jq_i^j$
:       $= \sum_{j=0}^{k-1}a_j\sum_{i=1}^k b_iq_i^{n+j}$
:       $= \sum_{j=0}^{k-1}a_jh_{n+j}$


Example:
    Let's solve the /exemplicus/ example.

    :   $h_{n+3} = 6h_n + 7h_{n+1}$,
    :   $h_0 = 100, h_1=0, h_2=700$.

    The characteristic polynomial is
    :   $x^3 - 7x - 6 = (x-3)(x+2)(x+1)$
    so the solutions are of the form
    :   $h_n = b_1 3^n + b_2 (-2)^n + b_3 (-1)^n$.

    Solving
    :   $100 = h_0 = b_1 + b_2 + b_3$
    :   $0 = h_1 = 3b_1 - 2b_2 - b_3$
    :   $700 = h_2 = 9b_1 + 4b_2 + b_3$

    gives
    :   $b_1=45, b_2=80, b_3=25$.

    So the solution is
    :   $h_n = 45*3^n + 80*(-2)^n - 25*(-1)^n$.
    \qed

# Bonus: Solving recurrence relations with generating functions

Generating functions provide a convenient device for solving recurrence
relations (although in theoretical terms, they only provide a different
way to package the same linear algebra).

If $g(x)$ is the generating function for the sequence $h_n$,
i.e. the coefficient of $x^n$ in $g(x)$ is $h_n$,
then the coefficient of $x^{n+1}$ in $xg(x)$ is $h_n$.

So if $h_n$ satisfy a recurrence relation 
:   $h_{n+k} = a_0 h_n + a_1 h_{n+1} + ... + a_{k-1} h_{n+k-1}$
then in
:   $g(x) - a_0 x^kg(x) - a_1x^{k-1}g(x) - ... - a_{k-1}xg(x)$,
$x^{n+k}$ has coefficient 0 for $n>=0$,
i.e. this is a polynomial of order $k-1$.

Using initial conditions, we can find this polynomial, and so express $g(x)$
as a rational function.

For example, consider the Fibonacci relations
$f_{n+2}=f_n+f_{n+1}, f_0=0,f_1=1$.

If $g(x)$ is the generating function, then
:   $g(x) - xg(x) - x^2g(x) = f_0 + f_1x + f_2x^2 + f_3x^3 + ... - f_0x - f_1x^2 - f_2x^3 - ... -
f_0 x^2 - f_1 x^3 - ...$
:       $= f_0 + (f_1-f_0)x + (f_2-f_1-f_0)x^2 + (f_3-f_2-f_1)x^3 + ...$
:       $= f_0 + (f_1-f_0)x$
:       $= 0 + (1-0)x$
:       $= x$,
so
:   $(1-x-x^2)g(x) = x$
so
:   $g(x) = \frac{x}{1-x-x^2}$.

Furthermore, by factoring the denominator and finding partial fractions, we
can expand this as a power series and so solve the recurrence equations.
In this case, the solutions to $1-x-x^2=0$ are the reciprocals of the
solutions $\phi,\phi'$ to $x^2-x-1$, so
:   $g(x) = \frac{x}{1-x-x^2}$
:       $= \frac{x}{(x-\phi^{-1})(x-\phi'^{-1})}$
:       $= \frac{\phi\phi'x}{(1-\phi x)(1-\phi' x)}$
:       $= \frac{a}{1-\phi x} + \frac{b}{1-\phi' x}$
where
:   $0 = a+b$
:   $\phi\phi' = -\phi' a - \phi b$
:   $=> b=-a$
:   $=> \phi\phi' = a(\phi-\phi')$
:       $=> a=\frac{1}{\sqrt 5}$ (using the definitions of $\phi,\phi'$)
:   $=> b=\frac{-1}{\sqrt 5}$
so
:   $g(x) = \frac{1}{\sqrt 5}((1-\phi x)^{-1} - (1-\phi' x)^{-1})$
:       $= \frac{1}{\sqrt 5}((\sum_{n=0}^\infty \phi^n) - (\sum_{n=0}^\infty \phi'^n))$
:       $= \frac{1}{\sqrt 5}\sum_{n=0}^\infty (\phi^n-\phi'^n)$
so we reclaim the formula we found before,
:   $f_n = \frac{\phi^n - \phi'^n}{\sqrt 5}$,
and we had to do the same algebra to get there.


# Bonus: abstract reformulation, and handling repeated roots

Consider the (infinite dimensional) complex vector space of complex number
sequences $h = h_0,h_1,...$. Let $\sigma$ be the downshift operator, which
from a sequence $h$ obtains the new sequence $(\sigma h)_n = h_{n+1}$. Note
that this is a linear map.

Then a linear homogeneous constant coefficient recurrence relation, which we
can write as $\sum_{i=0}^k a_i h_{n+i}=0$, with $a_k!=0$, can be rewritten as
:   $(\sum_{i=0}^k a_i\sigma^i) h = 0$.

The subspace of solutions to this is then the kernel of the linear operator
$\sum_{i=0}^k a_i\sigma^i$. This is a finite dimensional vector space. The
solution method described above is a matter of finding a basis of eigenvectors
of $\sigma$ on this space. Note that eigenvectors are precisely geometric
series $cq^n$.

In general of course, the eigenvectors of $\sigma$ won't span the space. But
its generalised eigenvectors will. One can check inductively that the $k$th
generalised $q$-eigenspace of $\sigma$, i.e. the kernel of $(\sigma-q)^k$,\
is the space of sequences $f(n)q^n$ where $f$ is a polynomial of degree at
most $k-1$. So if the characteristic polynomial factors as
:   $\sum_{i=0}^k a_i\sigma^i = \Pi (\sigma-q_i)^{k_i}$,
the space of solutions has a basis of eigensequences
:   $n^jq_i^n$ where $j < k_i$,
so any solution can be expressed as a linear combination of these.

We might as well note that this abstract formulation also applies to
homogeneous linear differential equations over the constants: replace the
space of sequences with, say, the space of complex analytic functions in one
variable, and replace $\sigma$ with the differentiation operator. The $k$th
generalised $\lambda$-eigenspace consists of $f(x)e^{\lambda x}$ for $f(x)$ a
polynomial of degree at most $k-1$.
