How to Solve Equations Using Fixed Point Iteration: A Step-by-Step Guide

The Fixed Point Iteration Formula

Once the equation has been rewritten as

$$x = g(x),$$

the method generates successive approximations using the iteration formula

$$x_{n+1} = g(x_n),$$

Where \( x_n \) is the current approximation, \( x_{n + 1} \) is the next approximation, and \( n=0,1,2,\ldots \).

The process begins with an initial guess \( x_0 \). Each new approximation is obtained by evaluating the iteration function at the previous approximation.

Choosing an Initial Guess

The starting value \( x_0 \) is called the initial guess or initial approximation. Its selection can significantly affect the method’s behavior.

If the initial guess is sufficiently close to a fixed point and the iteration function satisfies the appropriate convergence conditions, the sequence of approximations will converge to the desired solution. On the other hand, a poor initial guess or an unsuitable iteration function may cause the sequence to diverge, oscillate indefinitely, or converge to an unintended fixed point.

In the following sections, we’ll examine why fixed point iteration converges for some functions, why it fails for others, and how to determine whether a particular iteration function is suitable before performing any calculations.

Why Does Fixed Point Iteration Work?

At first glance, the fixed point iteration method may seem surprisingly simple. Starting from an initial approximation, the method repeatedly evaluates the iteration function

$$x_{n + 1}=g(x_n),$$

until the approximations stop changing significantly. The natural question is: Why should repeatedly applying a function solve the original equation?

The answer lies in the concept of a fixed point. If the sequence of approximations converges to a value \( x \), then successive approximations become nearly identical. Taking the limit of the iteration formula gives

$$x = g(x),$$

which is precisely the definition of a fixed point.

Since the original equation was rewritten in the form

$$x = g(x),$$

the fixed point \( x \) is also a solution of the original equation. Thus, the goal of the iteration process is to generate a sequence that approaches this fixed point.

The Iterative Process

The fixed point iteration method begins with an initial approximation \( x_0 \). Applying the iteration function repeatedly produces the sequence

$$\{ x_0, x_1=g(x_0), \dots, x_{n+1} = g(x_n) \}.$$

If the method converges, each new approximation is closer to the fixed point than the previous one. As the number of iterations increases, the difference between successive approximations becomes smaller until the desired level of accuracy is reached.

A Graphical Interpretation

The convergence of fixed point iteration can also be understood graphically. Consider the graphs of

$$y = g(x)$$

and

$$y = x.$$

Their point of intersection represents the fixed point because it satisfies

$$g(x) = x.$$

Suppose we begin with an initial guess \( x_0 \). Evaluating \( g(x_0) \) gives the next approximation \( x_1 \). If we alternate between the curve \( y = g(x) \) and the line \( y = x \), the sequence of approximations traces a staircase-like or spiral-like path toward the point where the two graphs intersect.

When the iteration function is chosen appropriately, this staircase moves closer to the fixed point with each iteration. If the function is poorly chosen, however, the staircase may move away from the intersection or oscillate without ever settling to a single value.

The Role of the Derivative

Whether the approximations converge depends largely on the behavior of the iteration function near the fixed point. In particular, the derivative \( g'(x) \) measures how rapidly the function changes, as discussed in the article How to Differentiate a Function Step by Step: A Beginner’s Guide.

A fundamental result in numerical analysis states that if

$$|g'(x)| < 1,$$

where \( x \) is the fixed point, then the iteration is locally convergent. Intuitively, each application of the iteration function reduces the error, causing successive approximations to converge toward the solution. On the other hand, if

$$|g'(x)| > 1,$$

errors tend to grow rather than shrink. In this case, the sequence usually diverges away from the fixed point.

In the next section, we’ll examine the mathematical conditions that determine when fixed point iteration succeeds and when it fails, allowing you to assess whether an iteration function is likely to converge before performing any calculations.

When Does Fixed Point Iteration Succeed?

Fixed point iteration is often simple to implement, but its convergence is not guaranteed. Whether the method succeeds depends on both the choice of the iteration function \( g(x) \) and the initial approximation. Understanding these requirements before performing any calculations can prevent unnecessary computations and help you select an iteration function that is likely to converge.

Continuity of the Iteration Function

The first requirement is that the iteration function \( g(x) \) be continuous on an interval containing the fixed point. For a refresher on continuous functions, please refer to the article A Complete Introduction to the Bisection Method with Worked Examples and Solutions.

A continuous function has no jumps, holes, or breaks, allowing each successive approximation to be computed without interruption.

Although continuity alone is not enough to guarantee convergence, it forms the foundation of the convergence analysis.

The Derivative Condition

The most important convergence condition involves the derivative of the iteration function.

If

$$|g'(x)| < 1.$$

throughout an interval containing the fixed point, then each iteration reduces the error. In other words, successive approximations move closer to the solution rather than farther away from it.

This condition also explains why different rearrangements of the same equation can produce different results. Two algebraically equivalent equations may generate different iteration functions, and one may converge while the other diverges.

Conversely, if

$$|g'(x)| > 1$$

near the fixed point, errors generally increase with each iteration. As a result, the approximations move away from the solution instead of toward it.

Choosing a Good Initial Guess

Even when the convergence conditions are satisfied, selecting a suitable initial approximation remains important.

An initial guess that is close to the fixed point generally leads to faster convergence. In contrast, a poor starting value may slow the iteration, cause convergence to an unintended fixed point, or even prevent convergence altogether.

In practice, an initial approximation is often obtained by graphing the function, examining a table of values, or using physical intuition about the problem being solved.

When these conditions are satisfied, the fixed point iteration method often converges rapidly to the desired solution. In the next section, we’ll examine the situations in which these conditions are violated and explain why fixed point iteration may fail.

When Does Fixed Point Iteration Fail?

In the previous section, we examined the conditions that result in convergence of the fixed point iteration method. Even when an equation can be rewritten in the form

$$x = g(x),$$

The resulting iteration function may not produce a sequence that converges to the desired solution.

Understanding why fixed point iteration fails is just as important as understanding why it succeeds. By recognizing the warning signs of divergence and slow convergence, you can often modify the iteration function or choose a different numerical method before wasting time on an ineffective computation.

When the Derivative Is Too Large

The most common reason for failure is that the iteration function does not satisfy the condition.

If

$$|g'(x)| > 1$$

near the fixed point, errors tend to grow with each iteration instead of shrinking. Rather than moving closer to the solution, the approximations move farther away from it.

For example, suppose an approximation contains a small error. If applying the iteration function magnifies the error at each iteration, the sequence will diverge instead of converging.

A Poor Choice of Iteration Function

One of the unique features of fixed point iteration is that a single equation can often be rewritten in many different ways.

One iteration function may satisfy the convergence conditions and quickly approach the solution, while another may diverge completely.

A Poor Initial Guess

Even with a well chosen iteration function, the choice of the initial approximation can influence the outcome.

If the starting value is too far from the desired fixed point, the sequence may diverge or converge to a different fixed point,

Multiple Fixed Points

In these cases, the solution obtained by fixed point iteration often depends on the initial approximation. Two different starting values may converge to two distinct fixed points, even when the same iteration function is used.

This behavior is not necessarily a failure of the algorithm, but it demonstrates why understanding the underlying function is important.

Error Computation and Stopping Criteria

The goal of the fixed point iteration method is to compute an approximation that is sufficiently close to the exact solution. Since the exact root is usually unknown, we cannot compute the true error during the iteration process. Instead, we estimate the error by comparing successive approximations and evaluating how well the current approximation satisfies the original equation.

These error estimates also determine when to stop iterating. Continuing the iteration after the desired accuracy has been achieved wastes computational effort, while stopping too early may produce an inaccurate solution.

Absolute Error

The simplest measure of error is the absolute error between two successive approximations.

It is defined by

$$\epsilon = |x_{n + 1} – x_n|.$$

As the iteration converges, the difference between successive approximations should become smaller.

Relative Error

To account for the size of an approximation, it is often preferable to compute the relative error.

The relative error is given by

$$\epsilon = \frac{|x_{n + 1} – x_n|}{|x_{n + 1}|}.$$

Because it measures the error relative to the magnitude of the approximation, it provides a more meaningful indication of accuracy when solutions are large.

Maximum Number of Iterations

To prevent an infinite loop when the method fails to converge, it is common to specify a maximum number of iterations,

If this limit is reached before the desired accuracy is obtained, the iteration is terminated, and the convergence conditions should be reexamined.

Which Stopping Criterion Should You Use?

In practice, no single stopping criterion is appropriate for every problem. The appropriate stopping criterion depends on the accuracy required for the problem being solved.

Step-by-Step Procedure for Fixed Point Iteration

Now that we’ve discussed when fixed point iteration succeeds and why it can fail, let’s look at the procedure for applying it.

Step 1: Write the Equation in Fixed Point Form

Rewrite the equation in the form

$$x = g(x),$$

where \( g(x) \) is the iteration function.

Step 2: Verify the Convergence Conditions

Before performing any iterations, verify that the chosen iteration function is likely to converge. In particular, check that \( g(x) \) is continuous on an interval containing the fixed point, and the derivative satisfies

$$|g'(x)| < 1$$

throughout a neighborhood of the fixed point.

Step 3: Choose an Initial Approximation

Select an initial guess,

$$x_0.$$

A good starting value is typically close to the desired solution. The initial approximation may come from graphing the function, inspecting a table of values, intuition, or another numerical method.

Step 4: Compute the Next Approximation

Use the fixed point iteration formula

$$x_{n + 1} = g(x_n)$$

to compute the successive approximations until the stopping criterion is satisfied.

Step 5: Compute the Approximation Error

After each iteration, estimate the error between successive approximations.

Step 6: Check the Stopping Criterion

Continue iterating until the stopping criterion is satisfied.

In the next section, we’ll apply the fixed point iteration method to several worked examples, illustrating each step of the algorithm.

Worked Out Examples

The first example shows how to use fixed point iteration.

Example 1: Using an initial approximation of \( x_0 = 0.5 \), approximate the fixed point of \( x^3 + x- 1 = 0 \), with an absolute error less than \( 0.01 \).

Solution: Adding 1 to both sides, we get

$$x^3 + x = 1.$$

Factoring the left-hand side, we obtain

$$x(x^2 + 1) = 1.$$

Dividing both sides by \( x^2 + 1 \) we find

$$x = \frac{1}{x^2 + 1}.$$

Hence, the iteration function is

$$g(x) = \frac{1}{x^2 + 1}.$$

The derivative is given by

$$g'(x) = (\frac{1}{x^2 + 1})’.$$

Using exponential properties, this is

$$g'(x) = (x^2 + 1)^{-1}.$$

Applying the chain rule, we get

$$g'(x) = -(x^2 + 1)^{-2}(x^2 + 1)’.$$

By the sum rule, this is

$$g'(x) = -(x^2 + 1)^{-2}((x^2)’ + (1)’).$$

Using the power rule and constant rule gives

$$g'(x) = -(x^2 + 1)^{-2}(2x + 0).$$

Simplifying we find

$$g'(x) = -(x^2 + 1)^{-2}(2x).$$

Using exponential properties we arrive at

$$g'(x) = -\frac{2x}{(x^2 + 1)^2}.$$

Since g(x) is continuous and \( |g'(x)| < 1 \) near the fixed point, we expect convergence. We have

$$x_1 = \frac{1}{0.5^2 + 1}.$$

Exponentiating gives

$$x_1 = \frac{1}{0.25 + 1}.$$

Adding this is

$$x_1 = \frac{1}{1.25}.$$

Dividing we get

$$x_1 = 0.8.$$

The absolute error is

$$\epsilon = |0.8 – 0.5|.$$

Subtracting gives

$$\epsilon = |0.3|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.3.$$

Since \( 0.3 > 0.01 \) we continue iterating. The next approximation is given by

$$x_2 = \frac{1}{0.8^2 + 1}.$$

Exponentiating gives

$$x_2 = \frac{1}{0.64 + 1}.$$

Adding this is

$$x_2 = \frac{1}{1.64}.$$

Dividing we get

$$x_2 = 0.609756.$$

The absolute error is

$$\epsilon = |0.609756 – 0.8|.$$

Subtracting gives

$$\epsilon = |-0.190243|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.190243.$$

Since \( 0.190243 > 0.01 \) we continue iterating. The next approximation is given by

$$x_3 = \frac{1}{0.609756^2 + 1}.$$

Exponentiating gives

$$x_3 = \frac{1}{0.371802+ 1}.$$

Adding this is

$$x_3 = \frac{1}{1.371802}.$$

Dividing we get

$$x_3 = 0.728967.$$

The absolute error is

$$\epsilon = |0.728967 – 0.609756|.$$

Subtracting gives

$$\epsilon = |-0.119211|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.119211.$$

Since \( 0.119211 > 0.01 \) we continue iterating. The next approximation is given by

$$x_4 = \frac{1}{0.728967^2 + 1}.$$

Exponentiating gives

$$x_4 = \frac{1}{0.531394+ 1}.$$

Adding this is

$$x_4 = \frac{1}{1.531394}.$$

Dividing we get

$$x_4 = 0.652999.$$

The absolute error is

$$\epsilon = |0. 652999 – 0.728967|.$$

Subtracting gives

$$\epsilon = |-0.075968|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.075968.$$

Since \( 0.075968 > 0.01 \) we continue iterating. The next approximation is given by

$$x_5 = \frac{1}{0.652999^2 + 1}.$$

Exponentiating gives

$$x_5 = \frac{1}{0.426408+ 1}.$$

Adding this is

$$x_5 = \frac{1}{1.426408}.$$

Dividing we get

$$x_5 = 0.701061.$$

The absolute error is

$$\epsilon = |0.701061 – 0. 652999|.$$

Subtracting gives

$$\epsilon = |0.048061|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.048061.$$

Since \( 0.048061 > 0.01 \) we continue iterating. The next approximation is given by

$$x_6 = \frac{1}{0.701061^2 + 1}.$$

Exponentiating gives

$$x_6 = \frac{1}{0.491487+ 1}.$$

Adding this is

$$x_6 = \frac{1}{1.491487}.$$

Dividing we get

$$x_6 = 0.670471.$$

The absolute error is

$$\epsilon = |0.670471 – 0.701061|.$$

Subtracting gives

$$\epsilon = |-0.030589|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.030589.$$

Since \( 0.030589 > 0.01 \) we continue iterating. The next approximation is given by

$$x_7 = \frac{1}{0.670471^2 + 1}.$$

Exponentiating gives

$$x_7 = \frac{1}{0.449532+ 1}.$$

Adding this is

$$x_7 = \frac{1}{1.449532}.$$

Dividing we get

$$x_7 = 0.689877.$$

The absolute error is

$$\epsilon = |0.689877 – 0.670471|.$$

Subtracting gives

$$\epsilon = |0.019405|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.019405.$$

Since \( 0.019405 > 0.01 \) we continue iterating. The next approximation is given by

$$x_8 = \frac{1}{0.689877^2 + 1}.$$

Exponentiating gives

$$x_8 = \frac{1}{0.475931 + 1}.$$

Adding this is

$$x_8 = \frac{1}{1.475931}.$$

Dividing we get

$$x_8 = 0.677538.$$

The absolute error is

$$\epsilon = |0.677538 – 0.689877|.$$

Subtracting gives

$$\epsilon = |-0.012339|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.012339.$$

Since \( 0.012339 > 0.01 \) we continue iterating. The next approximation is given by

$$x_9 = \frac{1}{0.677538^2 + 1}.$$

Exponentiating gives

$$x_9 = \frac{1}{0.459058 + 1}.$$

Adding this is

$$x_9 = \frac{1}{1.459058}.$$

Dividing we get

$$x_9 = 0.685373.$$

The absolute error is

$$\epsilon = |0.685373 – 0.677538|.$$

Subtracting gives

$$\epsilon = |0.007835|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.007835.$$

Since \( 0.007835 < 0.01 \) our final answer is

$$x = 0.685373.$$

The next example illustrates one of the primary applications of numerical analysis, namely solving equations that cannot be solved analytically.

Example 2: Using an initial approximation of \( x_0 = 0.5 \), approximate the fixed point of \( \cos x = x \), with an absolute error less than \( 0.01 \).

Solution: The iteration function is

$$g(x) = \cos x.$$

The derivative is given by

$$g'(x) = (\cos x)’.$$

Differentiating gives

$$g'(x) = -\sin x.$$

Since \( g(x) \) is continuous and \( |g'(x)| < 1 \) near the fixed point, we expect convergence. We have

$$x_1 = \cos 0.5.$$

Evaluating cosine gives

$$x_1 = 0.877582.$$

The absolute error is

$$\epsilon = |0.877582 – 0.5|.$$

Subtracting gives

$$\epsilon = |0.377582|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.377582.$$

Since \( 0.377582 > 0.01 \) we continue iterating. The next approximation is given by

$$x_2 = \cos 0.877582.$$

Evaluating cosine gives

$$x_2 = 0.639012.$$

The absolute error is

$$\epsilon = |0.639012 – 0.877582|.$$

Subtracting gives

$$\epsilon = |-0.238570|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.238570.$$

Since \( 0.238570 > 0.01 \) we continue iterating. The next approximation is given by

$$x_3 = \cos 0.639012.$$

Evaluating cosine gives

$$x_3 = 0.802685.$$

The absolute error is

$$\epsilon = |0.802685 – 0.639012|.$$

Subtracting gives

$$\epsilon = |0.163672|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.163672.$$

Since \( 0.163672 > 0.01 \) we continue iterating. The next approximation is given by

$$x_4 = \cos 0.802685.$$

Evaluating cosine gives

$$x_4 = 0.694778.$$

The absolute error is

$$\epsilon = |0.694778 – 0.802685|.$$

Subtracting gives

$$\epsilon = |-0.107907|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.107907.$$

Since \( 0.107907 > 0.01 \) we continue iterating. The next approximation is given by

$$x_5 = \cos 0.694778.$$

Evaluating cosine gives

$$x_5 = 0.768195.$$

The absolute error is

$$\epsilon = |0.768195 – 0.694778|.$$

Subtracting gives

$$\epsilon = |0.073417|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.073417.$$

Since \( 0.073417 > 0.01 \) we continue iterating. The next approximation is given by

$$x_6 = \cos 0.768195.$$

Evaluating cosine gives

$$x_6 = 0.719165.$$

The absolute error is

$$\epsilon = |0.719165 – 0.768195|.$$

Subtracting gives

$$\epsilon = |-0.049030|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.049030.$$

Since \( 0.049030 > 0.01 \) we continue iterating. The next approximation is given by

$$x_7 = \cos 0.719165.$$

Evaluating cosine gives

$$x_7 = 0.752355.$$

The absolute error is

$$\epsilon = |0.752355 – 0.719165|.$$

Subtracting gives

$$\epsilon = |0.033190|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.033190.$$

Since \( 0.033190 > 0.01 \) we continue iterating. The next approximation is given by

$$x_8 = \cos 0.752355.$$

Evaluating cosine gives

$$x_8 = 0.730081.$$

The absolute error is

$$\epsilon = |0.730081 – 0.752355|.$$

Subtracting gives

$$\epsilon = |-0.022274|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.022274.$$

Since \( 0.022274 > 0.01 \) we continue iterating. The next approximation is given by

$$x_9 = \cos 0.730081.$$

Evaluating cosine gives

$$x_9 = 0.745120.$$

The absolute error is

$$\epsilon = |0.745120 – 0.730081|.$$

Subtracting gives

$$\epsilon = |0.015039|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.015039.$$

Since \( 0.015039 > 0.01 \) we continue iterating. The next approximation is given by

$$x_{10} = \cos 0.745120.$$

Evaluating cosine gives

$$x_{10} = 0.735006.$$

The absolute error is

$$\epsilon = |0.735006 – 0.745120|.$$

Subtracting gives

$$\epsilon = |-0.010114|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.010114.$$

Since \( 0.010114 > 0.01 \) we continue iterating. The next approximation is given by

$$x_{11} = \cos 0.735006.$$

Evaluating cosine gives

$$x_{11} = 0.741826.$$

The absolute error is

$$\epsilon = |0.741826 – 0.735006|.$$

Subtracting gives

$$\epsilon = |0.006820|.$$

Taking the absolute value, we obtain

$$\epsilon = 0.006820.$$

Since \( 0.006820 < 0.01 \) our final answer is

$$x = 0.741826.$$

Conclusion

Understanding how to solve equations using fixed point iteration is an important step toward understanding root finding. By rewriting an equation in the form \( x = g(x) \) and repeatedly applying the iteration function, you can obtain increasingly accurate approximations of a solution when the appropriate convergence conditions are satisfied.

Throughout this guide, we’ve explored the mathematical foundation of fixed point iteration, examined why it converges for some iteration functions and fails for others, and discussed the importance of selecting a suitable initial approximation. We’ve also covered common error measures, stopping criteria, and provided worked examples of the algorithm.

Further Reading

Secant and Tangent Lines in Calculus: A Complete Beginner’s Guide – Other commonly used root-finding methods include the Secant Method and Newton’s Method. Both of these methods make use of secant and tangent lines.

Frequently Asked Questions