Newton’s Method Explained for Beginners

Why Newton’s Method Works

One reason Newton’s method is so popular is that it can converge quickly when the appropriate conditions are satisfied. Unlike root-finding algorithms that only use function values, Newton’s method also incorporates information from the function’s derivative. This additional information allows the method to predict where the root lies much more accurately, often requiring only a handful of iterations to achieve high precision.

To understand why Newton’s method works, it is helpful to examine the geometric idea behind the algorithm and the mathematical conditions that guarantee convergence.

Local Linear Approximation

The idea behind Newton’s method is that a smooth function behaves like its tangent line over a small interval. Suppose we have a differentiable function \( f(x) \) and a current approximation \( x_n \) that is reasonably close to the actual root.

Instead of solving the original equation

$$f(x) = 0,$$

Newton’s method replaces the curve with its tangent line at the point \( (x_n, f(x_n)) \). Since a line is much easier to work with than a nonlinear function, we find where this tangent line crosses the \( x \)-axis and use that point as the next approximation.

If the current approximation is already close to the true root, the tangent line provides an excellent approximation to the curve, so the next approximation is usually much closer to the solution.

This process is repeated until the approximations no longer change significantly.

Convergence Near the Root

Newton’s method is a local method, meaning that its convergence depends on the initial guess. If the starting value \( x_0 \) is sufficiently close to a simple root and the function is smooth, the sequence

$$x_0, x_1, x_2, \ldots$$

typically converges to the desired solution.

More specifically, Newton’s method performs well when:

• The function is differentiable near the root,
• The derivative is continuous,
• The derivative is not zero at the root.
• The initial guess is close to the solution.

When these conditions hold, each iteration results in a better approximation.

On the other hand, if the initial guess is too far from the root or the derivative behaves poorly, Newton’s method may converge slowly, converge to a different root, or fail to converge altogether. We’ll examine these situations later in this article.

Quadratic Convergence

One of the greatest strengths of Newton’s method is its quadratic convergence. Informally, quadratic convergence means that once the approximations are sufficiently close to the root, the number of correct digits roughly doubles with each iteration. Although the exact improvement depends on the function being solved, this rapid reduction in error explains why Newton’s method is considered one of the fastest root-finding algorithms.

Mathematically, if \( r \) denotes the root and

$$\epsilon_n = |x_n – r|$$

is the error after the \( n \)-th iteration, then for a simple root Newton’s method satisfies the relationship

$$\epsilon_{n+1} \approx C \epsilon_n^2,$$

where \( C \) is a constant that depends on the function. Since the error is approximately proportional to the square of the previous error, it decreases rapidly once the approximations enter the neighborhood of the root.

This quadratic convergence is the primary reason Newton’s method is often preferred over methods such as the bisection method and fixed-point iteration. While those methods are generally more robust, they usually converge only linearly, requiring many more iterations to achieve the same level of accuracy.

When Newton’s Method Succeeds

Newton’s method is one of the fastest algorithms for solving equations numerically. However, its performance depends on certain mathematical conditions being satisfied. When these conditions hold, the method typically converges rapidly to the desired root, often requiring only a few iterations to achieve high accuracy.

In this section, we’ll examine the situations in which Newton’s method performs well and explain why these conditions are necessary.

The Initial Guess Is Close to the Root

If the starting approximation \( x_0 \) is sufficiently close to the actual root, the tangent line at \( x_0 \) usually provides an excellent approximation to the function. As a result, the next approximation moves significantly closer to the solution.

Once the approximations enter the neighborhood of the root, Newton’s method exhibits quadratic convergence, causing the error to decrease rapidly.

In practice, choosing a good initial guess can make the difference between rapid convergence and failure.

The Derivative Is Continuous

Although Newton’s method only explicitly requires evaluating the derivative, the method generally performs best when the derivative is continuous in a neighborhood of the root.

A continuous derivative ensures that the slope of the tangent line changes smoothly as the iterations progress. This stability improves the quality of the tangent-line approximation and increases the likelihood of convergence.

For a review of continuity, please refer to the article A Complete Introduction to the Bisection Method with Worked Examples and Solutions.

The Derivative Is Nonzero at the Root

Another important condition is that the derivative should not vanish at the root. In other words,

$$f'(r) \neq 0,$$

where \( r \) denotes the true root.

When the derivative is nonzero, the tangent line intersects the \( x \)-axis at a well-defined angle, allowing Newton’s method to generate meaningful correction steps.

If the derivative is zero or close to zero, the correction term

$$\frac{f(x_n)}{f'(x_n)}$$

may become very large, causing the iterations to behave erratically or converge slowly.

The Root Is a Simple Root

Newton’s method performs best when the equation has a simple root, meaning the root has multiplicity one.

If the equation has a multiple root, the algorithm often converges much more slowly, and modifications to the algorithm may be necessary to restore fast convergence.

The Function Behaves Smoothly Near the Root

Newton’s method assumes that the tangent line provides a good local approximation to the function.

This assumption is generally valid when the function is smooth and does not change direction too abruptly near the solution. Smooth functions allow each tangent line to closely resemble the curve in the neighborhood of the current approximation, producing increasingly accurate estimates.

The Desired Root Is Isolated

Newton’s method is most successful when the root being sought is isolated from other nearby roots.

When several roots lie very close together, the tangent line may steer the iterations toward a different root than the one originally intended. An isolated root reduces this possibility and makes the convergence behavior more predictable.

When these conditions are met, Newton’s method is efficient. In the next section, we’ll examine situations in which these assumptions break down and see why Newton’s method can sometimes fail to converge.

When Newton’s Method Fails

Although Newton’s method is one of the most powerful root-finding algorithms, it is not guaranteed to converge for every problem. Unlike the bisection method, which converges whenever its assumptions are satisfied, Newton’s method depends heavily on the behavior of the function and the choice of the initial guess.

When the assumptions discussed in the previous section are violated, the method may converge slowly, converge to an unintended root, oscillate between points, or fail to converge altogether. Understanding the reasons for failure is just as important as understanding why the method succeeds.

Poor Initial Guess

Newton’s method is a local algorithm, meaning it works best when the starting approximation is already close to the desired root. If the initial guess is too far away, the tangent line may point toward a completely different part of the graph.

As a result, the iterations may converge to a different root or diverge without approaching any solution.

Whenever possible, it is a good idea to use graphical methods or another root-finding algorithm, such as the bisection method, to obtain a reasonable starting approximation before applying Newton’s method.

The Derivative Is Zero

The iteration formula is

$$x_{n + 1} = x_n – \frac{f(x_n)}{f'(x_n)}.$$

If

$$f'(x_n) = 0,$$

the denominator becomes zero, making the approximation undefined.

Geometrically, this occurs because the tangent line is horizontal and never intersects the \( x \)-axis. Since Newton’s method relies on the \( x \)-intercept of the tangent line, no new approximation can be computed.

For this reason, it is important to avoid starting values where the derivative is zero.

The Derivative Is Nearly Zero

Even when the derivative is not exactly zero, it may be extremely small.

In this case, the correction term

$$\frac{f(x_n)}{f'(x_n)}$$

can become very large, causing the next approximation to jump far from the current estimate.

Oscillation

In some cases, Newton’s method alternates between two or more approximations without ever converging.

This phenomenon is known as oscillation.

Rather than approaching the root, the sequence repeatedly revisits the same values, preventing the algorithm from making progress.

Oscillation is more likely when the initial guess is poorly chosen or when the function has complicated local behavior.

Divergence

Instead of approaching a root, the approximations may move farther away with each iteration.

This behavior is called divergence.

When divergence occurs, the magnitude of the approximations often increases without bound, indicating that Newton’s method is unsuitable for the chosen starting value or problem.

Multiple Roots

Newton’s method converges most rapidly for simple roots.

If the equation has a multiple root, the method generally loses its quadratic convergence and instead converges much more slowly.

Although Newton’s method may still converge, the improvement from one iteration to the next is much smaller than expected.

Nondifferentiable Functions

Newton’s method requires evaluating the derivative at every iteration.

If the function is not differentiable at or near the current approximation, the tangent line may not exist.

Examples include functions with corners, cusps, vertical tangents, or discontinuities.

For these functions, Newton’s method may not be applicable, and another numerical method should be used instead.

Inflection Points and Rapid Changes in Curvature

Even when a function is differentiable, Newton’s method may struggle if the curvature changes rapidly.

Near an inflection point, the tangent line may provide a poor approximation of the function, causing the next approximation to overshoot the desired root.

Similarly, functions with sharp bends or rapidly changing slopes may produce erratic iteration sequences.

For a review of concavity and points of inflection, please refer to the article Concavity of Functions for Beginners: Everything You Need to Know.

Fortunately, many of these issues can be avoided by selecting a good initial approximation, examining the graph of the function beforehand, and verifying that the function satisfies the assumptions required for Newton’s method.

Although Newton’s method is not universally reliable, it remains one of the most effective numerical algorithms available. When used under appropriate conditions, its rapid quadratic convergence far outweighs its potential drawbacks.

Error Computation

When applying Newton’s method, it is important to measure how close the current approximation is to the root. This process is known as error computation. By monitoring the error after each iteration, we can determine whether the approximations are good enough and decide when to stop iterating.

In practice, the exact root is often unknown, so several different error measures are commonly used. Each provides useful information about the accuracy of the approximation.

Absolute Error

If the exact root \( r \) is known, the absolute error after the \( n \)-th iteration is defined as

$$\epsilon = |x_n – r|,$$

Where \( x_n \) is the current approximation, and \( r \) is the exact root.

The absolute error measures the distance between the approximation and the true solution.

Relative Error

Although the absolute error is useful, it does not account for the magnitude of the true solution. For this reason, many numerical algorithms use the relative error, defined by

$$\epsilon = \frac{|x_n – r|}{|r|},$$

provided that \( r \neq 0 \).

The relative error expresses the error as a fraction of the exact value, making it easier to compare errors for problems involving roots of different sizes.

For large roots, a relatively large absolute error may still correspond to a very small relative error.

Approximate Relative Error

In most problems, the exact root is unknown. Since the absolute and relative errors cannot be computed directly, we instead compare two successive approximations.

The approximate relative error is

$$\epsilon = |\frac{x_{n + 1} – x_n}{x_{n + 1}}|.$$

Many textbooks multiply this quantity by \( 100\% \) to express it as a percentage:

$$\epsilon = |\frac{x_{n + 1} – x_n}{x_{n + 1}}| \times 100\%.$$

Error Behavior in Newton’s Method

One reason Newton’s method is so effective is that its error decreases very rapidly once the approximations are sufficiently close to a simple root.

If

$$\epsilon_n = |x_n – r|$$

denotes the error after the \( n \)-th iteration, then Newton’s method satisfies the relationship

$$\epsilon_{n + 1} \approx C\epsilon_n^2,$$

where \( C \) is a constant that depends on the function.

This equation illustrates the method’s quadratic convergence. Since each new error is approximately proportional to the square of the previous error, the error shrinks dramatically with each iteration. As a result, Newton’s method often reaches the desired precision in only a few iterations.

Stopping Criteria

Newton’s method is an iterative algorithm, meaning that it repeatedly computes better approximations to the root of an equation. In theory, we could continue iterating indefinitely, but in practice this is neither necessary nor efficient. Instead, we stop the algorithm once the approximations are considered sufficiently accurate.

The rules used to determine when to terminate the iteration are called stopping criteria. Choosing appropriate stopping criteria is necessary because stopping too early may produce an inaccurate solution, while stopping too late wastes computational resources.

Several stopping criteria are commonly used.

Maximum Number of Iterations

The simplest stopping criterion is to specify a maximum number of iterations.

Suppose we choose a maximum iteration count of \( N \). If Newton’s method has not converged after \( N \) iterations, the algorithm terminates automatically.

This criterion prevents the algorithm from running indefinitely if it fails to converge.

Since Newton’s method usually converges very quickly when it succeeds, reaching the maximum iteration count often indicates that something has gone wrong, such as a poor initial guess or an unsuitable function.

Absolute Error Tolerance

If the exact root \( r \) is known, the iteration can be stopped once the absolute error satisfies

$$|x_n – r| < \epsilon,$$

where \( \epsilon \) is a prescribed tolerance.

Although this criterion is conceptually simple, it is rarely used in practical applications because the exact root is usually unknown.

Relative Error Tolerance

A more common stopping criterion compares successive approximations rather than comparing the approximation with the exact root.

The approximate relative error is

$$|\frac{x_{n + 1} – x_n}{x_{n + 1}}|.$$

The iterations stop once

$$|\frac{x_{n + 1} – x_n}{x_{n + 1}}| < \epsilon.$$

This criterion is widely used because it requires only the computed approximations and does not depend on knowing the exact solution.

As Newton’s method converges, successive approximations become nearly identical, causing the approximate relative error to approach zero.

Step-by-Step Procedure

Newton’s method follows a simple sequence of calculations that is repeated until an acceptable approximation to the root is obtained. Each iteration uses the current approximation, together with the value of the function and its derivative, to produce a more accurate estimate.

The following steps summarize the algorithm.

Step 1: Choose an Initial Guess

Begin by selecting an initial approximation, denoted by

$$x_0.$$

The initial guess should be as close as possible to the desired root.

Graphs, tables of function values, or another root-finding algorithm such as the bisection method can help identify a suitable initial guess.

Step 2: Evaluate the Function

Compute the value of the function at the current approximation:

$$f(x_n).$$

This value indicates how close the current approximation is to satisfying the equation

$$f(x) = 0.$$

If \( f(x_n) \) is already very close to zero, the current approximation is likely close to the root.

Step 3: Evaluate the Derivative

Next, compute the derivative at the current approximation:

$$f'(x_n).$$

The derivative represents the slope of the tangent line at the point

$$(x_n, f(x_n)).$$

Before proceeding, verify that

$$f'(x_n) \neq 0.$$

If the derivative is zero, Newton’s method cannot be used because division by zero is undefined.

Step 4: Compute the Next Approximation

Use the iteration formula

$$x_{n + 1} = x_n – \frac{f(x_n)}{f'(x_n)}$$

to compute the next approximation.

This formula replaces the function with its tangent line and uses the tangent line’s \( x \)-intercept as the improved estimate of the root.

Step 5: Compute the Approximation Error

After obtaining the new approximation, estimate the error.

If the exact root is unknown, a common choice is the approximate relative error.

Step 6: Check the Stopping Criterion

Determine whether the algorithm should terminate.

Step 7: Repeat the Process

If the stopping criterion has not been satisfied, replace the current approximation with the newly computed approximation and repeat Steps 2 through 6 until the desired level of accuracy is achieved.

Worked Out Examples

Our first example uses Newton’s method to approximate \( \sqrt{2} \).

Example 1: Use Newton’s method to approximate \( \sqrt{2} \) accurate to six decimal places.

Solution: \( \sqrt{2} \) is a root of

$$f(x) = x^2 – 2.$$

Evaluating the function at \( x_0 = 2 \) we obtain

$$f(2) = 2^2 – 2.$$

Squaring we find

$$f(2) = 4 – 2.$$

Subtracting gives

$$f(2) = 2.$$

The derivative is given by

$$f'(x) = (x^2 – 2)’.$$

By the sum rule, this is

$$f'(x) = (x^2)’ – (2)’.$$

Using the power rule and constant rule gives

$$f'(x) = 2x + 0.$$

Simplifying we find

$$f'(x) = 2x.$$

Evaluating the derivative at \( x_0 = 2 \) we obtain

$$f'(2) = 2(2).$$

Multiplying gives

$$f'(2) = 4.$$

The next approximation is

$$x_1 = 2 – \frac{2}{4}.$$

Dividing we get

$$x_1 = 2 – 0.5.$$

Subtracting gives

$$x_1 = 1.5.$$

Evaluating the function at \( x_1 = 1.5 \) we obtain

$$f(1.5) = 1.5^2 – 2.$$

Squaring we find

$$f(1.5) = 2.25 – 2.$$

Subtracting gives

$$f(1.5) = 0.25.$$

Evaluating the derivative at \( x_1 = 1.5 \) we obtain

$$f'(1.5) = 2(1.5).$$

Multiplying gives

$$f'(1.5) = 3.$$

The next approximation is

$$x_2 = 1.5 – \frac{0.25}{3}.$$

Dividing we get

$$x_2 = 1.5 – 0.083333.$$

Subtracting gives

$$x_2 = 1.416666.$$

Evaluating the function at \( x_2 = 1.416666 \) we obtain

$$f(1.416666) = 1.416666^2 – 2.$$

Squaring we find

$$f(1.416666) = 2.006944 – 2.$$

Subtracting gives

$$f(1.416666) = 0.0069444.$$

Evaluating the derivative at \( x_2 = 1.416666 \) we obtain

$$f'(1.416666) = 2(1.416666).$$

Multiplying gives

$$f'(1.416666) = 2.8333333.$$

The next approximation is

$$x_3 = 1.416666 – \frac{0.006944}{2.8333333}.$$

Dividing we get

$$x_3 = 1.416666 – 0.002451.$$

Subtracting gives

$$x_3 = 1.414215.$$

Evaluating the function at \( x_3 = 1.414215 \) we obtain

$$f(1.414215) = 1.414215^2 – 2.$$

Squaring we find

$$f(1.414215) = 2.000006 – 2.$$

Subtracting gives

$$f(1.414215) = 0.000006.$$

Evaluating the derivative at \( x_3 = 1.414215 \) we obtain

$$f'(1.414215) = 2(1.414215).$$

Multiplying gives

$$f'(1.414215) = 2.828431.$$

The next approximation is

$$x_4 = 1.414215 – \frac{0.000006}{2.8284314}.$$

Dividing we get

$$x_4 = 1.414215 – 0.000002.$$

Subtracting gives

$$x_4 = 1.414213.$$

\( \sqrt{2} = 1.414213 \), so this is our final iteration.

The next example shows how to use Newton’s method to solve polynomial equations.

Example 2: Using an initial approximation of \( x_0 = 1.5 \), approximate the root of \( x^3 – x – 2 = 0 \), Use 2 iterations of Newton’s method.

Solution: Consider the function

$$f(x) = x^3 – x – 2.$$

Evaluating the function at \( x_0 = 1.5 \) we obtain

$$f(1.5) = 1.5^3 – 1.5 – 2.$$

Cubing we find

$$f(1.5) = 3.375 – 1.5 – 2.$$

Subtracting gives

$$f(1.5) = -0.125.$$

The derivative is given by

$$f'(x) = (x^3 – x – 2)’.$$

By the sum rule, this is

$$f'(x) = (x^3)’ – (x)’ – (2)’.$$

Using the power rule and constant rule gives

$$f'(x) = 3x^2 – 1 + 0.$$

Simplifying we find

$$f'(x) = 3x^2 – 1.$$

Evaluating the derivative at \( x_0 = 1.5 \) we obtain

$$f'(1.5) = 3(1.5)^2 – 1.$$

Squaring we find

$$f'(1.5) = 3(2.25) – 1.$$

Multiplying we get

$$f'(1.5) = 6.75 – 1.$$

Subtracting gives

$$f'(1.5) = 5.75.$$

The next approximation is

$$x_1 = 1.5 – \frac{-0.125}{5.75}.$$

Dividing we get

$$x_1 = 1.5 – (-0.021739).$$

Subtracting gives

$$x_1 = 1.521739.$$

Evaluating the function at \( x_1 = 1.521739 \) we obtain

$$f'(1.521739) = 1.521739^3 – 1.521739 – 2.$$

Cubing we find

$$f'(1.521739) = 3.523876 – 1.521739 – 2.$$

Subtracting gives

$$f'(1.521739) = 0.002136.$$

Evaluating the derivative at \( x_1 = 1.521739 \) we obtain

$$f'(1.521739) = 3(1.521739)^2 – 1.$$

Squaring we find

$$f'(1.521739) = 3(2.315690) – 1.$$

Multiplying we get

$$f'(1.521739) = 6.947069 – 1.$$

Subtracting gives

$$f'(1.521739) = 5.947069.$$

The next approximation is

$$x_2 = 1.521739 – \frac{0.002136}{5.947069}.$$

Dividing we get

$$x_2 = 1.521739 – 0.000359.$$

Subtracting gives a final answer of

$$x_2 = 1.521379.$$

The last example illustrates one of the primary applications of numerical analysis, namely solving equations that cannot be solved analytically.

Example 3: Using an initial approximation of \( x_0 = 0.7 \), approximate the root of \( \cos x = x \). Use 2 iterations of Newton’s method.

Solution: Subtracting x from both sides, we get

$$\cos x – x = 0.$$

Consider the function

$$f(x) = \cos x – x.$$

Evaluating the function at \( x_0 = 0.7 \) we obtain

$$f(0.7) = \cos 0.7 – 0.7.$$

Evaluating cosine, we find

$$f(0.7) = 0.764842 – 0.7.$$

Subtracting gives

$$f(0.7) = 0.064842.$$

The derivative is given by

$$f'(x) = (\cos x – x)’.$$

By the sum rule, this is

$$f'(x) = (\cos x)’ – (x)’.$$

Taking the derivatives gives

$$f'(x) = -\sin x – 1.$$

Evaluating the derivative at \( x_0 = 0.7 \) we obtain

$$f'(0.7) = -\sin 0.7 – 1.$$

Evaluating sine, we find

$$f'(0.7) = -0.644217 – 1.$$

Subtracting gives

$$f'(0.7) = -1.644217.$$

The next approximation is

$$x_1 = 0.7 – \frac{0.064842}{-1.644217}.$$

Dividing we get

$$x_1 = 0.7 – (-0.039436).$$

Subtracting gives

$$x_1 = 0.739436.$$

Evaluating the function at \( x_1 = 0.739436 \) we obtain

$$f(0.739436) = \cos 0.739436 – 0.739436.$$

Evaluating cosine, we find

$$f(0.739436) = 0.738848 – 0.739436.$$

Subtracting gives

$$f(0.739436) = -0.000588.$$

Evaluating the derivative at \( x_1 = 0.739436 \) we obtain

$$f'(0.739436) = -\sin 0.739436 – 1.$$

Evaluating sine, we find

$$f'(0.739436) = -0.673871 – 1.$$

Subtracting gives

$$f'(0.739436) = -1.673871.$$

The next approximation is

$$x_2 = 0.739436 – \frac{-0.000588}{-1.673871}.$$

Dividing we get

$$x_2 = 0.739436 – 0.000351.$$

Subtracting gives a final answer of

$$x_2 = 0.739085.$$

Conclusion

In this guide, we saw Newton’s method explained from both a theoretical and practical perspective. We began by introducing the purpose of the algorithm and developing its geometric interpretation using tangent lines. We then derived the Newton-Raphson formula, presented a step-by-step procedure for applying the method, and worked through several examples involving polynomial and transcendental equations. Along the way, we examined why Newton’s method converges so rapidly, identified the conditions under which it succeeds, discussed situations in which it can fail, and explained how to compute errors and choose appropriate stopping criteria.

Newton’s method remains one of the most used algorithms in numerical analysis because of its efficiency. When the initial guess is sufficiently close to a simple root, and the necessary mathematical assumptions are satisfied, the method exhibits quadratic convergence, allowing it to produce highly accurate approximations in only a few iterations. At the same time, understanding its limitations is necessary for applying the algorithm successfully.

Further Reading

How to Find Intervals of Increase and Decrease: A Step-by-Step Guide  – Earlier, we mentioned that Newton’s method fails when the derivative is zero. This guide explains how functions behave when the derivative is zero.

Frequently Asked Questions