The Newton-Raphson Method

Contents

The Newton-Raphson Method#

An alternative method for finding the minimum of a function is the Newton-Raphson Method (also called Newton’s Method).

The key insight is that at a minimum, the first derivative equals zero: \(f^\prime(x) = 0\). So finding a minimum is equivalent to finding where the derivative vanishes.

The Newton-Raphson method uses the idea that if we zoom in on any function, we can approximate the local shape of that function as quadratic, i.e.

\[f(x) \approx ax^2 + bx + c\]

over some small range of \(x\).

If \(f(x)\) is approximately quadratic near our current position, then its first derivative is approximately linear:

\[f^\prime(x) \approx 2ax + b\]

If we know the coefficients \(a\) and \(b\), we can directly solve for where this linear approximation crosses zero (i.e., where \(f^\prime(x) = 0\)):

\[2ax + b = 0\]
\[x = \frac{-b}{2a}\]

This gives us our predicted location of the minimum.

To find the coefficients \(a\) and \(b\) of our local quadratic approximation, we use the property that any smooth function can be expanded as a Taylor series (see, e.g., Chapter 6 in Foundations of Science Mathematics).

If we are at some point \(x_0\), then we can approximate our function locally as a polynomial in \((x-x_0)\):

\[f(x) = f(x_0) + f^\prime(x_0)(x-x_0) + \frac{1}{2}f^{\prime\prime}(x_0)(x-x_0)^2 + \ldots\]

If we remain close to \(x_0\), where the higher-order terms in \((x-x_0)\) are small, we can truncate this series after the quadratic term:

\[f(x) \approx f(x_0) + f^\prime(x_0)(x-x_0) + \frac{1}{2}f^{\prime\prime}(x_0)(x-x_0)^2\]

This is our quadratic approximation. Differentiating this expression with respect to \(x\) gives us the approximation for the first derivative:

\[f^\prime(x) \approx f^\prime(x_0) + f^{\prime\prime}(x_0)(x - x_0)\]

At a minimum, \(f^\prime(x) = 0\), so we can set this expression to zero and solve for \(x\):

\[f^\prime(x_0) + f^{\prime\prime}(x_0)(x - x_0) = 0\]
\[x = x_0 - \frac{f^\prime(x_0)}{f^{\prime\prime}(x_0)}\]

This is the Newton-Raphson equation. Starting from our current position \(x_0\), it predicts a new position \(x\) that should be closer to the minimum.

In the context of finding the minimum of a potential energy surface, we can rewrite this using our usual notation for energy and bond length:

  1. Starting at an initial guess \(r_n\), calculate the first and second derivatives of \(U(r_n)\) at that point.

  2. Use the Newton-Raphson equation to predict a new position:

\[r_{n+1} = r_n - \frac{U^\prime(r_n)}{U^{\prime\prime}(r_n)}\]

In reality, our potential energy surface is never exactly quadratic (except for the special case of a harmonic potential), so this equation only gives us an approximate guess for the minimum position. We therefore repeat steps 1 and 2 as an iterative process, recalculating the derivatives at each new position, until we (hopefully) converge on the energy minimum.

Exercise#

In this exercise you will explore how the Newton-Raphson method performs on a harmonic potential, where we know the analytical solution is \(r = r_0 = 0.74\) Å.

Part 1: Define the second derivative

  1. Write a function to calculate the second derivative of a harmonic potential:

\[U^{\prime\prime}(r) = k\]

Note that for a harmonic potential, the second derivative is simply the force constant \(k\) and does not depend on \(r\).

Part 2: Demonstrate one-step convergence

  1. Write code to perform Newton-Raphson optimisation on your harmonic potential starting from \(r = 1.0\) Å. Your code should:

    • Calculate \(U'(r)\) and \(U''(r)\) at the starting position

    • Apply the Newton-Raphson equation once: \(r_1 = r_0 - \frac{U'(r_0)}{U''(r_0)}\)

    • Print both the starting position and the position after one step

    • Compare the result to the analytical solution (\(r_0 = 0.74\) Å)

  2. Repeat this calculation for at least three different starting positions:

    • \(r = 0.5\) Å (starting below the minimum)

    • \(r = 1.0\) Å (starting above the minimum)

    • \(r = 1.5\) Å (starting well above the minimum)

Questions to consider:

  • Does each starting point converge to exactly \(r = 0.74\) Å in one step?

  • Why does the Newton-Raphson method work perfectly for a harmonic potential? (Hint: Consider what happens when you substitute the derivatives of a quadratic function into the Newton-Raphson equation. The method assumes the function is locally quadratic—what happens when the function is exactly quadratic everywhere?)

  • Will this one-step convergence property hold for other potential energy functions like the Lennard-Jones potential? Why or why not?

Contents