Day 13 in Mat360

Today:

• Announcements:
  • Exam 1
  • Exam 1 key (Mathematica)
  • You've got a homework due. I've offered one student the chance to turn it in later today, so everyone gets that chance. If you need more time, take it. Get it to me (e.g. under my door) for when I arrive tomorrow.
  • We're going to be moving in to Interpolation (chapter 5) next time. Please read ahead!

• Let's wrap up root-finding:
  Back to Tyler's question, in a root-finding context:
  how do we know which function to choose when root finding?

  My answer begins with the word "derivatives", and by generalizing Newton's method: it is an example of a process called "fixed point iteration".

  We'll pick up where we left off last time. Our objective today is to finish up chapter 3, by getting through the Secant method.

  Fixed Point Iteration

  Now here is how a problem looks using Newton's method:

  Newton's method as Fixed Point Iteration

  • Let's talk about order of convergence (p. 86).

    Definition 3.3 (p. 87): If a method converges to root , and if

    

    then the method is convergent of order , with asymptotic error constant .

    We can show that Newton's convergence is quadratic (with Taylor polynomials).

    Question: What would you expect for bisection? You might think that it is linear, but interesting things can happen....

    Let's

    • look at the convergence of a general fixed point method. This will answer Tyler's question about "which function to choose".
    • See again why Newton's method is much better, in general.

    Examples:

    • Newton's method showing off its better fixed-point credentials
    • Consider the fixed-point approach to finding a root of . We could consider these function (verify that they're both fixed-point iteration functions that solve this problem):
      • 
      • 
      • 
      • for non-zero .
      Which will converge fastest? We just look at the derivatives of these fixed point functions at the root (which, as one can see from the quadratic, is x=2).

      The rule is that the general fixed-point method has order of convergence 1 (linear), with asymptotic error constant (where is the root).

    You might think that bisection is linear -- we've talked about how you get the root narrowed down to one additional bit with each iteration of bisection. However we can show that bisection is not necessarily linear.

    Nonetheless, one can say that we narrow down the interval in which a root is found by one binary digit (that is, "linearly" -- the ratio of interval widths is 1/2 --

    with each iteration. This is the message of homework problem p. 80, Exercise #1.

• Secant Method

  Finally we're getting to the secant method, and we'll then take it one step further and consider Muller's method (based on quadratic functions -- which I've asked you to derive and demonstrate as a homework assignment, due next Thursday).

  Secant is Newton's method, where we approximate the derivative: in particular, we approximate the tangent line with a secant line, using a finite difference approximation to the derivative.

  • Let's see if we can derive the formula, using what is called the first divided difference.

    One thing that we notice right away is that this method requires two approximations to start -- to prime the pump.

  • Now let's check out the order of convergence:
    where (the golden ratio!).

    To demonstrate that, I won't start from scratch, but from a result which is sensible (at any rate):

    

    and

    

    It turns out that the Secant method may be preferable to Newton's, because

    1. it doesn't require derivative calculations, and
    2. it requires fewer function evaluations.

    So even though the order of convergence is less, the routine may run in about the same time as Newton's (depends on the problem, of course).