Taylor series, as a procedure

solution set ft. Patrick's brother, Auntie Bonnie's Chex mix, and a kinda bogus reference to the Curry-Howard Correspondence

Jaxon Jacobs Memorial Look-Mom-No-Formula DIY Taylor Series: On the Friday before Thanksgiving, I said that if we integrate a function using integration by parts, making the stupidest possible choices at each juncture, we get a Taylor series!

Jaxon interrupted: "How does that work for, like, sine?" "Let's do the proof!," I said. "I don't get how that would work for sine,"Jaxon said. "Be patient! You'll see!"

So we did the proof. We stupidly-integrated-by-parts an arbitrary function (e.g., $f(x)$ rather than $x^2$) and as a result got the general formula for a Taylor series (rather than the specific formula for the particular Taylor series of a particular function).

Jaxon said, "Yeah, this proof is nice, but I still don't see how if you integrate sine by parts, you get its Taylor series."

And... I really like Jaxon's point. One response (which I gave) is, "we proved it for an arbitrary function; if you want to know the Taylor series for sine, just plug sine into this formula we derived." But I feel that misses the thrust of Jaxon's question, which was the distinction between a formula and a procedure.

So... maybe I'll have a bit more to say later, but, with all this in mind, compute the Taylor series of $\sin(x)$ using the procedure we did in class that day (my more-coherent writeup of it is in the notes). You know what the answer should be---you plugged $\sin(x)$ into the Taylor series formula to get it a few problem sets ago; you did it last year, etc. So, show, as a procedure, and not just by plugging it into the formula, how, if we integrate $\sin(x)$ by parts an infinite number of times, making the stupidest possible choice at every decision point, we get its Taylor series.

(I worked this out on my kitchen whiteboard the morning of Thanksgiving as I was cooking!)

OK. Let me give a bit more context here.

On the day we derived/proved Taylor series in 1D, I said that Taylor series is really just integration by parts. If we take a function and integrate it by parts (writing it as $\int 1\cdot\text{(the fxn)}$), and make the stupidest possible choice for which part is the $f'$ and which part is the $g$, and keep doing that ad infinitum, we get a Taylor series!

I meant this as an outline to how we'd prove the general formula for a Taylor series. But in Block 7, Jaxon asked how it'd work for a basic function like $\sin x$. "Like, specifically, how does that work???" I said something like, well, let's do the proof; then we'll see. So we did the very nice proof, but at the end, Jaxon was still insistent: "How does this work for something like $\sin x$? Can we really just integrate that by parts and get its Taylor series?!?"

And... I think this brought up a deep point. Every formula is really just a procedure. If we do a procedure on more general objects (e.g., letters like $x$ or $y$ instead of numbers like $5$ and $7$; arbitrary functions like $f(x)$ or $g(x)$ instead of specific functions like $\sin(x)$ or $x^2$), then we get a formula. Every time we plug things into that formula, it's like we're invisibly re-doing that procedure, every time.

For example, we have this fun logs rule (we just talked about this in Math 3, so it's on my mind):

$$\log_a(b\cdot c) = \log_a(b) + \log_a(c)$$

In Math 3, we proved it by making an argument like this:

\begin{align*} \log_a(b\cdot c) &= \log_a\left( \underbrace{a^{\log_a(b)}}_{=b}\cdot c \right) \quad\text{(stupidly, $b=a^{\log_a(b)}$)}\\ &= \log_a\left( a^{\log_a(b)}\cdot \underbrace{a^{\log_a(c)}}_{=c} \right) \quad\text{(equally stupidly, $c=a^{\log_a(c)}$)}\\ &= \log_a\left( a^{\log_a(b)+\log_a(c)} \right) \quad\text{(properties of exponents)}\\ &= \cancel{\log_a}\left( \cancel{a}^{\log_a(b)+\log_a(c)} \right) \quad\text{(an exponent and its log cancel out)}\\ &= \log_a(b)+\log_a(c) \end{align*}

In other words, we did a procedure in which we stupidly rewrote $b$ as $a^{\log_a(b)}$ and stupidly rewrote $c$ as $a^{\log_a(c)}$, and then used laws of exponents to simplify. But we could just do this procedure every time! For instance, if we wanted to simplify $\ln(3\cdot 7)$, we could, instead of applying the formula, just do the same procedure:

\begin{align*} \ln(3\cdot 7) &= \ln\left(e^{\ln 3}\cdot e^{\ln 7} \right) \\ &= \ln\left(e^{\ln 3 + \ln 7} \right) \\ &= \cancel{\ln}\left(\cancel{e}^{\ln 3 + \ln 7} \right) \\ &= \ln(3) + \ln(7) \end{align*}

So every formula is just a procedure. Conversely, every procedure can be described as a formula. This is basically a hand-wavey gloss on what mathematicians/theoretical computer scientists call the Curry-Howard correspondence, which says that every computer program is really a mathematical proof, and that every proof is really a computer program---code is the same as math.

So, back to this sine/Taylor series example.

We've found the Taylor series of sine by plugging it into the formula; we've even derived that formula. So, mathematically, we're set! Logically, we're sound. But I'm still intrigued by Jaxon's question---what does it look like if we actually apply the same procedure we used to prove the general formula for a Taylor series to this specific case?

So that was the question here! (And it's one I've wondered about for a while, but it's never been enough of a priority to actually do it!)

(Incidentally, this also works out to be a really great way for you guys to actually make sure you understand the proof we did---you basically get to work back through it, yourselves, applying it to this example!)

The actual derivation!

Anyway, that's some important preamble and context.

So, I worked out the Taylor series of sine, using the same procedure we did in class the Friday before Thanksgiving, but instead of using the arbitrary/generic/general/nonspecific function "$p(x)$", I ued the specific function $\sin(x)$ (which we know its derivatives, properties, how to plug things into it, etc.).

For sake of convenience, I expanded grew it around the origin ($x=0$) rather than around an arbitrary starting point/$x$-value (like $x=c$ in my notes on the proof), just to make things a little easier.

And! I did it on my kitchen whiteboard!!! This is unusual for me, because even though I know that so many of you love doing math on whiteboards, I basically never do! My handwriting is so awful that I just can't make coherent explorations... usually when I do my own math, I scribble things on pieces of paper and then start typing them up in $\LaTeX$ as soon as they start getting coherent or making sense. But on Thanksgiving morning I was in my kitchen tending to the two huge roasting trays of Alexander family chex mix that I was making, and was working on the derivation in my free moments! Here's what I came up with:

So, as you can see, I had space on my whiteboard for four integrations-by-parts. Because the constant and quadratic terms of the Taylor series of sine are zero, that works out to only getting two (nonzero) terms ($x$ and $-\frac16 x^3$, though that does take us out to the cubic terms.

Stunningly, I managed to not make any sign errors, even though the whole thing is a party of negatives. (When I wrote those notes on proof two years ago, the vast majority of my time was spent tracking down whack-a-mole sign errors!) I guess it helps to know what the answer should be (and what we want the signs to work out to be.)

I'm realizing now, looking over this picture, that I left out the first step. It's not a big deal, but I like the opening steps of a proof to be extra-clear. You always gotta start strong, in everything you do!!!! So, if I were to rewrite the beginning of this slightly, I'd say that, via the FTC, we should start with something like:

$$ \int_0^x (\sin t)'\, dt \quad=\quad \sin(x) -\sin(0) $$

Which, rearranged, is:

$$\sin(x) = \sin(0) + \int_0^x 1\cdot (\sin t)'\, dt$$

But $\sin(0)=0$, so we just get what I actually started with for my proof:

\begin{align*} \sin(x) &= \int_0^x (\sin t)'\, dt \\ \\ &= \int_0^x 1\cdot (\sin t)'\, dt \\ \\ &= \int_0^x 1\cdot \cos(t)\, dt \end{align*}

Mistakes and Errors (aka Andrew's I'm-done-with-cooking-now-I'm-bored-let's-go-on-a-giant-tangent)

I want to make another broad comment about Taylor series. I think we might have mentioned this in passing in class, but I want to talk in a slightly more depth about it.

In my derivation above, I ended up with:

$$\sin(x)\quad =\quad x - \frac16x^3 + \int_0^x \frac{1}{24}(x-t)^4\cos(t)\,dt$$

So it's a cubic polynomial, plus a trailing integral that captures the "rest" of sine. One way of thinking of that trailing integral is that it's the "error" term. In other words, we've written sine here as this nice cubic polynomial:

$$\sin(x) \quad\approx\quad x \,\,-\,\, \frac16 x^3$$

cubic_approximation(x) = x - (1/6)*x^3

xt = [k * pi for k in range(-2,4)]
xt_lbl = ['$-2\pi$', '$-\pi$', '', '$\pi$', '$2\pi$', '$3\pi$',]

cubic_approximation_plot = plot(cubic_approximation,
         ( x,-2.5*pi,2.5*pi),
             ticks = [xt, [-1,+1]],
             tick_formatter = [xt_lbl, ['$-1$','$+1$']],
     thickness = 3,
     color = 'red',
     aspect_ratio = 1,
    ymin = -1.5,
    ymax = 1.5,
    title=''
    )

show(cubic_approximation_plot)

But of course this cubic is really only accurate as an approximation of sine near the origin. Let's watch it SPIKE UP TO INFINITY once it gets too far away from the origin:

cubic_approximation_plot.show(ymin=-5,ymax=5)

So, this trailing integral is describing everything about sine that this cubic doesn't describe!

$$\sin(x)\quad = \underbrace{x \quad - \quad \frac16x^3}_{\text{the approximating cubic}} + \underbrace{\int_0^x \frac{1}{24}(x-t)^4\cos(t)\,dt}_{\substack{\text{everything that approximating cubic}\\\text{doesn't take into account}}}$$

In other words, this trailing integral is describing the error in our approximation of sine. (You could also call it the remainder term, like how you get a remainder in the long-division algorithm.) It's zero near the origin, because our error is very small. Sine does look like this cubic near the origin! But as we get further away from the origin, the error/remainder---the difference between our cubic approximating polynomial, and what sine actually is---gets really big! So the error term (the trailing integral) has to get really big, too (but with the opposite sign, to cancel things out):

$$\sin(x)\quad = \underbrace{x \quad - \quad \frac16x^3}_{\substack{\text{looks like sine near the origin;}\\\text{goes crazy elsewhere}}} + \underbrace{\int_0^x \frac{1}{24}(x-t)^4\cos(t)\,dt}_{\substack{\text{zero near the origin;}\\\text{makes up for the craziness}\\\text{of the approximating cubic elsewhere}}}$$

Visually, here's what the error term looks like (in blue:

var('t, x_2')
from sage.symbolic.integration.integral import definite_integral #WHY DO I HAVE TO IMPORT THIS EVERY DARn TIME

assume(x>0)
#i get crazy errors unless i add this assumption, which sage's error message suggested i add
# "maxima," the numerical computation library that powers a lot of sage's backend, was upset
# but obviously for the purposes of this problem x can be less than zero
# desmos doesn't complain!
# idk.

error_term(x) = definite_integral(
                            (1/24)*(x-t)^4 * cos(t),
                            t,
                            0,
                            x
        )

error_term_plot = plot(error_term(x),
            ( x,-2.5*pi, 2.5*pi),
            thickness = 3,
            color = 'blue',
            aspect_ratio = 1,
            ymin = -1.5,
            ymax = 1.5,
    )

show(cubic_approximation_plot + error_term_plot)

With more vertical drama:

(cubic_approximation_plot + error_term_plot).show(ymin=-5,ymax=5)

So it's the approximating cubic in red, and the error term in blue!!! If we add them together, we get sine, in its full glory. I'm going to plot it with alternating red and blue, to represent them combined:

var('x n')
import numpy as np
sine_plot = sum([plot(sin(x), 
               (x,n/5,n/5 +1), 
                      thickness = 4,
              color = ['red', 'blue'][ n % 2]
              ) for n in np.arange(-30,30,1)])

( sine_plot + cubic_approximation_plot + error_term_plot).show(ymin=-5,ymax=5)

Actually, I then had an even BETTER idea for a visualization, in which sine itself slowly fades from red to blue, as the percentage of the cubic approximation "in" the sine function diminishes... but actually, it turns out that this is kind of undefined (what do I actually mean by that?) and I played with variants of this for a while (and whined about it to Patrick's brother, who happened to be nearby), but couldn't make anything work and eventually decided that I had way too many other more pressing projects to work on. SAD!

WAIT WAIT JUST KIDDING I JUST REALIZED HOW TO DO WHAT I WANT. OK, I'm not going to explain the details, but here we go:

sine_plot_continuous_coloring = sum([plot(sin(x), 
               (x,n/5,n/5 +1), 
                thickness = 4,
             ticks = [xt, [-1,+1]],
             tick_formatter = [xt_lbl, ['$-1$','$+1$']],
              color = Color(   1 / (1 + abs(error_term(n/5))),  # percent red
                                            0, # percent green
                                             1 - 1 / (1 + abs(error_term(n/5))), # percent blue
                                              space = 'rgb')
              )
               for n in np.arange(-30,30,1)])

(sine_plot_continuous_coloring ).show(ymin=-1.5,ymax=1.5)

Anyway, the coloring is basically related to how good the cubic approximation is at that point... details in the code if you want to poke at it, or have ideas for improvements. It's definitely not all I envisioned, but better than nothing!

Here's a second version, in which it's reddest where the

sine_plot_continuous_coloring_v2 = sum([plot(sin(x), 
               (x,n/5,n/5 +1), 
                thickness = 4,
             ticks = [xt, [-1,+1]],
             tick_formatter = [xt_lbl, ['$-1$','$+1$']],
              color = Color(  1,  # percent red
                                            1-1 / (1 + abs(error_term(n/5))), # percent green
                                             1-1 / (1 + abs(error_term(n/5))), # percent blue
                                              space = 'rgb')
              )
               for n in np.arange(-50,50,1)])

(sine_plot_continuous_coloring_v2 ).show(ymin=-1.5,ymax=1.5)

Here's all three of them:

(cubic_approximation_plot + error_term_plot + sine_plot_continuous_coloring ).show(ymin=-3,ymax=3)

So, anyway, that's a long fun tangent; I just wanted to make some points about how we can represent functions as their Taylor approximation plus some error/remainder term:

$$ \text{a function}\quad =\quad \left(\substack{\text{its approximating}\\\text{polynomial}}\right) \quad+\quad \left(\substack{\text{the error/}\\\text{remainder term}}\right) $$

Chex mix

If you want my family Chex Mix recipe (shoutout to Auntie Bonnie!), here it is: