Calculus Small Changes, Big Ideas

Section 12.5 Multivariable optimization

• If a function has a local maximum or minimum at \((a,b)\text{,}\) then both partial derivatives must be zero (meaning that \((a,b)\) is a critical points.

  🔗
  🔗
• However, just because we have a critical point doesn’t mean it’s a local maximum or minimum; it can also be a saddle point.

  🔗
  An example is \(f(x,y)=x^{2}-y^{2}\) at the origin.

  🔗
  🔗
• Second Derivative Test:

  🔗
  Suppose \(f\) has a critical point at \((a,b)\text{.}\) Let:

  \begin{equation*} A=f_{xx}(a,b)\qquad B=f_{xy}(a,b)\qquad C=f_{yy}(a,b) \end{equation*}

  The discriminant is \(K=AC-B^{2}\text{.}\)

  • If \(K\lt 0\text{,}\) then \((a,b)\) is a saddle point.

    🔗
    🔗
  • If \(K\gt 0\text{,}\) then \(f(a,b)\) is a local maximum (if \(A\lt 0\)) or local minimum (if \(A\gt 0\)). This is just like the single variable case.

    🔗
    🔗
  • If \(K=0\text{,}\) no conclusion can be made.

    🔗
    🔗

  🔗
  🔗
• Justify this using the Taylor approximations. (Look at Thomas for reference.)

  🔗
  🔗
• Potential motivating application could be least squares regression.

  🔗
  Given a set of points \((x_{1},y_{1}),\cdots,(x_{n},y_{n})\text{,}\) we want to find the line \(y=ax+b\) that ‘best fits’ the data.

  🔗
  The residuals are \(\eps_{k}=y_{k}-(ax_{k}+b)\text{,}\) so we want to minimize the sum of the squares of the residuals, which we can represent as the function

  \begin{equation*} S(a,b)=\sum_{k=1}^{n} \eps_{k}^{2}\text{.} \end{equation*}

  🔗
  The solution ends up being the solution to the equations

  🔗
  \begin{align*} a\sum_{k=1}^n x_{k} \amp = \sum_{k=1}^n x_{k}y_{k}\\ b\sum_{k=1}^n 1 \amp = \sum_{k=1}^n y_{k} - a\sum_{k=1}^n x_{k} \end{align*}
  🔗