The given function is
h(x,y) = x² - 9xy - y²
Calculate partial derivatives.
[tex]h_{x} = 2x \\ h_{xx} = 2 \\ h_{y} = -2y \\ h_{yy} = -2 \\ h_{xy} = 0
[/tex]
For the critical points,
[tex]h_{x} = 2x=0 \, \Rightarrow \, x=0 \\h_{y}=-2y=0 \,, \Rightarrow \, y=0[/tex]
To perform a test for a saddle point, calculate
[tex]D = h_{xx}(0,0) h_{yy} (0,0) -[ h_{xy}(0,0)]^{2} = (2)(-2) - (2)^{2} = -8[/tex]
Because D < 0, a saddle point exists at (0,0).
