If $y=mx+c$ is a normal to the ellipse $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1$, then $c^2$ equals

Suppose $y=mx+c$ is a normal to the ellipse $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1$ at point $(a\cos \theta , b\sin \theta )$. An equation of normal at $(a\cos \theta , b\sin \theta )$ is $ax\sec \theta -by\mathrm{cosec}\theta =\left(a^2-b^2\right)$     (i)

Note that this equation is the same as

$mx-y= -c$       (ii)

Comparing (i) and (ii) we get

$\begin{array}{l}\frac{m}{a\sec \theta }=\frac{-1}{-b\mathrm{cosec}\theta }=\frac{-c}{a^2-b^2}\\ \Rightarrow \cos \theta =\frac{-ac}{m\left(a^2-b^2\right)},\sin \theta =\frac{-bc}{a^2-b^2}\end{array}$

Now, $1={\cos }^2\theta +{\sin }^2\theta =\frac{c^2\left(a^2+b^2{m}^2\right)}{m^2{\left(a^2-b^2\right)}^2}$

$\Rightarrow c^2=\frac{{\left(a^2-b^2\right)}^2{m}^2}{a^2+b^2{m}^2}$.