A ray enters a glass slab at an angle $\mathrm{\theta }$ from air and the refractive index varies along the slab as $\mathrm{\mu }={\mathrm{\mu }}_0-{\mathrm{kt}}^2$, where t is the distance measured along the normal.  ${\mu }_0$  and k are positive constants. If the glass slab is sufficiently thick, how far along the normal will the ray go before it reflects back?

As t is increasing $\mathrm{\mu }$ is decreasing. So, refraction is taking place from denser to rarer medium as light is proceeding in the medium. Using continuous varying refractive index, let the ray reflect back after traveling $t=t_0$ along the normal, then at this location, angle of refraction is $\frac{\pi }{2}.$ 

$1\times \mathrm{sin\theta }=({\mathrm{\mu }}_0-{\mathrm{kt}}_0^2)\sin \left(\frac{\mathrm{\pi }}{2}\right)$

$\Rightarrow {\mathrm{t}}_0=\sqrt{\frac{{\mathrm{\mu }}_0-\mathrm{sin\theta }}{\mathrm{k}}}$