We have, 

$\overrightarrow{a}\times \overrightarrow{b}+\overrightarrow{b}\times \overrightarrow{c}=p\overrightarrow{a}+q\overrightarrow{b}+r\overrightarrow{c} ....\left(i\right)$

Taking dot product with$\overrightarrow{a}$, we obtain 

$\overrightarrow{a}\cdot (\overrightarrow{a}\times \overrightarrow{b})+\overrightarrow{a}\cdot (\overrightarrow{b}\times \overrightarrow{c})=p(\overrightarrow{a}\cdot \overrightarrow{a})+q(\overrightarrow{a}\cdot \overrightarrow{b})+r(\overrightarrow{a}\cdot \overrightarrow{c})$

$\Rightarrow \left[\overrightarrow{a}\overrightarrow{b}\overrightarrow{c}\right]=p+\frac{q}{2}+\frac{r}{2} ....\left(ii\right)$

Taking dot product on both sides of (i) by $\overrightarrow{b}$,  we get 

$\overrightarrow{b}\cdot (\overrightarrow{a}\times \overrightarrow{b})+\overrightarrow{b}\cdot (\overrightarrow{b}\times \overrightarrow{c})=p(\overrightarrow{b}\cdot \overrightarrow{a})+q(\overrightarrow{b}\cdot \overrightarrow{b})+r(\overrightarrow{c}\cdot \overrightarrow{c})$

$\Rightarrow 0=\frac{p}{2}+q+\frac{r}{2} ...\left(iii\right)$

Taking dot product on both sides of (i) by $\overrightarrow{c}$, we get

$\overrightarrow{c}\cdot (\overrightarrow{a}\times \overrightarrow{b})+\overrightarrow{c}\cdot (\overrightarrow{b}\times \overrightarrow{c})=p(\overrightarrow{c}\cdot \overrightarrow{a})+q(\overrightarrow{c}\cdot \overrightarrow{b})+r(\overrightarrow{c}\cdot \overrightarrow{c})$

$\begin{array}{l}\Rightarrow \left[\overrightarrow{c}\overrightarrow{a}\overrightarrow{b}\right]=\frac{p}{2}+\frac{q}{2}+r\\ \Rightarrow \left[\begin{array}{lll}\overrightarrow{a}& \overrightarrow{b}& \overrightarrow{c}\end{array}\right]=\frac{p}{2}+\frac{q}{2}+r ...\left(iv\right) \end{array}$

From (ii) and (iv), we get 

$p+\frac{q}{2}+\frac{r}{2}=\frac{p}{2}+\frac{q}{2}+r\Rightarrow p=r$

Putting $p = r$ in (iii), we get $q = - r.$ Thus, we obtain 

$\begin{array}{l} p=r=-q.\\ \therefore \frac{p^2+2q^2+r^2}{q^2}=\frac{(-q{)}^2+2q^2+(-q{)}^2}{q^2}=4\end{array}$