Two lumps of clay each of rest mass $m_0$ collide with a speed of  $\frac{4}{5}c$ head on and stick together The mass of the composite lump thus formed is

Energy of each lump before collision is

$E=m{c}^2=\frac{m_0{c}^2}{\sqrt{1-{\left(\frac{4}{5}\right)}^2}}=\frac{5}{3}{m}_0{c}^2$

The energy of the composite lump after the collision will be $M{c}^2$ .

By energy conservation principle,  

Total initial energy = Total final energy

$$\begin{gathered} \Rightarrow \frac{5}{3}{m}_0{c}^2+\frac{5}{3}{m}_0{c}^2=M{c}^2 \\ \Rightarrow M=\frac{10}{3}{m}_0 \end{gathered}$$