Find the mass M (in kg) so that it remains at rest in the below diagram. Both the pulley and string are ideal and friction is absent everywhere

 

Let T be the tension in the string attached to M and T’ the tension in the string attached to 3kg and 2kg. 

Free body diagram for M as shown above. 

Since M is at rest, $Mg=T\mathrm{.......}\left(a\right)$

Free body diagram for pulley 2 yields $T=2T\text{'}---------\left(b\right)$

For the 3kg mass, the equation of motion is $3\times g-T\text{'}=3a\mathrm{......}\left(c\right)$

For the 2kg mass, the equation of motion is $T\text{'}-2g=2a\mathrm{......}\left(d\right)$

$from \left(c\right) and \left(d\right), we get, g=5a$

$\therefore a=\frac{g}{5}\mathrm{......}\left(5\right)$

Substituting for a in (d), we get, $T\text{'}=2\times \frac{g}{5}+2g=\frac{12g}{5}\mathrm{..........}\left(6\right)$

From (b), we get, $T=2\times \frac{12g}{5}\mathrm{....}\left(7\right)$

From (a), we get, $Mg=\frac{24g}{5}, \therefore M=\frac{24}{5}=4.8\mathrm{kg}$

Therefore the correct answer is 4.80