Magnesium starts floating on water because

Magnesium starts floating on water because hydrogen gas formed at the initial stage of the reaction.
The hydrogen gas evolved due to the reaction between magnesium and water. 

Magnesium does not react with cold water. It reacts with hot water and forms $\mathit{Mg}{\left(\mathit{OH}\right)}_2$ and hydrogen. Bubbles of hydrogen gas stick to the surface. That’s why Mg starts floating on the water. 

Its reaction is given as
$\mathit{Mg}+2H_{2}O\to \mathit{Mg}{\left(\mathit{OH}\right)}_2+H_2\uparrow$