A compass needle which is allowed to move in a horizontal plane is taken to a geomagnetic pole. It

Horizontal component of earth's magnetic field is given by 

${\mathrm{B}}_{\mathrm{H}}=\mathrm{Bcos\delta }$

At the geometric poles, the angle of dip becomes $\pm {90}^0$ and therefore ${\mathrm{B}}_{\mathrm{H}}$ becomes zero. 

Hence there won't be any force on the compass needle and it will stay in any random position.