What determines the sex of the baby?

The sex chromosome of the father determines the sex of the baby. 

The sperm cell of the father contains either an X or a Y-chromosome. The Y-chromosome is responsible for a male child while X-chromosome is responsible for a female child. 

Thus, if a sperm cell with Y-chromosome fuses with the egg cell then the zygote will form a male foetus whereas, a sperm cell with an X-chromosome fusing with the egg cell will result in a female foetus.