Cassar's date of birth is not known, but he is believed to have been born in around 1520 in either Birgu or Gudja. The Cassar family, probably originating from Sicily, had been established in Malta since at least the year 1440. He had two brothers, named Andreas and Ians, and married his wife Mathia in the early 1560s after which they had five children -two boys, Vittorio and Gabriele; and three girls, Marietta, Battistina and Caterinella. Cassar had another child, Gio Domenico, from an extra-matrimonial relationship which he made sure to inherit none of his will and testament, ironically justifying himself by writing that his son was disobedient. Cassar was initially a "capomastro" (master builder), and he was a pupil of Evangelista Menga, the resident engineer of the Order of St. John. He worked as a military engineer during the Battle of Djerba in 1560 and the Great Siege of Malta in 1565. During the latter siege, he worked on repairing fortifications damaged by the Ottoman assaults, sometimes at great personal risk.