http://thepeerage.com/p24747.htm
This is the clearest indication that he fathered illegitimate children by Dorothy McIlroy and that the oldest , James Annesley, born 1801 was tried in Dublin and transported to NSW for 7 years (as James Ansley) for "Crimes :Various" His name does not appear on a number of records.
He married Elizabeth Quinn and their son born 1831, John Annesley married Merab Wenban. Their daughter Mary Ann Annesley married James Jackson, born in Castlecomer, Kilkenny, Ireland in 1854, and she was my great grandmother, through the Jackson line.