The man said he came forward with sexual abuse allegations after learning Gareth Ward had been appointed the minister responsible for foster children.