He insists that none of the shared houses have been occupied by illegal migrants or used as a halfway house for former prisoners