The former first minister warned the toxicity of the debate around immigration had "managed to find its way up here".