The Duke of Sussex set up the African charity in honour of his mother but now its chair claims he was the ‘number one’ risk to it