A leading housing charity blasted John Swinney's Government and said it was time for ministers to be "frank" with the public.