A grandfather has been left £36,000 out of pocket and forced to sell his holiday home in Somerset after the ground rent was increased by £5,000.