The clock is ticking on the interior design guru to finish renovating his Georgian townhouse, and he admits the project has been "really difficult"