The Duke is due to spend four days solo in the UK this week, but would love to make another trip in the future with wife Meghan and their two children so they can "see where he grew up"