The six-time Olympic champion said his trips to Wales have become 'one of the things he looks forward to most'