The musician supported Will Smith at Cardiff Castle, but he has a special place in his heart for one Welsh town