The equestrian show jumping venue has drawn many A-list visitors over the years, including Queen Elizabeth