Gerakan says it is "sorely disappointed" that the number of tourists visiting Penang last year plunged, despite the state being declared a Unesco world heritage site two years ago.