Rumours and a negative perception of the security situation in Sabah following the Sulu terrorist intrusion have caused a decline in tourist arrivals in the state, Tourism Minister Ng Yen Yen said today.