Malay rights group Perkasa today suggested that an Islamic affairs ministry be set up to tackle all issues related to Islam which are becoming increasingly challenging.