A Penang executive councillor said the federal government should listen to the people if it is serious about amending the Internal Security Act (ISA).