A ceramah organised by Anything But Umno (ABU) was cancelled at the last minute due to a disturbance from a "rowdy" group.