Thousands of people gathered at the Batu Caves temple this morning to protest against the Sri Lankan government despite a police warning against the rally.