PKR leader Anwar Ibrahim today admitted that he should have taken action sooner to sack recalcitrant Pelabuhan Klang state assemblyperson Badrul Hisham Abdullah who has since defected from the party.