PAS president Abdul Hadi Awang has conceded that it would be an uphill task for the Islamic party to win the Bagan Pinang by-election.