Malaysia and Thailand realise that a tussle over the fate of 131 displaced Thai Muslims must not poison relations and their handling of the issue is slowly improving, Malaysia said today.