Family members of Hindraf legal advisor and ISA detainee P Uthayakumar are worried that his health was getting worse under detention.