Malaysia is looking at the possibility of replacing the Johor-Singapore Causeway with a straight bridge, said Youth and Sports Minister Khairy Jamaluddin.