Defence Minister Peter MacKay said the Conservatives will not try to remove Richard Colvin from his post in Washington, even though they question the credibility of his testimony on Afghan prisoners.