Prime Minister Cameron says links to the political group should be considered "a possible indicator of extremism".