They had been the unclean. A Millennia ago a plague had struck the united worlds. It warped those who survived. Some it drove insane. Other's, it changed their bodies so much, their own families could no longer recognize them. The worst it appeared, was that the plague changed the genetic code of the survivors, they were no longer human.
In the hysteria of the times, what was left of the government gathered the survivors and put them on an old colony ship and sent them into the void. They felt it was the best they could do, at least this way they had a chance of survival. In time the United worlds healed and grew. The plague was long forgotten except for the ancient archives and legends.....
And except for the survivors.