At its most basic, they tried to kill (and EAT) you and yours. That deserves death right there.
In the larger sense, any survivors are apt to keep cannibalizing people.
Like Whiskey said, returning would be dangerous, but morally, there should be no real issues.
But this IS Walking Dead. Its supposed to make people think and discuss...