Generally, it just depends on cultural norms. Like the the word onanism, meaning masturbation, used to be very common. Being called an 'onanist was like being gay it meant you were a pervert. Now we consider masturbation to be completely normal behaviour, that has no bearing on what kind of person your are. The main difference is that there are behaviours which very clearly harm others (pedophilia, bestiality), and ones which very clearly don't (masturbation, non-procreative sex).