Both morals and ethics fit in with it being a mental issue. That's where the mental aspect stems from. Also feeling of possible persecution from people who may judge the female and probably less so the male. It isn't right but I have seen it too much to know the woman often suffers more indignation than the male - even from fellow women.
The other physical problem might come in the future, There have been women, though I do not know any statistics on this at all, who have also had difficulty or found it impossible to become pregnant after having had an abortion in the past. I cannot even begin to believe how emotionally devastating that may be.
Just to be clear on it I do know two women who were unable to becaome pregnant and each had received abortions in the past. The fact that the abortion may have had little to do with their inability did little to mitigate the psycological impact. That said I would expect the incidence of that happening today is rather low.
I think I'm trying to separate the moral/ethical and psychological as being separate things. The moral/ethical being more of an issue with society. The psychological part being whether the person in question is comfortable with the idea of preventing or ending a possible human life (if they even view it in that way at all).
And the physical, I agree with you on that. It's a significant procedure for the human body to cope with.
But assuming that you're (the person is) comfortable with the psychological part (eg, maybe you don't see it as a human life) and accept the possible future physical issues I don't believe society should dictate.