Soft Hand Problem

I am 18(M).Recently almost every one that I have shaken hands with told me that my hands are very soft. They were trying to say that I'm a boy and I shouldn't have soft hands like girls. One guy even told me to punch the wall everyday to make my hands harder. I don't get it , is that really that big of a deal? It's kinda annoying ngl.