Realistic Views on Sex and Love?

A recent incident with my LD boyfriend has made me start reflecting on if my expectations and how I view commitment is realistic. To sum it up without going into details of the conversation too much, he asked if I would be interested in or thought about having “hookups” because we are LD and we cannot meet the physical needs of each other. I told him I have not thought about having sex with others and I never would entertain the idea. I asked if he had. He admitted that he has thought about having sex with other people, and it’s essentially human nature for men to want to have sex with others, even after you know that you are committed to and love someone already. My emotional reaction was not good. I felt truly devastated. I feel like even if you are apart, if you truly love that person having sex with someone else shouldn’t even cross your mind, that your person is the only person you want in that capacity. But I’m starting to think maybe I’m being unrealistic to be the only person your partner would ever want to sleep with. I guess my question is this: Is the expectation of being the only one your partner wants sexually unrealistic? And if not how do I deal with the feelings of hurt that I’m experiencing around this?