Strange Woman Problem

I realized lately that I have a rather weird problem with women, strange in that it's about how they perceive me rather than how I feel around them. In short, my difficulty is . . . I'm way too comfortable with them! Why is this a problem? Well, it makes cultivating friendships for friendship's sake a lot more difficult since many seem to think I view them as a romantic prospect due to my ease and comfort, I think.

I grew up around virtually nothing but women. I never met my father and my grandfather died while I was still somewhat young, so my particular section of the family made it so I was in contact with women incredibly more often than with men. It was not a rarity that I'd attend a gathering and be the only male there. This environmental influence has probably made it so I treat women with super casual ease, rather than as if they were the mysterious "other sex."

It surprises me how often some people will assume I'm dating a girl just by the way I speak to her, and for the past several months that's been a very frequent question at work. Gosh, it seems like it's impossible to speak one-on-one with a female without being asked if I'm in romantic relations with her. Are men today so sheltered from women?

One thing I have to wonder is if there's some aspect of my behavior that I could alter in order to avoid the confusion, or if I'm acting reasonably and the misinterpretation lies solely in their incorrect perception. Sure I'd like to cultivate a romantic relationship, but first I'd like to tend to my friendships, including that with women. It complicates matters when my only intention is to be a friend to someone and I observe from their behavior a development of a crush. I'm vain certainly, but I don't much respond to that kind of attention, though am not resistant to it.

Thoughts?