The pan-Arabic culture of the era that made use of scimitars up thru the European Renaissance was easily the or one of the most advanced enlightened civilizations up to that point in history. What's wrong w/ being associated w/ that?
I've also somewhat stereotyped many aspects of
Hobbits as 'English' and
Rohirrim as 'Northern European'. Should I be cast upon the pillar of shame?
All descriptions have to be relative to a common base so even if JRRT meant to create a purely new fantasy world - it's reasonable to expect such inferences among readers. I don't think there is any right or wrong here - readers are free to read and interpret the descriptive images in their own minds based on their own experiences, knowledge, prejudices etc... that we all have.
As for descriptive words like swarthy I simply don't know what swarthy means - the dictionary tells me it means dark skinned or dark complected but I've always drawn a blank on that word.