My XDs was my first polymer pistol. I like it very much. I don't think the recoil is bad at all, and at the realistic ranges I would use it at it is more than accurate enough. I did get it in .45, based on nothing more than I like the .45. My other .45 is a full sized 1911A1.
To me what a pistol is made of is less important than reliability and accuracy for the task.