Is it possible other men don't like me due to my appearance ?

Ill start off by saying I'm the least vain person ever I am just genetically blessed that I train hard and can eat whatever I want and end up looking like a seasoned body builder... sure I shower and brush my teeth but that is about all I do towards my appearance... I shave my head so my grooming takes 5 mins max lol.

However whenever I meet new guys for the first time we get on great! i'm very chatty and complimentary and genuinely care about people and their feelings doing my best to big them up whilst giving them good laughs usually at my own expense... I've never felt any jealousy towards someone well not that I can remember and live in my own little bubble of working out, eating, sleeping and watching movies or writing music...

However I begin to notice with every male friends I make certain things that begin to play on my mind... the little putdowns for no reason and I began to also notice them comparing for example the size of their arms to mine whilst in a conversation... my arms biceps/triceps are very big again I train hard but my genetics have a big part to play in this.

Am I being paranoid or are these new male friends possibly being half friends half resentful/sabotaging ?

It's a weird dynamic of they want to be my friend as they seek me out when ever is possible but then when in my company look for ways to try and bring me down... even if it is very subtle..