chafing my mind
It is a common term, often used when 'grinding my gears' or 'annoying me' have already been used or aren't quite alternative and
imaginative enough. Much like the
chafing of
tight underwear against one's thighs, the chafing of the mind is both annoying and unwanted. When someone's rubbing your brain and leaving a rash, just tell 'em how you feel.