I handled Butter form day 1, when he was about 3 weeks old. He probably tolerated me out of fear then, but now, with his big boy bravado, is fine letting me know that he hates me.
He tolerates me as a feeding machine. Yes, if he is interested in what I have in my hand, he will tolerate me more, but I don't tell myself that he likes me, he likes what my hand is bringing (dinner).