"Did anyone ever tell you that you look like a penis with that little hat on?"
That is a great line on more than one level. Folks who know sports - baseball in particular - know Jimmy said that to be sure he'd be thrown out of the game.
But then in his next breath, Jimmy acts like there has been some sort of miscommunication, which is hilarious.