Jeff was best when he didn't narrate during competitions in the earlier seasons, or at least very little. His voice is unbelievably distracting--I couldn't imagine being a player and having to hear Jeff's voice telling you, the player, how everyone else is doing and how poorly you're doing.
Not to mention, he loudly points out strategies or techniques people are using during the challenge that others probably wouldn't have noticed because they were focusing on doing the challenge themselves.