The "How do you cope with loneliness, Harvey?" scene is amazing
They were a real-life odd couple, the nerd and the grouch, so it was great to see them interact. I rarely see loneliness discussed by characters, let alone real people, so that's unique. And it's funny to see Giamatti chuckle in the background when real-Toby says, "...you do your stick figures..."
Pathos, comedy, meta-references - there's a lot going on in that one scene. Definitely the highlight of this movie for me.