Ben did not have sex with Sarah. They kissed several times throughout the journey. Maybe he told his wife, maybe he didn't, but don't forget, the wife also kissed her ex-boyfriend too so I would say they were about even.
He didn't have a physical affair, but rather an emotional affair. It was very brief and probably only occured because Ben had some doubts about getting married the worse the journey got. Sarah was exciting, quirky and beautiful, and probably brought a new side of Ben. However, once he saw his fiancee in her wedding dress, he did realize that she was the one he wanted and that Sarah was not.