You all know, I'm sure the lines, "We'll always have Paris," followed by, "And what we lost we got back a little last night." Juxtapose that with, "and I LET HER PRETEND," --
Did Rick and Ilsa, ahem, do the "nasty" in his office that night?
Of course they did. And Victor knows that Ilsa wasn't just pretending. Rick's need to tell him this is probably his way of testing Victor to see what his reaction will be--will he be disgusted with his wife and leave her with Rick, displaying similar behavior to Rick's when he got drunk and berated Ilsa the night she returned to Casablanca? Then Rick will have Ilsa for himself. But Victor loves Ilsa so much that he's instantly willing to forgive her, which shows Rick that Victor is the bigger man and more worthy of Ilsa. So he and Ilsa escape together, leaving Rick and Louis to go off to Brazzaville.
I'm not crying, you fool, I'm laughing!
Hewwo.
Yes, there is a lot of evidence to support what you are saying, with the possible exception of Lazlo's accepting Ilsa's infidelity so easily and instantly. That doesn't make sense to me at all. Unless he believed that she did it ONLY to get the letters of transit from Rick. Hard for me to believe that such a sensitive man such as Lazlo would 1) forgive such an indiscretion and 2. would want to keep Ilsa from the man she loves that she will always love.
I don't know. A case could be made either way I think. But I think you're right, there doesn't seem to be any doubt that they "did the nasty.'