Literary and Musical Allusions (probably SPOILERS)
I caught two:
1. The song "Clementine" includes the words "You are lost and gone forever... " I take that to allude to both the Lacuna procedure* and the probable final result of their reconnection.
(* somehow reminds me of the "Ludovico Technique" from "A Clockwork Orange."
2. The poem Eloisa to Abelard by Alexander Pope, from which the movie's title was taken. A wonderful tear-jerker of an epic poem based on real events. Read it, you;ll be glad you did. It also alludes to a wish to wipe away the sorrow of a failed relationship. Read Wikipedia entry first to get the background. It'll help as you read the 18th century language.
3. There's the song at the end "I Need Your Lovin' (like the sunshine)." I think it's a repeat from Joel hearing it on tape in his car, after which he discards the tape.