So basically…
Ben Affleck’s character barely knew Liv Tyler’s. The kid was too young to fully understand the situation. So basically Ben stayed because his dad didn’t want his son to move out.
The dad wanted Ben to put aside the thing he had been wishing for for almost a decade, the thing that made him happy, and stay in the same sad job as him. The dad never considered moving with his son or at least visiting his son for a change.
There could be a sequel where the daughter comes home from the city to visit alcoholic Ben living in Jersey and the Ben manipulates the daughter into staying.