Was Truman only real resident in town?
After reading other threads, believe the actors only returned to set when he was scheduled to be up. You will notice when he leaves at end, they are called in to rtn early to locate him. Even the scene between him and his friend after his wife leaves is just between them and sure the other actors left the building to go home.
But does beg a question that if he was to ever need or chooses to visit a neighbor, what excuse they would give to an empty house.