So was Moses her father?
Certain details hint that he might have been:
1). Various people remark that they resemble
2). He admits that he was close to her mother, and he had met the mother in a bar. He talks of how the mother was beautiful, like Addie.
3). He doesn't put up a huge fight when they tell him to take Addie, besides saying, "Well, let me think of it" briefly. That detail hints at some fatherly affection
What I think is that Moses had hooked up with Addie's mother (a possible prostitute / slut), and then he and Addie's mother split before the pregnancy. So he really can't say for sure that he is father, but he also can't rule it out