I don't know how well I understand Augustine, but I can relate to the fear of growing too deeply into earthly attachments. I take great comfort in the company of my wife and children, but I worry sometimes about becoming too dependent in them, of relying so much on earthly kin that love drifts imperceptably into idolatry. According to Jesus, these kinships are the very kind that can separate us from God (Luke 14:26).
But 2 Cor. 2:12-13 shows the other side. When Paul came to Troas he was disappointed not to find Titus, who would have provided a rest for Paul's spirit. Eventually, Paul did meet up with Titus and found comfort in his presence (2 Cor. 7:6-7). Though Paul could be content in any circumstance (Phil. 4:11), he took special comfort in a beloved son (Titus 1:4). The point? Faithfulness isn't stoicism.
Paul wasn't alone in such a view. Jesus, after all, sent out his ambassadors in pairs, and in his own night of anguish Jesus himself sought the company of his three closest friends.
The OT makes clear that God created human beings to share life in community with one another, and that as long as the Lord builds the house, loving human relationships are very good. That should already be clear enough to a Christian. But sometimes it's good to be reminded that taking joy in our loved ones is even better than OK.