Populate an initial library based on the graph information