Not necessarily. These granules are intended to resist solar radiation and help extend the life of the roof. It is completely normal for some excess granules to loosen themselves because wind, rain, and other forms of erosion are commonplace. This is expected and only becomes a concern when the shingles become physically damaged, punctured, or worn down until the tar is exposed.