Algorithm for detection of filamentary structure in molecular clouds