added logic with crawler for all twitter format