scrapy/scrapy — pull request #941 by bernardotorres (2014-11-05 22:41:13+00:00)
Title: Adding support to FTP file listing

Make possible to scan FTP repositories. If the path of the request ends in / in a FTP URL, calls FTP list.

It assigns a variable: response.file with the structure of a FTPFileListProtocol (see: http://twistedmatrix.com/documents/8.2.0/api/twisted.protocols.ftp.FTPFileListProtocol.html):

filetype: e.g. 'd' for directories, or '-' for an ordinary file
perms: e.g. 'rw-r--r--'
nlinks: e.g. 1
owner: e.g. 'root'
group: e.g. 'other'
size: e.g. 531
date: e.g. 'Jan 29 03:26'
filename: e.g. 'README'
linktarget: e.g. 'some/file'