Q:

scrapy can't get right response

I'm trying to get song&singerdata from http://music.163.com/#/artist?id=16686, but I can't get the right response. 
I checked on scrapy shell, when I request "music.163.com/#/artist?id=16686", the response is "music.163.com".  I didn't know the reason. 
Below is the log
C:\Users\lszxw\PycharmProjects\untitled\scrapy\tutorial\tutorial>scrapy shell http://music.163.com/#/artist/album?id=16686
2017-01-16 22:47:03 [scrapy.utils.log] INFO: Scrapy 1.3.0 started (bot: tutorial)
2017-01-16 22:47:03 [scrapy.utils.log] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutorial.spiders', 'LOGSTATS_INTERVAL': 0, 'BOT_NAME': 'tutorial', 'DUPEFILTER_CLASS': 'scrapy.dupefilters. BaseDupeFilter', 'SPIDER_MODULES': ['tutorial.spiders'], 'ROBOTSTXT_OBEY': True}
2017-01-16 22:47:03 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats. CoreStats',
 'scrapy.extensions.telnet. TelnetConsole']
2017-01-16 22:47:03 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt. RobotsTxtMiddleware',
 'scrapy.downloadermiddlewares.httpauth. HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout. DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders. DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent. UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry. RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect. MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression. HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect. RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies. CookiesMiddleware',
 'scrapy.downloadermiddlewares.stats. DownloaderStats']
2017-01-16 22:47:03 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror. HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite. OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer. RefererMiddleware',
 'scrapy.spidermiddlewares.urllength. UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth. DepthMiddleware']
2017-01-16 22:47:03 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2017-01-16 22:47:03 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023
2017-01-16 22:47:03 [scrapy.core.engine] INFO: Spider opened
2017-01-16 22:47:03 [scrapy.core.engine] DEBUG: Crawled (404) <GET http://music.163.com/robots.txt> (referer: None)
2017-01-16 22:47:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://music.163.com/#/artist/album?id=16686> (referer: None)
2017-01-16 22:47:04 [traitlets] DEBUG: Using default logger
2017-01-16 22:47:04 [traitlets] DEBUG: Using default logger
[s] Available Scrapy objects:
[s]   scrapy     scrapy module (contains scrapy. Request, scrapy. Selector, etc)
[s]   crawler    <scrapy.crawler. Crawler object at 0x0000018893DAB9E8>
[s]   item       {}
[s]   request    <GET http://music.163.com/#/artist/album?id=16686>
[s]   response   <200 http://music.163.com/>
[s]   settings   <scrapy.settings. Settings object at 0x0000018893DCEF98>
[s]   spider     <DefaultSpider 'default' at 0x18893fd39b0>
[s] Useful shortcuts:
[s]   fetch(url[, redirect=True]) Fetch URL and update local objects (by default, redirects are followed)
[s]   fetch(req)                  Fetch a scrapy. Request and update local objects
[s]   shelp()           Shell help (print this help)
[s]   view(response)    View response in a browser

Below is my code, it contains the real url address:
import scrapy

class KokiaSpider(scrapy. Spider):
    name = 'kokia'

    def start_requests(self):
        start_urls = ["http://music.163.com/#/artist/album?id=16686&limit=12&offset=0",
                  "http://music.163.com/#/artist/album?id=16686&limit=12&offset=12",
                  "http://music.163.com/#/artist/album?id=16686&limit=12&offset=24",
                  "http://music.163.com/#/artist/album?id=16686&limit=12&offset=36",
                  "http://music.163.com/#/artist/album?id=16686&limit=12&offset=48",
                  "http://music.163.com/#/artist/album?id=16686&limit=12&offset=60",]

        start_urls=["http://music.163.com/#/artist/album?id=16686",
                    ]
        for url in start_urls:
            yield scrapy. Request(url=url, callback=self.parse)

    def parse(self, response):
        #url = 'http://music.163.com/#'
        for item in response.xpath('//*[@id="m-song-module"]/li/p[1]/a/@href'):
           # full_url = url + item.extract()
            full_url = response.urljoin(item.extract())
            self.log('full_url %s' %full_url)
            # yield scrapy. Request(full_url,callback=self.parse_album)

    def parse_album(self,response):
        for item in response.xpath('//table[@class="m-table"]/tbody/tr/td[2]//a/@href'):
            # full_url = url + item.extract()
            full_url = response.urljoin(item.extract())
            self.log('full_url %s' %full_url)
            yield scrapy. Request(full_url,callback=self.parse_song)
    def parse_song(self,response):
        song_name = response.xpath('//div[@class="hd"]/div/em/text()').extract_first()
        singer_name = response.xpath('//p[@class="s-fc4"][1]/span/a/text()').extract_first()
        album_name = response.xpath('//p[@class="s-fc4"][2]/a/text()').extract_first()
        comments_num = response.xpath('//*[@id="cnt_comment_count"]/text()')
        yield{
            "song:":song_name,
            "singer:":singer_name,
            "album:":album_name,
            "comments:":comments_num
        }

A:

Your start_urls seem to be incorrect.  If you inspect the networks tab and the page source you'll notice that the album/song data is actually contained in an <iframe> tag which leads to pretty much the same url just without the #:
"http://music.163.com/#/artist/album?id=16686"
# becomes:
"http://music.163.com/artist/album?id=16686"

Afterwards your song xpath in parse_album is incorrect.  I've got it working with this one:
"//ul[@class='f-hide']/li/a/@href[contains(.,'song')]"

Afterwards everything seems to be working.