def parse(self, response): current_url = response.request.url
Here is what the above code is Doing:
1. We’re creating a new class called ‘MySpider’ that inherits from ‘scrapy.Spider’.
2. We’re giving it a name, ‘myspider’.
3. We’re defining a start_requests() method that will tell Scrapy what to do when it starts crawling.
4. We’re creating a Request object and passing it the URL we want to scrape.
5. We’re passing the Request object to the parse() method.
6. We’re defining a parse() method that will tell Scrapy what to do when it receives a response from a Request.
7. We’re printing the URL of the response.