#add this lines into settings.py DOWNLOADER_MIDDLEWARES = { 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None, 'scrapy_user_agents.middlewares.RandomUserAgentMiddleware': 400, }
Here is what the above code is Doing:
1. We are importing the RandomUserAgentMiddleware class from the scrapy_user_agents.middlewares module.
2. We are setting the DOWNLOADER_MIDDLEWARES setting to a dictionary.
3. We are disabling the default UserAgentMiddleware by setting it to None.
4. We are adding the RandomUserAgentMiddleware to the DOWNLOADER_MIDDLEWARES setting.