Part 4 of our latest Scrapy project / Web Scraping Tutorial. Fully working spider.
Web Scraping a large site with "infinite scroll" aka "AJAX load more".
(This is based on a real world project from Upwork).
*** PART 7 of 7 ***
This video covers adding the "name" value to the CSV, some fine tuning, and checking the output CSV.
Please comment, subscribe, like, as this is taking a lot of effort and the more comments/feedback the better we can tailor future videos to you.
GitHub Code
Cheers,
Dr Pi.
#scrapy #infinitescroll #loadmore