Create an account to use AI

Create a free account to unlock all features:

Ask questions about videos with AI
Use smart search (AI)
Save your favorite videos
Bookmark your favorite categories
Organize your favorite subcategories
Access your personalized favorites page
Access your conversation history
Personalize your homepage with your favorites

Create an account

Already have an account? Log in

Scrape Infinite Scroll Pages with Python Scrapy

Code [RE] Code

Download Source Code: https://coderecode.com/scroll_code_yt

This video will show you how you can examine a page with infinite scrolling, get the URL that matters, and scrape it.

-----------------

Watch the Playlist to Learn the Basics of Scrapy: https://www.youtube.com/playlist?list=PLj4hN6FewnwrimhTJzAtKz22t1DWQa57q

Take the full crash course:

https://coderecode.com/scrapy-crash-course?src=yt

What is Web Scraping?

In a nutshell: Web Scraping = Getting Data from Websites with Code

What is Scrapy?

Scrapy is a Python library to make web scraping very powerful, fast, and efficient.

There are other libraries too like BeautifulSoup, for web scraping. However, when it comes to true power and flexibility, Scrapy is the most powerful.

Why Learn Scrapy?

- Most powerful library for scraping

- Easy to master

- Cross-platform: doesn't matter which OS you are using

- Cloud-ready: Can be run on the cloud with a free account

Most Important: You would be able to earn by taking up some of the web scraping as a freelancer

-~-~~-~~~-~~-~-

Please watch: "Making Scrapy Playwright fast and reliable"

https://www.youtube.com/watch?v=2UtLq7-rqeE

-~-~~-~~~-~~-~-

Scrape Infinite Scroll Pages with Python Scrapy | Knoovi