Create an account to use AI

Create a free account to unlock all features:

Ask questions about videos with AI
Use smart search (AI)
Save your favorite videos
Bookmark your favorite categories
Organize your favorite subcategories
Access your personalized favorites page
Access your conversation history
Personalize your homepage with your favorites

Already have an account? Log in

Use multiple concurrent threads to increase scraping speed | Scraper API

Name: Use multiple concurrent threads to increase scraping speed | Scraper API
Uploaded: 2022-06-22T10:00:29+00:00
Duration: 18 min 16 s

jobstr

◼️ Faster Web Scraping with Python Concurrent Threads | Scraper API ◼️

◼️ Scraper API demonstration + concurrent.futures — Launching parallel tasks ◼️

🌏 10% off ScraperAPI : https://www.scraperapi.com?fpr=ken49

◼️ Coupon Code: DRPI10

(You can also get started with 5000 free API calls. No credit card required.)

*Easily scrape any site with JS rendering, geotargeting or residential proxies*

Developer Guides : 🔗 https://www.scraperapi.com/guides/

Scraper API playlist : https://youtube.com/playlist?list=PLKMY3XNPiQ7v_n_tc7KRhYlR8-lVL2LPU

-----------------------------------------------------------------

- chapters -

00:00 intro - what is a thread?

03:35 concurrent.futures

05:30 python.org example

09:58 Scraper API example

15:10 executor objects submit and map

----------------------------------------------------------------

🔗 https://docs.python.org/3/library/concurrent.futures.html#module-concurrent.futures

Examples from this video :

🔗 https://github.com/RGGH/Python/tree/master/ScraperAPI_Examples

👍 Become a patron 👍

🌏 https://www.patreon.com/drpi

Buy Dr Pi a coffee (or Tea)

☕ https://www.buymeacoffee.com/DrPi

Proxies

=================================================

If you need a good, easy to use proxy, I was recommended this one, and having used ScraperAPI for a while I can vouch for them. If you were going to sign up anyway, then maybe you would be kind enough to use the link and the coupon code below?

You can also do a full working trial first as well, (unlike some other companies). The trial doesn't ask for any payment details either so all good! 👍

🌏 10% off ScraperAPI : https://www.scraperapi.com?fpr=ken49

◼️ Coupon Code: DRPI10

(You can also get started with 1000 free API calls. No credit card required.)

Thumbs up yeah? (cos Algos..)

#webscraping #concurrentthreads #scraperapi