Create an account to use AI

Create a free account to unlock all features:

Ask questions about videos with AI
Use smart search (AI)
Save your favorite videos
Bookmark your favorite categories
Organize your favorite subcategories
Access your personalized favorites page
Access your conversation history
Personalize your homepage with your favorites

Create an account

Already have an account? Log in

Scraping Secret Sauce: Reverse Engineering API

Geraldus Wilsen

In this video, we're going to dive deeper into scraping using Reverse Engineering API

00:00:00 - Introduction

00:01:05 - Scraping Scenario (Case Study)

00:02:11 - Explanation of How Data Is Displayed on a Website (Front-End and Back-End)

00:04:21 - Biggest Mistake in Scraping

00:05:49 - Code Overview

00:06:15 - Scraping Ethics

00:06:45 - Finding the Hidden API

00:08:57 - Retrieving Data Using the Basic Request Library

00:09:46 - TLS Fingerprinting

00:10:46 - Setting Request Headers

00:12:41 - Processing Data Using Pydantic

00:14:38 - Introduction to Camoufox

00:15:59 - Creating a Synchronous Scraper

00:22:27 - Explanation: Sync vs Async

00:23:57 - I/O Bound vs CPU Bound

00:25:06 - Optimizing the Scraper Using Async and Parallel Processing

00:28:26 - Outro

Github: https://github.com/projectwilsen/scraping

#scraping #python #data

Scraping Secret Sauce: Reverse Engineering API | Knoovi