Python, NoSQL & FastAPI Tutorial: Web Scraping on a Schedule
In this series you'll learn how to:
- Integrate Cassandra with Python, FastAPI, & Celery
- Setup & Configure Astra DB (managed cassandra db)
- Schedule & Offload Tasks with Celery
- Web Scraping Techniques
- Selenium & requests-html to extract & parse data
Links
0:00:00 Welcome
00:00:58 Demo
00:12:58 Overview & Requirements
00:15:22 Project Setup
00:19:26 Start the Python & Cassandra Integration
00:25:11 Configure Python cassandra-driver
00:30:13 Your First Cassandra Model
00:36:08 Create Data using our Cassandra Model
00:43:20 Adding a New Column to an Existing Model
00:46:26 Using UUID1 as Primary Key
00:55:19 Using Jupyter with Cassandra Models
01:06:35 Using Pydantic for Data Validation and Cleaning
01:14:22 FastAPI & Environment Variables
01:21:26 FastAPI + Cassandra & Pydantic
01:36:01 Convert Cassandra UUID Field to Pydantic Datetime Strv
01:44:50 Endpoint to Ingest Data for FastAPI & AstraDB
01:56:41 Celery, Redis & Basic Task Offload
02:13:18 Integrate Cassandra Driver with Celery
02:23:47 Running Periodic Tasks
02:35:24 Basic Scraping with Selenium
02:45:35 Selenium & JavaScript Endless Scrolling
02:52:52 requests-html & Parsing Data
03:11:34 Implement the Scrape Client Parser
03:27:20 Putting it all together
03:36:58 Thank you