Hello Everyone! My name is Andrew Fung, in this video, I will be showing you how to automate your everyday web scraping tasks with beautiful soup and schedule it to run on a regular basis with cron jobs on Mac. In this second episode, I will be firstly using the web scraping script that we have written in the first episode, then create a function for sending email notifications to our mailboxes, and finally, setting up cron jobs to run this task periodically and on a regular basis.
Problem statement: I am currently on a waitlist in my uni's hall application, and my waitlist can be found from the university webpage's pdf link, which will be updated at the same time every week. Thus, I will run a cron job to fetch my waitlist position one hour after the weekly update from the web page and send me an email of my new position.
#taskautomation #cronjob #python #webscraping #beautifulsoup
How I make my YouTube videos:
Installation and Setup!
GMail App passwords:
Potential error and solution:
Crontab scheduling and naming conventions:
Source code for this project:
Check out my Github!
Timestamps
-------------------------------
0:00 | Intro
2:58 | Run Part 1 Locally
4:41 | Creating a Virtual Environment
7:50 | Send Email with smtp
15:11 | Setting Up Cron Jobs
24:55 | Out tro
-------------------------------
Feel free to drop a like and comment if you enjoy and video and let me know if you want me to do other types of programming videos ;) !!!