#python,#webscraping
This tutorial aims to give an overview of the process of a web scraping project for a real-world business by briefly explaining what web scraping is, why and who uses web scraping.
With a brief overview of the legality of web scraping and who to ensure you are scaping legally.
We then demonstrate how one can quickly build and deploy a use case project. In this case, it will be a company researching commercial lettings in Dublin.
We will look at a common problem people face that is the website is rendered in javascript and how to overcome it.
The process of research and construction of xpath's will be reviewed.
The final result is an excel spreadsheet containing information on Dublin's commercial renting.
TimeStamp:
Timestamps
What is scrapy 0:25
Who uses scrapy 1:05
Legal 1:29
Envirment set up 2:17
Acquiring xpaths 4:22
Code overview 5:25
Acquiring remaing xpath 6:32
Running the project 9:15
Final data 9:32
Resources:
technologies: