In this Python web scraper tutorial I use Python 3 and the Python requests package, to scrape a list of online users from a chat application. In my next upcoming video I will teach you how to scrape Amazon so make sure you hit that subscribe button to stay updated.
To follow along with this tutorial you must have the following installed:
I demonstrate the use of sessions, and cookies, using the python requests package and show you how easy it is to make requests to applications that require login. I also breifly demonstrate how to parse the data returned using JSON.
If you like this video and would like to see more videos and tutorials on web scraping please show some support by giving me a thumbs up and subscribing. Leave a comment with any questions, or requests for future videos.
What is Web scraping ?
Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. Web scraping software may access the World Wide Web directly using the Hypertext Transfer Protocol, or through a web browser. While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a bot or web crawler. It is a form of copying, in which specific data is gathered and copied from the web, typically into a central local database or spreadsheet, for later retrieval or analysis.
Web Scraping Playlist:
Other Web Scraping videos:
Web Scraping : Web Scraping with Python by BangPypers
Web Scraping : Web Scraping with Python by M.Yasoob Khalid
Web Scraping : Web Scraping with Python by Nikola Milojica
Web Scraping : Powerful Web Scraping and Searching with Python by Michael Ruegg
Web Scraping : Steps to perform Web Scraping with Python
Web Scraping : Python Web Scraping Tools by Singapore Python User Group
Web Scraping : Sessions, Requests, Cookies & JSON
Web Scraping : using requests to pass search parameters to craigslist
Web Scraping : using requests to scrape a website for data
Web Scraping : Web scraping with Python using Requests and LXML
Web Scraping : Python Async basics video (100 million HTTP requests)
Web Scraping : How do you scrape behind login auth page using Python Requests
Web Scraping : How to download All WordPress Media
Web Scraping : Quick and dirty web scraping with Python
Web Scraping : Using Python to Buy Avengers Tickets First!
Web Scraping : Writing an Email Address Scraper in Python
CURL:
Web Scraping : URL Requests with cURL and Chrome Console
Web Scraping : Scraping Flickr with cURL for Fun Tutorial
Celery:
Web Scraping : Scraping w/ Celery
Scrapy:
Web Scraping : Introduction to Web Scraping using Scrapy
Web Crawler:
Web Scraping : Python Web Crawler
Pandas:
Web Scraping : Python Pandas: storing data into a Pandas data-frame
Web Scraping : Easily extract tables from websites with pandas and python
Web Scraping : Extract tabular data from PDF with Python - Tabula, Camelot, PyPDF2
XPATH:
Web Scraping : Python + XPath = Extra Parsing Power
Re Module:
Web Scraping : using python re module to process text files part1
Web Scraping : using python re module to process text files part2
API:
Web Scraping : Moving Average Calculation using Python