Create an account to use AI

Create a free account to unlock all features:

Ask questions about videos with AI
Use smart search (AI)
Save your favorite videos
Bookmark your favorite categories
Organize your favorite subcategories
Access your personalized favorites page
Access your conversation history
Personalize your homepage with your favorites

Create an account

Already have an account? Log in

Web scraping data with n8n and Puppeteer

Name: Web scraping data with n8n and Puppeteer
Uploaded: 2023-07-02T08:19:45+00:00
Duration: 9 min 27 s

Oskar

This tutorial shows two methods on how to scrape data from websites with n8n and Puppeteer.

In this tutorial I cover:

1. Data scraping in n8n using HTTP request node

2. Retrieving email addresses from scraped source code

3. Key features of Puppeteer and writing basic script

4. Deploying Puppeteer function to Google Cloud Functions

5. Using Puppeteer in n8n via Google Cloud Functions

Resources:

1. Puppeteer script (for local usage): https://github.com/workfloows/workfloows-tutorials/blob/7d8c4d1f4c90a1118848caa89be696f8509c1691/scrape-data-n8n-puppeteer/puppeteer-script/index.js

2. Puppeteer script (for Google Cloud Functions): https://github.com/workfloows/workfloows-tutorials/tree/7d8c4d1f4c90a1118848caa89be696f8509c1691/scrape-data-n8n-puppeteer/puppeteer-script/google-cloud-functions-adjusted

3. Function to retrieve email addresses from string in n8n: https://github.com/workfloows/workfloows-tutorials/blob/7d8c4d1f4c90a1118848caa89be696f8509c1691/scrape-data-n8n-puppeteer/retrieve-email-addresses.js

My other tutorials:

1. Build undetectable Amazon scraper with n8n, Puppeteer and Scraping Browser: https://youtu.be/ArRMJhdEweg

2. How to build Telegram AI bot with n8n and 🦜🔗 LangChain (FlowiseAI): https://youtu.be/Gc2lW5BiGDQ

3. How to build voice Telegram AI bot with n8n, Whisper and ElevenLabs: https://workfloows.gumroad.com/l/telegram-ai-voice-bot

4. Generate PDFs in n8n with Gotenberg (for free!): https://www.youtube.com/watch?v=bo15xdjXf1Y

5. Using webhooks in n8n (parameters, responses and triggers): https://www.youtube.com/watch?v=IvUYJQkf6sA

Create your n8n cloud account here (affiliate): https://app.n8n.cloud/register?ref=workfloows&utm_source=affiliate

Subscribe my newsletter: https://workfloows.com/

Visit my Gumroad profile: https://workfloows.gumroad.com/

Follow me on Twitter/X: https://twitter.com/workfloows

Follow Workfloows on LinkedIn: https://www.linkedin.com/company/workfloows/

Screen recording software that I use (affiliate): https://www.screen.studio/@jmMwX

Disclaimer: Please note that web scraping should be performed only on websites that explicitly grant permission. I cannot be held responsible for any actions or consequences resulting from the use of the information provided in this video. Please make sure you have proper authorization before engaging in web scraping activities.

0:00 Introduction

0:20 Step 1: Basic scrape with HTTP request

2:51 Step 2: Prepare Puppeteer script

5:28 Step 3: Adjust code to Google Cloud Functions

6:06 Step 4: Deploy function to Google Cloud Functions

8:16 Step 5: Scrape data with Puppeteer in n8n