Shopify stores expose their full product catalog through a built-in /products.json endpoint – prices, variants, availability, and images, all as clean JSON. In this tutorial, you'll learn how to scrape it with Python, paginate through the full catalog, deal with stores that disable the endpoint, and add Decodo residential proxies for multi-store scraping at scale.
What's covered:
• How the /products.json endpoint works and what fields it returns
• Confirming a site runs on Shopify before scraping
• Paginating through 250-product pages to get the full catalog
• Fallback method: XML sitemap + JSON-LD for stores that disable the endpoint
• Adding Decodo residential proxies for multi-store scraping
• Saving output as JSON and CSV
⏰ Timestamps:
00:00 Why scrape Shopify stores
00:32 Use cases for Shopify data
00:42 Getting started
01:05 Installing the libraries
01:09 Why curl_cffi over Requests
01:31 How the script works
02:21 Fallback method for blocked stores
02:54 Running the script
03:02 Output files (JSON + CSV)
03:23 Full code and blog post
03:35 Outro
FAQ
Does every Shopify store have a /products.json endpoint?
Most public stores do, but some merchants disable it. The script detects this and falls back to sitemap + JSON-LD extraction automatically.
Do I need proxies for this?
For single-store, low-frequency scraping — probably not. For monitoring multiple stores on a schedule, rotating residential proxies help avoid rate limits and blocks.
What Python libraries does this use?
curl_cffi for browser TLS fingerprinting and BeautifulSoup for HTML parsing in the fallback method. Both are installable via pip.
Let's connect on other platforms!
🔹 Linked.in: linkedin.com/company/decodo
🔹 Discord community: discord.gg/gvJhWJPaB4
🔹 GitHub: github.com/decodo
Need some direct support?
🔹 For sales queries, email: sales@decodo.com
🔹 24/7 live customer support: direct.lc.chat/12092754