Modern Sites
SPA / React / Vue Targets
Pages where the data only materializes after JavaScript runs and React Query / fetch calls resolve. Playwright handles the wait-for-content side natively.
// Topic
Playwright Automation for Modern Web Targets
Playwright is the modern default for browser automation: faster than Selenium, better selectors, real network control, and first-class support for the kind of evasion tooling (Kameleo, proxies) that production scraping needs. This topic covers how I use it.
What This Covers
Playwright is what I reach for whenever a job needs a real browser. Compared to Selenium, it's faster, has better selector engines (text-based, role-based, layout-based), gives proper async control, exposes the network layer for request interception, and integrates cleanly with anti-detect tooling like Kameleo. For ~95% of modern browser-automation work, it's the right tool.
This topic covers how I structure Playwright code for production: worker pools, defensive selectors, network interception, proxy integration, error capture, and the operational patterns that make a Playwright pipeline stable enough to run unattended for hours.
When You'd Need This
Network Interception
Catch the JSON, Skip the DOM
Many "scrape this site" problems are really "intercept this XHR response" problems. Playwright lets you sniff the JSON the page is fetching internally — much faster and more stable than parsing rendered HTML.
Anti-Detect
Attach to Kameleo / Browser Profiles
Playwright connects cleanly to a Kameleo browser via CDP (Chrome DevTools Protocol). Real fingerprint + real proxy, scripted by Playwright on top.
Workflows
Multi-Step Logged-In Actions
Order processing, support workflows, account onboarding — anything that requires session state, navigation, form filling, and reading back results.
E2E Testing
End-to-End Verification
Real-browser tests against real environments. Catch regressions that unit tests miss because they test against the actual rendered page.
Scale
Parallel Worker Pools
Multiple Playwright instances coordinated by a queue, partitioned across proxies, all writing to a shared database — the pattern behind most of my production scraping systems.
How I Approach It
Selectors first. Playwright's role-based and text-based selectors (page.get_by_role(), page.get_by_text()) are dramatically more stable than CSS selectors against most modern sites — they break less when the styling changes, and they document intent. Where I have to fall back to CSS or XPath, I keep the selector list in one place per page-object so updates are localized.
Wait strategy is opinionated: wait_for_load_state("networkidle") by default, with explicit wait_for_selector() on the specific element I'm about to interact with. Hard sleeps are a code smell — every one of them gets replaced with a real wait condition before the code ships.
Network interception is the secret weapon. Many sites render data through XHR calls that return clean JSON. Playwright's page.on("response") handler lets you grab that JSON directly, which is faster, more stable, and more accurate than parsing the rendered HTML. Whenever I see a page I'm about to scrape, the first thing I check is the network tab.
For high-stakes targets with bot protection, Playwright attaches to a Kameleo profile via CDP. Kameleo handles the fingerprint and the proxy; Playwright handles the scripting on top. The two layers are clean and separable — the Playwright code doesn't know or care that there's a Kameleo session underneath.
Concurrency: one Playwright process per worker, workers in a multi-process pool, queue partitioning so workers never collide. Errors capture a screenshot + the page HTML + a network log to disk before the error is recorded — debugging a flaky Playwright workflow without those artifacts is miserable.
Typical Stack
- Playwright (Python)
- Async + sync APIs
- Kameleo (via CDP)
- Page-object pattern
- Network response interception
- Multi-process worker pool
- SQLite (WAL) / MySQL queue
- Screenshot + HTML on error
- Rotating proxy pools
- Structured logging
Case Studies Using Playwright
Case Study
Tariff-Exposure Data Pipeline
6-worker Playwright + Kameleo fetch layer with block-only proxy rotation, surviving Cloudflare across multi-hour runs.
Read the case study →Case Study
AI Customer Service Automation
Playwright drives live customer service chat sessions inside isolated Kameleo profiles, with structured outcome capture.
Read the case study →Related Topics
Playwright work usually overlaps with Browser Automation, Kameleo Automation, and Undetectable Browser Automation.
// Let's Build
Need a Playwright-Based System Built?
From single-purpose scrapers to large parallel worker pools, I build Playwright automation that holds up against real-world site behavior, bot protection, and operational scale.