{"id":1249,"date":"2026-04-15T19:03:51","date_gmt":"2026-04-15T19:03:51","guid":{"rendered":"https:\/\/aharonyan.org\/?p=1249"},"modified":"2026-04-15T19:39:13","modified_gmt":"2026-04-15T19:39:13","slug":"bot-resistant-e2e-tests-shopify-playwright","status":"publish","type":"post","link":"https:\/\/aharonyan.org\/?p=1249","title":{"rendered":"Building Bot-Resistant E2E Tests for Shopify Stores with Python, Playwright, and SeleniumBase"},"content":{"rendered":"\n

You point Playwright at a Shopify store fronted by Cloudflare. You get a 403 or a permanent challenge page. You try headless=False<\/code>. Same result. You add a user agent. Same result. You add a residential proxy. Now you get rate-limited instead of blocked, which is progress of a sort.<\/p>\n\n\n\n

This is a build log for the stack that got us past all of that \u2014 about 300 lines of Python and one GitHub Actions file, running ~10 checkout and catalog flows against a live store every day without tripping.<\/p>\n\n\n\n

1. The problem<\/h2>\n\n\n\n

Four things are fighting you, and they compound:<\/p>\n\n\n\n

Your Chrome looks wrong.<\/strong> Vanilla Playwright ships a Chromium build with navigator.webdriver = true<\/code>, missing chrome.runtime<\/code>, mismatched Navigator.permissions<\/code>, and a suspicious WebGL vendor string. Cloudflare’s challenges.cloudflare.com\/turnstile<\/code> runs a few hundred of these checks in under a second. You lose before your first page.goto<\/code>.<\/p>\n\n\n\n

Your mouse doesn’t exist.<\/strong> A real user lands somewhere random on a button, often drifts past it, hovers for 200ms, then clicks. Playwright’s .click()<\/code> teleports the cursor to the exact center of the element’s bounding box and fires a mousedown<\/code> in the same frame. Bot detection systems have been pattern-matching that for years.<\/p>\n\n\n\n

Your IP is hot.<\/strong> Datacenter IPs from AWS, GCP, Hetzner \u2014 all pre-flagged. Even free residential proxies arrive pre-burned because ten thousand other scrapers used them yesterday.<\/p>\n\n\n\n

Your rotation is worse than your static IP.<\/strong> The naive fix \u2014 rotate the proxy on every request \u2014 is what tips Cloudflare from “mildly suspicious” to “definitely a bot.” Cloudflare’s cf_clearance<\/code> cookie is bound to the IP it was issued to. Rotating nukes the clearance every time.<\/p>\n\n\n\n

The stack below addresses all four.<\/p>\n\n\n\n

2. The architecture<\/h2>\n\n\n\n
\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502  pytest + Playwright sync API                               \u2502\n\u2502     \u2502                                                       \u2502\n\u2502     \u2502  (CDP over ws:\/\/127.0.0.1:<port>)                     \u2502\n\u2502     \u25bc                                                       \u2502\n\u2502  SeleniumBase (uc=True) Chrome \u2014 stealth fingerprint        \u2502\n\u2502     \u2502                                                       \u2502\n\u2502     \u2502  (HTTP CONNECT)                                       \u2502\n\u2502     \u25bc                                                       \u2502\n\u2502  Local auth-bridge proxy  127.0.0.1:18888                   \u2502\n\u2502     \u2502                                                       \u2502\n\u2502     \u2502  (basic auth injected)                                \u2502\n\u2502     \u25bc                                                       \u2502\n\u2502  Webshare static residential proxy                          \u2502\n\u2502  (auto-replaced on health signals \u2014 unavailable >15min,     \u2502\n\u2502  low country confidence, slowdown \u2014 via dashboard           \u2502\n\u2502  Replace Proxies setting)                                   \u2502\n\u2502     \u2502                                                       \u2502\n\u2502     \u25bc                                                       \u2502\n\u2502  Shopify store (behind Cloudflare)                          \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518<\/pre>\n\n\n\n
A few deliberate choices here:<\/p>\n\n\n\n
    \n
  • SeleniumBase drives Chrome, Playwright drives the test.<\/strong> SeleniumBase’s uc=True<\/code> mode handles undetected-chromedriver patching, argument scrubbing, and CDP flags. Once Chrome is up, we connect Playwright to the existing browser over CDP and throw the SeleniumBase driver away except for its Cloudflare solver. The test code gets Playwright’s much nicer API.<\/li>\n\n\n\n
  • Static residential, with provider-side auto-replace.<\/strong> One Webshare residential endpoint. The cf_clearance<\/code> cookie sticks to it for the lifetime of a session. Rotation happens out of band at the dashboard layer, not per-request from the client.<\/li>\n\n\n\n
  • Local auth bridge.<\/strong> Chrome cannot handle HTTP proxy auth via CDP. You cannot pass user:pass@host:port<\/code> through --proxy-server<\/code>. The workaround is a tiny local forwarder that listens on 127.0.0.1<\/code> unauthenticated, injects basic auth, and forwards to Webshare.<\/li>\n\n\n\n
  • Two-tier Cloudflare bypass.<\/strong> SeleniumBase has a free built-in Turnstile clicker (sb.uc_gui_click_captcha()<\/code>). It works maybe 70% of the time. For the other 30%, we call CapSolver’s AntiCloudflareTask<\/code> and inject the returned cf_clearance<\/code>. CapSolver is one option; 2Captcha, Anti-Captcha, and a few others expose equivalent APIs.<\/li>\n<\/ul>\n\n\n\n
    3. Setup walkthrough<\/h2>\n\n\n\n
    Dependencies<\/h3>\n\n\n\n
    # requirements.txt\nseleniumbase==4.32.7\nplaywright==1.47.0\npytest==8.3.3\npytest-rerunfailures==14.0\npython-dotenv==1.0.1\nrequests==2.32.3<\/code><\/pre>\n\n\n\n
    pip install -r requirements.txt\nplaywright install chromium<\/code><\/pre>\n\n\n\n
    Environment<\/h3>\n\n\n\n
    # .env\nCAPSOLVER_API_KEY=CAP-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\nWEBSHARE_PROXY=user-session-abc123:password@p.webshare.io:80<\/code><\/pre>\n\n\n\n
    Only one proxy, one key. If either is missing, the fixtures skip the run with a clear message instead of falling back to a clean browser and pretending everything is fine.<\/p>\n\n\n\n
    The local auth-bridge proxy (appendix, but you need it)<\/h3>\n\n\n\n
    Chrome can’t do HTTP CONNECT with basic auth via command-line flags. This small forwarder sits between Chrome and Webshare and adds the Proxy-Authorization<\/code> header for you:<\/p>\n\n\n\n
    # tests\/support\/auth_proxy.py\nimport base64\nimport select\nimport socket\nimport threading\n\n\ndef start_auth_proxy(upstream_host: str, upstream_port: int,\n                     username: str, password: str,\n                     listen_port: int = 18888) -> threading.Thread:\n    creds = base64.b64encode(f\"{username}:{password}\".encode()).decode()\n\n    def handle(client: socket.socket) -> None:\n        try:\n            request = b\"\"\n            while b\"\\r\\n\\r\\n\" not in request:\n                chunk = client.recv(4096)\n                if not chunk:\n                    return\n                request += chunk\n\n            # inject Proxy-Authorization if missing\n            if b\"Proxy-Authorization:\" not in request:\n                headers_end = request.find(b\"\\r\\n\\r\\n\")\n                injected = (\n                    request[:headers_end]\n                    + f\"\\r\\nProxy-Authorization: Basic {creds}\".encode()\n                    + request[headers_end:]\n                )\n                request = injected\n\n            upstream = socket.create_connection((upstream_host, upstream_port))\n            upstream.sendall(request)\n            pipe(client, upstream)\n        except Exception:\n            pass\n        finally:\n            client.close()\n\n    def pipe(a: socket.socket, b: socket.socket) -> None:\n        sockets = [a, b]\n        while True:\n            r, _, _ = select.select(sockets, [], [], 30)\n            if not r:\n                break\n            for s in r:\n                data = s.recv(8192)\n                if not data:\n                    return\n                (b if s is a else a).sendall(data)\n\n    def serve() -> None:\n        server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n        server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)\n        server.bind((\"127.0.0.1\", listen_port))\n        server.listen(16)\n        while True:\n            client, _ = server.accept()\n            threading.Thread(target=handle, args=(client,), daemon=True).start()\n\n    t = threading.Thread(target=serve, daemon=True)\n    t.start()\n    return t<\/code><\/pre>\n\n\n\n
    It’s sixty lines, it handles CONNECT tunnelling, and you’ll never think about it again.<\/p>\n\n\n\n
    conftest.py \u2014 the real work<\/h3>\n\n\n\n
    # tests\/conftest.py\nimport os\nimport random\nimport re\nimport time\nfrom dataclasses import dataclass\nfrom pathlib import Path\nfrom typing import Any\n\nimport pytest\nimport requests\nfrom dotenv import load_dotenv\nfrom playwright.sync_api import Page, BrowserContext, expect, sync_playwright\nfrom seleniumbase import SB\n\nfrom tests.support.auth_proxy import start_auth_proxy\n\nload_dotenv()\n\nSCREENSHOT_ROOT = Path(\"latest_logs\/screenshots\")\nLOCAL_PROXY_PORT = 18888\n\n\n@dataclass(frozen=True)\nclass StealthPage:\n    sb: Any\n    page: Page\n    context: BrowserContext\n    proxy_info: dict\n\n\ndef _parse_proxy(raw: str) -> dict:\n    # user:pass@host:port\n    auth, host = raw.split(\"@\", 1)\n    user, pw = auth.split(\":\", 1)\n    hostname, port = host.split(\":\", 1)\n    return {\"host\": hostname, \"port\": int(port), \"user\": user, \"pass\": pw}\n\n\ndef bypass_cloudflare(sb: Any, page: Page, target_url: str,\n                      proxy_info: dict) -> None:\n    challenge_markers = [\n        \"challenge-platform\",\n        \"cf-challenge-running\",\n        \"Just a moment\",\n    ]\n    title = page.title() or \"\"\n    html = page.content()\n    challenged = any(m in title or m in html for m in challenge_markers)\n    if not challenged:\n        return\n\n    # Tier 1 \u2014 SeleniumBase free solver\n    try:\n        sb.uc_gui_click_captcha()\n        page.wait_for_timeout(4000)\n        if \"challenge-platform\" not in page.content():\n            return\n    except Exception:\n        pass\n\n    # Tier 2 \u2014 CapSolver AntiCloudflareTask\n    api_key = os.environ[\"CAPSOLVER_API_KEY\"]\n    create = requests.post(\n        \"https:\/\/api.capsolver.com\/createTask\",\n        json={\n            \"clientKey\": api_key,\n            \"task\": {\n                \"type\": \"AntiCloudflareTask\",\n                \"websiteURL\": target_url,\n                \"proxy\": f\"http:{proxy_info['host']}:{proxy_info['port']}\"\n                         f\":{proxy_info['user']}:{proxy_info['pass']}\",\n            },\n        },\n        timeout=30,\n    ).json()\n    task_id = create.get(\"taskId\")\n    if not task_id:\n        raise RuntimeError(f\"CapSolver createTask failed: {create}\")\n\n    deadline = time.time() + 180\n    while time.time() < deadline:\n        time.sleep(4)\n        result = requests.post(\n            \"https:\/\/api.capsolver.com\/getTaskResult\",\n            json={\"clientKey\": api_key, \"taskId\": task_id},\n            timeout=30,\n        ).json()\n        if result.get(\"status\") == \"ready\":\n            solution = result[\"solution\"]\n            clearance = solution[\"cookies\"][\"cf_clearance\"]\n            ua = solution[\"userAgent\"]\n            page.context.add_cookies([{\n                \"name\": \"cf_clearance\",\n                \"value\": clearance,\n                \"domain\": solution.get(\"domain\", \".shopify.com\"),\n                \"path\": \"\/\",\n                \"secure\": True,\n                \"httpOnly\": True,\n            }])\n            page.context.set_extra_http_headers({\"User-Agent\": ua})\n            page.reload(wait_until=\"domcontentloaded\")\n            return\n    raise RuntimeError(\"CapSolver timed out after 180s\")\n\n\n@pytest.fixture(scope=\"function\")\ndef stealth_page(request):\n    raw_proxy = os.environ.get(\"WEBSHARE_PROXY\")\n    if not raw_proxy:\n        pytest.skip(\"WEBSHARE_PROXY not set\")\n    proxy_info = _parse_proxy(raw_proxy)\n    start_auth_proxy(\n        upstream_host=proxy_info[\"host\"],\n        upstream_port=proxy_info[\"port\"],\n        username=proxy_info[\"user\"],\n        password=proxy_info[\"pass\"],\n        listen_port=LOCAL_PROXY_PORT,\n    )\n    time.sleep(0.3)  # let the listener bind\n\n    with SB(uc=True, headless=False,\n            proxy=f\"127.0.0.1:{LOCAL_PROXY_PORT}\") as sb:\n        cdp_port = sb.driver.capabilities[\"goog:chromeOptions\"][\"debuggerAddress\"]\n        with sync_playwright() as pw:\n            browser = pw.chromium.connect_over_cdp(f\"http:\/\/{cdp_port}\")\n            context = browser.contexts[0]\n            page = context.pages[0] if context.pages else context.new_page()\n            yield StealthPage(sb=sb, page=page,\n                              context=context, proxy_info=proxy_info)\n\n\n@pytest.fixture\ndef step(request):\n    test_slug = re.sub(r\"[^a-z0-9]+\", \"_\", request.node.name.lower()).strip(\"_\")\n    folder = SCREENSHOT_ROOT \/ test_slug\n    folder.mkdir(parents=True, exist_ok=True)\n    counter = {\"n\": 0}\n\n    def _step(name: str, page: Page) -> None:\n        counter[\"n\"] += 1\n        n = counter[\"n\"]\n        slug = re.sub(r\"[^a-z0-9]+\", \"_\", name.lower()).strip(\"_\")\n        print(f\"\\n===== STEP {n:02d}: {name} =====\", flush=True)\n        page.screenshot(path=str(folder \/ f\"{n:02d}_{slug}.png\"),\n                        full_page=True)\n\n    return _step\n\n\n@pytest.fixture\ndef random_sleep():\n    def _sleep(page: Page, lo: float = 2.0, hi: float = 5.0) -> None:\n        close_cookie_popup(page)\n        page.wait_for_timeout(int(random.uniform(lo, hi) * 1000))\n    return _sleep\n\n\ndef close_cookie_popup(page: Page) -> None:\n    selectors = [\n        'button:has-text(\"Accept\")',\n        'button:has-text(\"Accept all\")',\n        'button:has-text(\"Alle akzeptieren\")',\n        '[aria-label*=\"accept\" i]',\n        '#onetrust-accept-btn-handler',\n    ]\n    for sel in selectors:\n        try:\n            btn = page.locator(sel).first\n            if btn.is_visible(timeout=500):\n                btn.click(timeout=1000)\n                return\n        except Exception:\n            continue\n    # JS fallback \u2014 some Shopify apps refuse to respect the click\n    try:\n        page.evaluate(\"\"\"\n            () => {\n                const nodes = document.querySelectorAll(\n                    'button, [role=\"button\"]'\n                );\n                for (const n of nodes) {\n                    const t = (n.innerText || '').toLowerCase();\n                    if (t.includes('accept') || t.includes('akzeptieren')) {\n                        n.click();\n                        return;\n                    }\n                }\n            }\n        \"\"\")\n    except Exception:\n        pass\n\n\n@pytest.fixture\ndef human_click():\n    def _click(page: Page, selector: str) -> None:\n        locator = page.locator(selector).first\n        locator.scroll_into_view_if_needed()\n        box = locator.bounding_box()\n        if box is None:\n            locator.click()\n            return\n        # land somewhere off-center\n        fx = random.uniform(0.30, 0.70)\n        fy = random.uniform(0.30, 0.70)\n        tx = box[\"x\"] + box[\"width\"] * fx\n        ty = box[\"y\"] + box[\"height\"] * fy\n        steps = random.randint(8, 18)\n        page.mouse.move(tx, ty, steps=steps)\n        page.wait_for_timeout(random.randint(120, 420))  # hover\n        page.mouse.click(tx, ty)\n    return _click<\/code><\/pre>\n\n\n\n
    A few things worth calling out:<\/p>\n\n\n\n
    The StealthPage<\/code> dataclass is frozen. You cannot reassign .page<\/code> halfway through a test and confuse yourself. Tests that need the raw SeleniumBase driver still have it via stealth_page.sb<\/code>; tests that only need Playwright use stealth_page.page<\/code>.<\/p>\n\n\n\n
    bypass_cloudflare<\/code> is a no-op when there’s no challenge. You can call it freely after every navigation without paying for a CapSolver task. Only an actual challenge page triggers the paid path.<\/p>\n\n\n\n
    close_cookie_popup<\/code> runs inside random_sleep<\/code>. Cookie dialogs show up unpredictably \u2014 after the first paint, after a GDPR geolocation check, sometimes after the first scroll. Tying it to the sleep rhythm means you don’t have to think about it in test code.<\/p>\n\n\n\n
    4. The test-writing pattern<\/h2>\n\n\n\n
    Before the first test, lock prices and selectors in frozen dataclasses. When the store later moves to an API-driven price, you swap the dataclass; the tests don’t move.<\/p>\n\n\n\n
    # tests\/fixtures\/catalog.py\nfrom dataclasses import dataclass\n\n\n@dataclass(frozen=True)\nclass Product:\n    slug: str\n    title: str\n    price_eur: str  # \"29.95\" \u2014 compare as string to avoid float drift\n\n\n@dataclass(frozen=True)\nclass Selectors:\n    add_to_cart: str = 'button[name=\"add\"]'\n    cart_drawer: str = '[data-cart-drawer]'\n    checkout_button: str = 'button:has-text(\"Checkout\")'\n    product_price: str = '[data-product-price]'\n\n\nHERO_TEE = Product(slug=\"hero-tee\", title=\"Hero Tee\", price_eur=\"29.95\")\nSEL = Selectors()<\/code><\/pre>\n\n\n\n
    And the rhythm in a test \u2014 step<\/code> \u2192 human_click<\/code> \u2192 random_sleep<\/code>, every action, including the last one:<\/p>\n\n\n\n
    # tests\/test_add_to_cart.py\nfrom playwright.sync_api import expect\n\nfrom tests.fixtures.catalog import HERO_TEE, SEL\nfrom tests.conftest import bypass_cloudflare\n\nBASE = \"https:\/\/store.example.com\"\n\n\ndef test_add_to_cart_shows_correct_price(stealth_page, step,\n                                         random_sleep, human_click):\n    sp = stealth_page\n    page = sp.page\n\n    step(\"open home\", page)\n    page.goto(BASE, wait_until=\"domcontentloaded\")\n    bypass_cloudflare(sp.sb, page, BASE, sp.proxy_info)\n    random_sleep(page)\n\n    step(\"navigate to product\", page)\n    page.goto(f\"{BASE}\/products\/{HERO_TEE.slug}\",\n              wait_until=\"domcontentloaded\")\n    bypass_cloudflare(sp.sb, page, f\"{BASE}\/products\/{HERO_TEE.slug}\",\n                      sp.proxy_info)\n    random_sleep(page)\n\n    step(\"verify listed price\", page)\n    expect(page.locator(SEL.product_price)).to_contain_text(\n        HERO_TEE.price_eur, timeout=15_000,\n    )\n    random_sleep(page)\n\n    step(\"add to cart\", page)\n    human_click(page, SEL.add_to_cart)\n    random_sleep(page)\n\n    step(\"verify cart drawer\", page)\n    expect(page.locator(SEL.cart_drawer)).to_contain_text(\n        HERO_TEE.title, timeout=15_000,\n    )\n    expect(page.locator(SEL.cart_drawer)).to_contain_text(\n        HERO_TEE.price_eur, timeout=15_000,\n    )\n    random_sleep(page)<\/code><\/pre>\n\n\n\n
    Two things the expect(...).to_contain_text(timeout=...)<\/code> form buys you: it retries on Playwright’s internal polling loop, so the async DOM settles naturally; and it fails with a readable diff instead of a stale assertion snapshot. Never use locator.inner_text() == \"\u2026\"<\/code> for prices on a Shopify store \u2014 Shopify’s cart drawer mutates in two passes and you will flake.<\/p>\n\n\n\n
    5. The human_click helper, explained<\/h2>\n\n\n\n
    Here it is again, isolated, so you can lift it into another project without importing the rest of the file:<\/p>\n\n\n\n
    def human_click(page, selector):\n    locator = page.locator(selector).first\n    locator.scroll_into_view_if_needed()\n    box = locator.bounding_box()\n    if box is None:\n        locator.click()\n        return\n    fx = random.uniform(0.30, 0.70)\n    fy = random.uniform(0.30, 0.70)\n    tx = box[\"x\"] + box[\"width\"] * fx\n    ty = box[\"y\"] + box[\"height\"] * fy\n    steps = random.randint(8, 18)\n    page.mouse.move(tx, ty, steps=steps)\n    page.wait_for_timeout(random.randint(120, 420))\n    page.mouse.click(tx, ty)<\/code><\/pre>\n\n\n\n
    Twenty lines. Here’s what each part earns:<\/p>\n\n\n\n
      \n
    • scroll_into_view_if_needed<\/code><\/strong> \u2014 if the button is below the fold, a real human scrolled to it. Playwright’s own .click()<\/code> does this, but we’re not calling .click()<\/code>, we’re calling .mouse.click()<\/code> at absolute coordinates, which doesn’t.<\/li>\n\n\n\n
    • fx, fy \u2208 [0.30, 0.70]<\/code><\/strong> \u2014 nobody clicks dead-center of a button. The 30\u201370% window keeps you away from the edge while still being off-axis.<\/li>\n\n\n\n
    • steps=8..18<\/code><\/strong> \u2014 page.mouse.move(x, y, steps=n)<\/code> interpolates n<\/code> intermediate mousemove<\/code> events between the current position and (x, y)<\/code>. Bot detectors watch the distribution of those events; a single teleport from (0,0)<\/code> is a dead giveaway.<\/li>\n\n\n\n
    • 120..420ms hover<\/code><\/strong> \u2014 the gap between the mouse arriving and the click firing. Humans hesitate. Bots don’t.<\/li>\n<\/ul>\n\n\n\n
      Full Bezier-curve libraries like ghost-cursor<\/code> do more \u2014 acceleration curves, overshoot, micro-corrections. They also add a dependency, a Puppeteer bridge, and a handful of edge cases. Twenty lines of linear interpolation plus a hover covers roughly 80% of the signal for none of the cost. Revisit it only when you have evidence the simple version got caught.<\/p>\n\n\n\n
      6. CI setup with GitHub Actions<\/h2>\n\n\n\n
      One job, sequential, per-test ::group::<\/code> blocks, single artifact upload:<\/p>\n\n\n\n
      # .github\/workflows\/e2e.yml\nname: e2e\n\non:\n  schedule:\n    - cron: \"0 6 * * *\"\n  workflow_dispatch:\n\njobs:\n  run:\n    runs-on: ubuntu-latest\n    timeout-minutes: 60\n    steps:\n      - uses: actions\/checkout@v4\n\n      - uses: actions\/setup-python@v5\n        with:\n          python-version: \"3.11\"\n\n      - name: Install system deps for Chrome\n        run: |\n          sudo apt-get update\n          sudo apt-get install -y xvfb libnss3 libatk-bridge2.0-0 \\\n            libgbm1 libasound2\n\n      - name: Install Python deps\n        run: |\n          pip install -r requirements.txt\n          playwright install chromium\n\n      - name: Clean old logs\n        run: rm -rf latest_logs && mkdir -p latest_logs\/junit\n\n      - name: Run tests (sequential, one proxy)\n        env:\n          CAPSOLVER_API_KEY: ${{ secrets.CAPSOLVER_API_KEY }}\n          WEBSHARE_PROXY: ${{ secrets.WEBSHARE_PROXY }}\n          DISPLAY: :99\n        run: |\n          Xvfb :99 -screen 0 1920x1080x24 &\n          sleep 1\n          failed=0\n          for f in tests\/test_*.py; do\n            name=$(basename \"$f\" .py)\n            echo \"::group::${name}\"\n            xvfb-run -a pytest \"$f\" \\\n              --reruns 1 --reruns-delay 20 \\\n              --junit-xml=\"latest_logs\/junit\/${name}.xml\" \\\n              -v || failed=1\n            echo \"::endgroup::\"\n          done\n          exit $failed\n\n      - name: Upload logs and screenshots\n        if: always()\n        uses: actions\/upload-artifact@v4\n        with:\n          name: e2e-latest-logs\n          path: latest_logs\/\n          retention-days: 14<\/code><\/pre>\n\n\n\n
      The ::group::<\/code> wrapping is the quality-of-life upgrade nobody thinks about until they’ve scrolled through 4,000 lines of combined pytest output looking for which test actually broke. With groups, the Actions UI collapses each test to a one-liner; you click only the ones you care about.<\/p>\n\n\n\n
      The per-file --junit-xml<\/code> is mandatory, not optional \u2014 if every test run writes to the same report.xml<\/code>, you lose everything but the last one.<\/p>\n\n\n\n
      Worth spelling out why this beats a matrix strategy: you have one residential IP. Two runners hitting the store simultaneously from the same exit IP doubles your request rate without doubling your credibility, which is the exact failure mode Cloudflare watches for. Sequential is slower but it gets to the end.<\/p>\n\n\n\n
      7. What’s intentionally not solved<\/h2>\n\n\n\n
      Per-request proxy rotation.<\/strong> The stack uses one static residential IP at any given time, on purpose \u2014 but “static” here means static for the duration of a session<\/em>, not static forever. Webshare’s dashboard exposes a Replace Proxies<\/strong> setting (Proxy Settings \u2192 Replace Proxies) that auto-swaps the underlying IP when any of these trigger:<\/p>\n\n\n\n
        \n
      • proxy unavailable for more than 15 minutes<\/li>\n\n\n\n
      • proxy has low country confidence<\/li>\n\n\n\n
      • proxy experiencing temporary slowdown<\/li>\n<\/ul>\n\n\n\n
        That’s the right layer for rotation. The client keeps pointing at the same Webshare endpoint (p.webshare.io:80<\/code>), the cf_clearance<\/code> cookie stays valid for the lifetime of one test run, and IP health is handled by the provider in the background. When Webshare rotates the upstream IP, the next test run gets a fresh cf_clearance<\/code> on first challenge \u2014 no client-side pool logic, no round-robin, no cookie juggling.<\/p>\n\n\n\n
        What we explicitly don’t<\/em> do is rotate the proxy per-request from inside the test. That’s the failure mode: Cloudflare issues cf_clearance<\/code> bound to the IP it saw, you rotate mid-session, the next request arrives from a new IP carrying a clearance cookie that was minted for a different one, Cloudflare invalidates it, and you re-challenge every single navigation. Rotation belongs at the session boundary (provider-side, between runs), not the request boundary.<\/p>\n\n\n\n
        Rule of thumb for the split:<\/p>\n\n\n\n
        Layer<\/th>Who handles it<\/th>When<\/th><\/tr><\/thead>
        IP health, dead-proxy replacement<\/td>Webshare dashboard (Replace Proxies)<\/td>Background, auto<\/td><\/tr>
        Per-session IP stickiness<\/td>Webshare static endpoint<\/td>Duration of one test run<\/td><\/tr>
        Per-request rotation<\/td>Nobody. Don’t do it.<\/td>\u2014<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
        Full ghost-cursor Bezier curves.<\/strong> The lightweight human_click<\/code> above is ~20 lines and buys ~80% of the humanness signal. If you find yourself caught specifically on mouse-motion fingerprints \u2014 not all Cloudflare fingerprints, not TLS, not headless giveaways \u2014 then swap it. Otherwise, leave it.<\/p>\n\n\n\n
        Parallel test execution.<\/strong> pytest-xdist<\/code> would cut wall-clock time by 4x. It would also multiply your requests-per-second from one exit IP by 4x, and Cloudflare’s rate limiter is tighter than you think. Sequential with --reruns 1<\/code> is the setting that stays green.<\/p>\n\n\n\n
        8. Conclusion<\/h2>\n\n\n\n
        Total build: about 300 lines of Python across conftest.py<\/code>, auth_proxy.py<\/code>, and a small catalog.py<\/code>, plus one GitHub Actions file. It survives Cloudflare on a live Shopify store, it looks human enough that the challenge rate stays under 10%, and it scales comfortably to around ten flows on a single residential IP.<\/p>\n\n\n\n
        The pieces are independent on purpose. Lift human_click<\/code> into a Selenium codebase and it still works. Use bypass_cloudflare<\/code> with a different test runner and it still works. Keep the auth-bridge proxy around even if you throw the rest out \u2014 any time you need authenticated proxying from a tool that can’t do CONNECT auth, it’s the same sixty lines.<\/p>\n\n\n\n
        Adapt what you need, ignore what you don’t, and check the latest_logs\/<\/code> artifact on every failed run. The screenshots are almost always faster than reading the stack trace.<\/p>\n","protected":false},"excerpt":{"rendered":"
        You point Playwright at a Shopify store fronted by Cloudflare. You get a 403 or a permanent challenge page. You try headless=False. Same result. You add a user agent. Same result. You add a residential proxy. Now you get rate-limited instead of blocked, which is progress of a sort. This is a build log for […]<\/p>\n","protected":false},"author":1,"featured_media":1253,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_gspb_post_css":"","footnotes":""},"categories":[2,3,9],"tags":[],"class_list":["post-1249","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-automation","category-development","category-shopify"],"blocksy_meta":[],"yoast_head":"\nBot-Resistant E2E Tests for Shopify with Playwright<\/title>\n<meta name=\"description\" content=\"Run Playwright E2E tests against Cloudflare-protected Shopify stores without getting blocked. Python, SeleniumBase stealth, residential proxies, full code.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/aharonyan.org\/?p=1249\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Bot-Resistant E2E Tests for Shopify with Playwright\" \/>\n<meta property=\"og:description\" content=\"Run Playwright E2E tests against Cloudflare-protected Shopify stores without getting blocked. Python, SeleniumBase stealth, residential proxies, full code.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/aharonyan.org\/?p=1249\" \/>\n<meta property=\"og:site_name\" content=\"Aharonyan\" \/>\n<meta property=\"article:published_time\" content=\"2026-04-15T19:03:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-04-15T19:39:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1376\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"aharonyan\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"aharonyan\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/aharonyan.org\/?p=1249#article\",\"isPartOf\":{\"@id\":\"https:\/\/aharonyan.org\/?p=1249\"},\"author\":{\"name\":\"aharonyan\",\"@id\":\"https:\/\/aharonyan.org\/#\/schema\/person\/52b1f716de6ec82d2bbe5acba6580116\"},\"headline\":\"Building Bot-Resistant E2E Tests for Shopify Stores with Python, Playwright, and SeleniumBase\",\"datePublished\":\"2026-04-15T19:03:51+00:00\",\"dateModified\":\"2026-04-15T19:39:13+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/aharonyan.org\/?p=1249\"},\"wordCount\":1498,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/aharonyan.org\/#organization\"},\"image\":{\"@id\":\"https:\/\/aharonyan.org\/?p=1249#primaryimage\"},\"thumbnailUrl\":\"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png\",\"articleSection\":[\"Automation\",\"Development\",\"Shopify\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/aharonyan.org\/?p=1249#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/aharonyan.org\/?p=1249\",\"url\":\"https:\/\/aharonyan.org\/?p=1249\",\"name\":\"Bot-Resistant E2E Tests for Shopify with Playwright\",\"isPartOf\":{\"@id\":\"https:\/\/aharonyan.org\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/aharonyan.org\/?p=1249#primaryimage\"},\"image\":{\"@id\":\"https:\/\/aharonyan.org\/?p=1249#primaryimage\"},\"thumbnailUrl\":\"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png\",\"datePublished\":\"2026-04-15T19:03:51+00:00\",\"dateModified\":\"2026-04-15T19:39:13+00:00\",\"description\":\"Run Playwright E2E tests against Cloudflare-protected Shopify stores without getting blocked. Python, SeleniumBase stealth, residential proxies, full code.\",\"breadcrumb\":{\"@id\":\"https:\/\/aharonyan.org\/?p=1249#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/aharonyan.org\/?p=1249\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/aharonyan.org\/?p=1249#primaryimage\",\"url\":\"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png\",\"contentUrl\":\"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png\",\"width\":1376,\"height\":768},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/aharonyan.org\/?p=1249#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/aharonyan.org\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Building Bot-Resistant E2E Tests for Shopify Stores with Python, Playwright, and SeleniumBase\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/aharonyan.org\/#website\",\"url\":\"https:\/\/aharonyan.org\/\",\"name\":\"Aharonyan\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/aharonyan.org\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/aharonyan.org\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/aharonyan.org\/#organization\",\"name\":\"Aharonyan\",\"url\":\"https:\/\/aharonyan.org\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/aharonyan.org\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/02\/minimal-monogram-logo-aa-with-code-brackets-geo-e1770379558388.png\",\"contentUrl\":\"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/02\/minimal-monogram-logo-aa-with-code-brackets-geo-e1770379558388.png\",\"width\":604,\"height\":590,\"caption\":\"Aharonyan\"},\"image\":{\"@id\":\"https:\/\/aharonyan.org\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/aharonyan.org\/#\/schema\/person\/52b1f716de6ec82d2bbe5acba6580116\",\"name\":\"aharonyan\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/aharonyan.org\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/aaba86516a6722078e46f3ed6ee49fa7477de12dfdc7e91212df95c4834b8775?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/aaba86516a6722078e46f3ed6ee49fa7477de12dfdc7e91212df95c4834b8775?s=96&d=mm&r=g\",\"caption\":\"aharonyan\"},\"sameAs\":[\"https:\/\/aharonyan.org\"],\"url\":\"https:\/\/aharonyan.org\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Bot-Resistant E2E Tests for Shopify with Playwright","description":"Run Playwright E2E tests against Cloudflare-protected Shopify stores without getting blocked. Python, SeleniumBase stealth, residential proxies, full code.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/aharonyan.org\/?p=1249","og_locale":"en_US","og_type":"article","og_title":"Bot-Resistant E2E Tests for Shopify with Playwright","og_description":"Run Playwright E2E tests against Cloudflare-protected Shopify stores without getting blocked. Python, SeleniumBase stealth, residential proxies, full code.","og_url":"https:\/\/aharonyan.org\/?p=1249","og_site_name":"Aharonyan","article_published_time":"2026-04-15T19:03:51+00:00","article_modified_time":"2026-04-15T19:39:13+00:00","og_image":[{"width":1376,"height":768,"url":"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png","type":"image\/png"}],"author":"aharonyan","twitter_card":"summary_large_image","twitter_misc":{"Written by":"aharonyan","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/aharonyan.org\/?p=1249#article","isPartOf":{"@id":"https:\/\/aharonyan.org\/?p=1249"},"author":{"name":"aharonyan","@id":"https:\/\/aharonyan.org\/#\/schema\/person\/52b1f716de6ec82d2bbe5acba6580116"},"headline":"Building Bot-Resistant E2E Tests for Shopify Stores with Python, Playwright, and SeleniumBase","datePublished":"2026-04-15T19:03:51+00:00","dateModified":"2026-04-15T19:39:13+00:00","mainEntityOfPage":{"@id":"https:\/\/aharonyan.org\/?p=1249"},"wordCount":1498,"commentCount":0,"publisher":{"@id":"https:\/\/aharonyan.org\/#organization"},"image":{"@id":"https:\/\/aharonyan.org\/?p=1249#primaryimage"},"thumbnailUrl":"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png","articleSection":["Automation","Development","Shopify"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/aharonyan.org\/?p=1249#respond"]}]},{"@type":"WebPage","@id":"https:\/\/aharonyan.org\/?p=1249","url":"https:\/\/aharonyan.org\/?p=1249","name":"Bot-Resistant E2E Tests for Shopify with Playwright","isPartOf":{"@id":"https:\/\/aharonyan.org\/#website"},"primaryImageOfPage":{"@id":"https:\/\/aharonyan.org\/?p=1249#primaryimage"},"image":{"@id":"https:\/\/aharonyan.org\/?p=1249#primaryimage"},"thumbnailUrl":"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png","datePublished":"2026-04-15T19:03:51+00:00","dateModified":"2026-04-15T19:39:13+00:00","description":"Run Playwright E2E tests against Cloudflare-protected Shopify stores without getting blocked. Python, SeleniumBase stealth, residential proxies, full code.","breadcrumb":{"@id":"https:\/\/aharonyan.org\/?p=1249#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/aharonyan.org\/?p=1249"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/aharonyan.org\/?p=1249#primaryimage","url":"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png","contentUrl":"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/04\/Gemini_Generated_Image_jrxs07jrxs07jrxs.png","width":1376,"height":768},{"@type":"BreadcrumbList","@id":"https:\/\/aharonyan.org\/?p=1249#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/aharonyan.org\/"},{"@type":"ListItem","position":2,"name":"Building Bot-Resistant E2E Tests for Shopify Stores with Python, Playwright, and SeleniumBase"}]},{"@type":"WebSite","@id":"https:\/\/aharonyan.org\/#website","url":"https:\/\/aharonyan.org\/","name":"Aharonyan","description":"","publisher":{"@id":"https:\/\/aharonyan.org\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/aharonyan.org\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/aharonyan.org\/#organization","name":"Aharonyan","url":"https:\/\/aharonyan.org\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/aharonyan.org\/#\/schema\/logo\/image\/","url":"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/02\/minimal-monogram-logo-aa-with-code-brackets-geo-e1770379558388.png","contentUrl":"https:\/\/aharonyan.org\/wp-content\/uploads\/2026\/02\/minimal-monogram-logo-aa-with-code-brackets-geo-e1770379558388.png","width":604,"height":590,"caption":"Aharonyan"},"image":{"@id":"https:\/\/aharonyan.org\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/aharonyan.org\/#\/schema\/person\/52b1f716de6ec82d2bbe5acba6580116","name":"aharonyan","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/aharonyan.org\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/aaba86516a6722078e46f3ed6ee49fa7477de12dfdc7e91212df95c4834b8775?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/aaba86516a6722078e46f3ed6ee49fa7477de12dfdc7e91212df95c4834b8775?s=96&d=mm&r=g","caption":"aharonyan"},"sameAs":["https:\/\/aharonyan.org"],"url":"https:\/\/aharonyan.org\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/aharonyan.org\/index.php?rest_route=\/wp\/v2\/posts\/1249","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aharonyan.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aharonyan.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aharonyan.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aharonyan.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1249"}],"version-history":[{"count":1,"href":"https:\/\/aharonyan.org\/index.php?rest_route=\/wp\/v2\/posts\/1249\/revisions"}],"predecessor-version":[{"id":1250,"href":"https:\/\/aharonyan.org\/index.php?rest_route=\/wp\/v2\/posts\/1249\/revisions\/1250"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/aharonyan.org\/index.php?rest_route=\/wp\/v2\/media\/1253"}],"wp:attachment":[{"href":"https:\/\/aharonyan.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1249"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aharonyan.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1249"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aharonyan.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1249"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}