ProxyGuys.com
Get started

Python · Selenium

Selenium with a ProxyGuys proxy

Selenium controls a real Chrome or Firefox window. The catch with proxies is the login: a browser's proxy setting takes a host and a port but no username or password, and the sign-in box a browser shows for a proxy isn't something Selenium can fill in reliably. There are two clean ways round it.

  • IP whitelist (recommended). You allow your machine's IP on the proxy, and it connects to a separate whitelist port with no login at all. Nothing extra to install.
  • selenium-wire. A package that runs a small proxy inside your script and adds your username and password on the way out. Useful when you can't whitelist.

Selenium 4.6 and newer find or download a matching browser driver on their own, so pip install selenium is the only setup step. See selenium.dev for the Selenium reference.

pip install selenium

Route 1: whitelist your IP

Start by making sure the proxy is running; a new one stays Pending until you press Start proxy on My Proxies. Then set up the whitelist:

  1. On My Proxies, press the "IPs" button in the Whitelist column, or open the proxy's ⋮ menu and choose IP whitelist. The "IPv4 whitelist" window opens.

  2. Add the public IPv4 address of the machine that runs Selenium and press Add. The "+ your IPv4" chip fills in the address you are on right now, which is what you want if the script runs on this computer. Up to 15 addresses fit, and saving replaces the whole list.

  3. Set Protocol to HTTP and press Enable IP auth, which saves the list and switches the whitelist on.

  4. The whitelist port now shows in the window. Press Copy endpoint to copy its host and port, and store them in an environment variable:

    # paste the host:port that Copy endpoint gave you in the IP whitelist window
export PROXY_WHITELIST_ENDPOINT="PASTE_THE_COPIED_ENDPOINT"
# PowerShell: $env:PROXY_WHITELIST_ENDPOINT = "PASTE_THE_COPIED_ENDPOINT"

Your username and password keep working on the normal ports alongside the whitelist. On a few servers the whitelist isn't offered, and the window says so; use route 2 then. The IP whitelist guide explains the window in more detail.

Chrome

Chrome takes the proxy as a command-line switch, which Selenium passes for you:

import os

from selenium import webdriver
from selenium.webdriver.common.by import By

# host:port of the whitelist endpoint (a leading scheme, if any, is dropped)
endpoint = os.environ["PROXY_WHITELIST_ENDPOINT"].split("://")[-1]

options = webdriver.ChromeOptions()
options.add_argument(f"--proxy-server=http://{endpoint}")

driver = webdriver.Chrome(options=options)
try:
    driver.get("https://api.ipify.org")
    print("Chrome exit IP:", driver.find_element(By.CSS_SELECTOR, "body").text)
finally:
    driver.quit()

Firefox

Firefox reads its proxy from preferences, set here on the options object:

import os

from selenium import webdriver
from selenium.webdriver.common.by import By

host, port = os.environ["PROXY_WHITELIST_ENDPOINT"].split("://")[-1].rsplit(":", 1)

options = webdriver.FirefoxOptions()
options.set_preference("network.proxy.type", 1)  # 1 = manual proxy configuration
for kind in ("http", "ssl"):  # "ssl" is the setting Firefox uses for HTTPS traffic
    options.set_preference(f"network.proxy.{kind}", host)
    options.set_preference(f"network.proxy.{kind}_port", int(port))

driver = webdriver.Firefox(options=options)
try:
    driver.get("https://api.ipify.org")
    print("Firefox exit IP:", driver.find_element(By.CSS_SELECTOR, "body").text)
finally:
    driver.quit()

If you chose SOCKS5 as the whitelist protocol, use socks5:// in the Chrome switch. In Firefox, set network.proxy.socks and network.proxy.socks_port in place of the two HTTP settings. Firefox 128 and newer look up hostnames through a SOCKS5 proxy by default; on an older Firefox, also set network.proxy.socks_remote_dns to True.

Route 2: username and password with selenium-wire

selenium-wire is no longer maintained, so it can break with newer Python or package releases. The blinker pin below avoids a known import error. selenium-wire passes the username and password on to the proxy just as they appear in the URL, with no decoding step, so leave any symbols in your password as they are rather than percent-encoding them. Take the host and the HTTP port from the "HTTP proxy" box in the Connection column, and the login from Credentials (Copy username, Copy password), then export them as PROXY_HOST, PROXY_HTTP_PORT, PROXY_USER and PROXY_PASS.

pip install selenium selenium-wire "blinker<1.8"
import os

from selenium.webdriver.common.by import By
from seleniumwire import webdriver  # selenium-wire's stand-in for selenium's webdriver

# selenium-wire forwards this login without decoding it: no %-encoding here.
upstream = (
    f"http://{os.environ['PROXY_USER']}:{os.environ['PROXY_PASS']}"
    f"@{os.environ['PROXY_HOST']}:{os.environ['PROXY_HTTP_PORT']}"
)

driver = webdriver.Chrome(
    seleniumwire_options={"proxy": {"http": upstream, "https": upstream}}
)
try:
    driver.get("https://api.ipify.org")
    print("exit IP:", driver.find_element(By.CSS_SELECTOR, "body").text)
finally:
    driver.quit()

If selenium-wire won't install or import on your setup, the Playwright guide shows a browser automation library that accepts a proxy username and password directly.

Check the exit IP

Each script prints the address websites see. It should not be your own. A lookup site may show it in a different city or state from the location you chose; lookup databases differ, and with mobile IPs that is normal.

Change the IP while the browser is open

From the proxy's ⋮ menu, Copy change-IP link gives you a URL that changes the IP when opened, no login or key required. Store it in PROXY_CHANGE_IP_URL and keep it private: anyone who has it can change your IP, and New change-IP link replaces a leaked one. The link answers with a short text message, and the new IP usually takes 30 to 90 seconds to appear, so the script below reloads the echo page every 10 seconds until a different address shows. The link can be used once every 60 seconds; a refusal comes back as an HTTP error with the reason in its text, which the script prints.

import ipaddress
import os
import time
import urllib.error
import urllib.request

from selenium import webdriver
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.common.by import By


def browser_ip(driver):
    driver.get("https://api.ipify.org")
    text = driver.find_element(By.CSS_SELECTOR, "body").text.strip()
    ipaddress.ip_address(text)  # raises ValueError if the browser showed an error page
    return text


def change_ip(driver, wait_up_to=120):
    before = browser_ip(driver)
    try:
        # the reply can take as long as the change itself, so allow plenty of time
        with urllib.request.urlopen(os.environ["PROXY_CHANGE_IP_URL"], timeout=150) as reply:
            print(reply.read().decode())
    except urllib.error.HTTPError as err:  # e.g. asked again within 60 seconds
        print(err.read().decode())
        return None
    deadline = time.monotonic() + wait_up_to
    while time.monotonic() < deadline:
        time.sleep(10)
        try:
            now = browser_ip(driver)
        except (ValueError, WebDriverException):
            continue
        if now != before:
            return now
    return None  # no change seen yet


endpoint = os.environ["PROXY_WHITELIST_ENDPOINT"].split("://")[-1]
options = webdriver.ChromeOptions()
options.add_argument(f"--proxy-server=http://{endpoint}")
driver = webdriver.Chrome(options=options)
try:
    print("new IP:", change_ip(driver))
finally:
    driver.quit()

To change the IP through the API instead (a Bearer key and a reply that arrives once the new IP is ready), see the requests guide. Timed rotation from the dashboard is in Changing and rotating IPs.

Common problems

  • "This site can't be reached" or ERR_PROXY_CONNECTION_FAILED with the whitelist. Your public IP may have changed since you added it (home connections often do). Open the whitelist window, add the current address with "+ your IPv4" and save. Also check the protocol: an http:// switch needs the HTTP whitelist protocol.
  • ModuleNotFoundError mentioning blinker. Install "blinker<1.8" as shown above.
  • Pages fail with selenium-wire after a 407. The proxy rejected the login. Copy the password again with Copy password and check that the variable holds it unchanged: selenium-wire sends a percent-encoded character as literal text.
  • Nothing loads with selenium-wire. Make sure the URL uses the HTTP port. The ⋮ → Details line holds the SOCKS5 port.