Python · httpx
httpx and ProxyGuys, from one request to many at once
httpx offers the same client in a blocking flavour and an asyncio flavour. Either way, the proxy is a single proxy argument on the client, and every request that client makes goes through it. This page sets up both, runs several requests in parallel through one ProxyGuys proxy, and changes the IP from async code. The httpx manual is at python-httpx.org.
Install a current httpx
pip install -U httpx
# for the SOCKS5 port (adds the socksio package)
pip install -U "httpx[socks]"The examples need httpx 0.28 or newer. That release removed the old proxies= argument still shown in many tutorials, and current releases accept socks5h:// proxy URLs, which an outdated install may reject.
Where each value comes from
Your proxy must be started first: a new proxy sits as Pending until you press Start proxy on My Proxies. In the Connection column, the "HTTP proxy" and "SOCKS5 proxy" boxes each show the host and a port; the two ports are different. Credentials holds the username (Copy username) and the password, hidden until you use the eye button and copied with Copy password.
The code below reads three environment variables. For SOCKS5, write the endpoint with socks5h://, the form the SOCKS5 copy button gives you. httpx hands hostnames to a SOCKS5 proxy to look up whether the scheme is socks5:// or socks5h://; releases before 0.28 just don't accept the second spelling.
# pick one endpoint: the HTTP port with http://, or the SOCKS5 port with socks5h://
export PROXY_ENDPOINT="http://HOST:HTTP_PORT"
# export PROXY_ENDPOINT="socks5h://HOST:SOCKS5_PORT"
export PROXY_USER="USERNAME"
export PROXY_PASS="PASSWORD"On Windows PowerShell the same lines read $env:PROXY_USER = "USERNAME" and so on. Prefer one variable? The Connection copy buttons give a full URL with the login inside, and httpx accepts that as proxy= directly. Only a password containing # or % then needs those characters percent-encoded (%23, %25). If you work with the API, a GET to /proxies/pp123456/credentials returns host, ports, username, password and ready-made URLs; change its socks5:// URL to socks5h:// before use.
A blocking client
import os
import httpx
# The login travels apart from the address, so a password containing
# # or % needs no URL-encoding.
proxy = httpx.Proxy(
os.environ["PROXY_ENDPOINT"],
auth=(os.environ["PROXY_USER"], os.environ["PROXY_PASS"]),
)
with httpx.Client(proxy=proxy, timeout=30.0) as client:
r = client.get("https://api.ipify.org", params={"format": "json"})
r.raise_for_status()
print("exit IP:", r.json()["ip"])The printed address should differ from the one you see with the proxy switched off. A lookup site might put it in another city or state than the one you chose; lookup databases disagree about mobile IPs, and that's normal.
Many requests at once with AsyncClient
AsyncClient takes the same proxy object. Here three pages load concurrently, all through the same ProxyGuys proxy. Limits caps how many connections the client opens at a time; choose a number that suits the sites you visit. With return_exceptions=True, one failed page doesn't cancel the others.
import asyncio
import os
import httpx
PROXY = httpx.Proxy(
os.environ["PROXY_ENDPOINT"],
auth=(os.environ["PROXY_USER"], os.environ["PROXY_PASS"]),
)
URLS = [
"https://httpbin.org/ip",
"https://httpbin.org/user-agent",
"https://httpbin.org/get?page=3",
]
async def main():
limits = httpx.Limits(max_connections=4)
async with httpx.AsyncClient(proxy=PROXY, timeout=30.0, limits=limits) as client:
results = await asyncio.gather(
*(client.get(u) for u in URLS), return_exceptions=True
)
for url, result in zip(URLS, results):
if isinstance(result, Exception):
print(f"{url}: failed ({result!r})")
else:
print(f"{url}: {result.status_code}")
asyncio.run(main())Rotate the IP without blocking the event loop
Create a key on the API page (Create key; it is shown once, so save it). Put it in PROXYGUYS_API_KEY and the proxy's API id, such as pp123456, in PROXYGUYS_PROXY_ID. The id is not the proxy login; GET /proxies lists your proxies.
The rotate endpoint gives a new IP in the same city and on the same carrier. It answers when the change is complete, usually 30 to 90 seconds later, with a new_ip field that may be null. Within 60 seconds of the last change it returns a 409 whose body holds "code": "cooldown" and retry_after in seconds; asyncio.sleep waits that out while your other tasks keep running.
import asyncio
import os
import httpx
API = "https://proxyguys.com/api/v1"
async def rotate(api, proxy_id, attempts=3):
for _ in range(attempts):
r = await api.post(f"/proxies/{proxy_id}/rotate")
data = r.json()
if r.status_code == 409 and data.get("code") == "cooldown":
await asyncio.sleep(float(data.get("retry_after", 60)))
continue
if r.is_error:
raise RuntimeError(f"rotate failed ({r.status_code}): {data.get('error')}")
return data.get("new_ip") # can be None
raise RuntimeError("cooldown did not clear")
async def main():
headers = {"Authorization": "Bearer " + os.environ["PROXYGUYS_API_KEY"]}
# The rotate call replies once the change is done, so give it plenty of time.
async with httpx.AsyncClient(base_url=API, headers=headers, timeout=150.0) as api:
print("new IP:", await rotate(api, os.environ["PROXYGUYS_PROXY_ID"]))
asyncio.run(main())The API client is a separate AsyncClient with no proxy argument, kept apart from the one that browses through your proxy. Timed rotation and the change-IP link are described in Changing and rotating IPs.
Error messages and what they mean
- An unexpected keyword argument 'proxies'. The code was written for an older httpx. Use
proxy=as shown here. Ifproxyis the unknown argument instead, your httpx is too old: upgrade it. - Unknown scheme for proxy URL. Your httpx is too old to take
socks5h://. Upgrade it withpip install -U httpx. - The 'socksio' package is not installed. Run
pip install "httpx[socks]". - ProxyError with a 407 status. Username or password is wrong. Copy them again from Credentials; if you recently saved a New password, update the variable.
- ConnectTimeout, ReadTimeout or a SOCKS error. Check that the scheme matches the port:
http://with the HTTP port,socks5h://with the SOCKS5 port. The ⋮ → Details line carries the SOCKS5 port. - Using the IP whitelist. Copy endpoint gives the whitelist port's host and port. Hand
httpx.Proxythat address withhttp://orsocks5h://in front, whichever matches the whitelist Protocol, and dropauth. Details are in the IP whitelist guide.