Scrapy 代理配置
Python crawling framework with its own downloader middleware stack. 按下面的配置把它指向 Eveses 住宅池 — 地区写在用户名里，所以换国家不需要动其他任何设置。
proxy.eveses.com:12321 · SOCKS5 在 32325 · 199 个国家 · $7.35/GB 起
Scrapy will not pick the proxy up from `http_proxy` for HTTPS targets. It has to go through `HttpProxyMiddleware`, and the credentials must be set on the request rather than in the URL — a proxy URL with a password in it is silently stripped from `meta["proxy"]`.
配置
# settings.py
DOWNLOADER_MIDDLEWARES = {
"scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware": 110,
"myproject.middlewares.EvesesProxyMiddleware": 100, # must run BEFORE 110
}
# middlewares.py
import base64
class EvesesProxyMiddleware:
USER = "eveses_user_country-us" # add geo tokens to the username
PASS = "YOUR_PROXY_PASSWORD"
def process_request(self, request, spider):
request.meta["proxy"] = "http://proxy.eveses.com:12321"
creds = base64.b64encode(f"{self.USER}:{self.PASS}".encode()).decode()
request.headers["Proxy-Authorization"] = f"Basic {creds}"Scrapy 相关说明
Give the middleware a lower number than HttpProxyMiddleware (110) so it sets `meta["proxy"]` first.
Rotate per request by appending a fresh `_session-…` token to the username inside `process_request`.
Leave `RETRY_TIMES` at its default; a blocked exit IP retries onto a different one automatically.
换国家而不动 Scrapy
eveses_user_country-deeveses_user_country-de_session-ab12cd34_lifetime-30m地区和会话都写在用户名里，而不是主机或端口。换国家时本页所有配置保持原样 — 你只改一个字符串。可定位 199 个国家，其中大多数还支持城市与州/省令牌。