# Configure TrueProxies with Crawlee

> Configure TrueProxies in Crawlee for JavaScript and Python with ProxyConfiguration, and map Crawlee sessions to sticky exits.

Source: https://docs.trueproxies.com/integrations/crawlee/

> **Use your own endpoints**
>
> The examples below use placeholders. Copy the connection host, ports, username, and proxy password from the **Username and password** tab of your service in the dashboard. See [How to connect](https://docs.trueproxies.com/proxy-instructions/how-to-connect/#connection-host-and-ports).

Crawlee takes a `ProxyConfiguration`. For a new exit IP on each connection, give it one TrueProxies proxy URL. For a sticky exit per Crawlee session, pass a function that writes the Crawlee session ID into the username as a `session` option. The same approach works in Crawlee for JavaScript and Crawlee for Python.

## Installation

**JavaScript**

```bash
npm install crawlee playwright
npx playwright install chromium
```

**Python**

```bash
pip install "crawlee[playwright]"
playwright install chromium
```

## Basic usage

One proxy URL is enough: TrueProxies picks a new exit for each new connection. URL-encode the username and proxy password when you build the URL.

**JavaScript**

```javascript
import { PlaywrightCrawler, ProxyConfiguration } from 'crawlee';

const username = encodeURIComponent('your_username');
const password = encodeURIComponent('your_password');

const proxyConfiguration = new ProxyConfiguration({
  proxyUrls: [`http://${username}:${password}@YOUR_HOST:YOUR_HTTP_PORT`],
});

const crawler = new PlaywrightCrawler({
  proxyConfiguration,
  async requestHandler({ page, request, log }) {
    log.info(`${request.url}: ${await page.textContent('body')}`);
  },
});

await crawler.run(['https://httpbin.org/ip']);
```

**Python**

```python
import asyncio
from urllib.parse import quote

from crawlee.crawlers import PlaywrightCrawler, PlaywrightCrawlingContext
from crawlee.proxy_configuration import ProxyConfiguration

username = quote("your_username", safe="")
password = quote("your_password", safe="")


async def main() -> None:
    proxy_configuration = ProxyConfiguration(
        proxy_urls=[f"http://{username}:{password}@YOUR_HOST:YOUR_HTTP_PORT"],
    )
    crawler = PlaywrightCrawler(proxy_configuration=proxy_configuration)

    @crawler.router.default_handler
    async def handler(context: PlaywrightCrawlingContext) -> None:
        context.log.info(f"{context.request.url}: {await context.page.content()}")

    await crawler.run(["https://httpbin.org/ip"])


asyncio.run(main())
```

For Residential IPv4 country targeting, add the option to the username, for example `your_username-country-de`.

## Residential IPv4 sticky exit per Crawlee session

Crawlee’s session pool gives each session its own ID and cookies. Pass that ID through as a TrueProxies `session` so every request in a Crawlee session asks for the same exit IP. When Crawlee retires a session, for example after blocked responses, the next session ID asks for a new exit.

TrueProxies session values are 1 to 32 letters or digits. Crawlee for JavaScript creates IDs such as `session_Ab3dE6gH9k`, so remove every other character before you use the ID. Without this step the gateway refuses the username.

**JavaScript**

```javascript
import { PlaywrightCrawler, ProxyConfiguration } from 'crawlee';

const USERNAME = 'your_username';
const PASSWORD = 'your_password';
const PROXY = 'YOUR_HOST:YOUR_HTTP_PORT';

const proxyConfiguration = new ProxyConfiguration({
  newUrlFunction: (sessionId) => {
    const session = String(sessionId ?? '').replace(/[^A-Za-z0-9]/g, '').slice(0, 32);
    const user = session ? `${USERNAME}-session-${session}-lifetime-600` : USERNAME;
    return `http://${encodeURIComponent(user)}:${encodeURIComponent(PASSWORD)}@${PROXY}`;
  },
});

const crawler = new PlaywrightCrawler({
  proxyConfiguration,
  useSessionPool: true,
  persistCookiesPerSession: true,
  async requestHandler({ page, request, session, log }) {
    log.info(`${request.url} (${session?.id}): ${await page.textContent('body')}`);
  },
});

await crawler.run(['https://httpbin.org/ip']);
```

**Python**

```python
import asyncio
import re
from urllib.parse import quote

from crawlee import Request
from crawlee.crawlers import PlaywrightCrawler, PlaywrightCrawlingContext
from crawlee.proxy_configuration import ProxyConfiguration

USERNAME = "your_username"
PASSWORD = "your_password"
PROXY = "YOUR_HOST:YOUR_HTTP_PORT"


def new_url(session_id: str | None = None, request: Request | None = None) -> str:
    session = re.sub(r"[^A-Za-z0-9]", "", session_id or "")[:32]
    user = f"{USERNAME}-session-{session}-lifetime-600" if session else USERNAME
    return f"http://{quote(user, safe='')}:{quote(PASSWORD, safe='')}@{PROXY}"


async def main() -> None:
    crawler = PlaywrightCrawler(
        proxy_configuration=ProxyConfiguration(new_url_function=new_url),
        use_session_pool=True,
    )

    @crawler.router.default_handler
    async def handler(context: PlaywrightCrawlingContext) -> None:
        context.log.info(f"{context.request.url}: {await context.page.content()}")

    await crawler.run(["https://httpbin.org/ip"])


asyncio.run(main())
```

`lifetime` is in seconds. The accepted range depends on the product; see the [options table](https://docs.trueproxies.com/proxy-instructions/how-to-connect/#options).

## When requests fail

Crawlee retries failed requests and logs the browser’s or HTTP client’s error, which does not include the proxy’s reason. Send the same username and proxy password with `curl -v`, read the `X-Proxy-Reason` header, and look it up in [Proxy errors and refusal reasons](https://docs.trueproxies.com/proxy-instructions/errors/).

## Related guides

- [Connection and username format](https://docs.trueproxies.com/proxy-instructions/how-to-connect/)
- [HTTP CONNECT and SOCKS5](https://docs.trueproxies.com/proxy-instructions/proxy-protocols/)
- [Trusted IPs](https://docs.trueproxies.com/proxy-instructions/trusted-ips/)
- [Proxy errors and refusal reasons](https://docs.trueproxies.com/proxy-instructions/errors/)
- [Troubleshooting checklist](https://docs.trueproxies.com/support/)