Submit a request. Fetch the result later.

Submit and poll with Python

Install httpx with pip install httpx and set LITESCRAPE_API_KEY in your server environment. This example submits a Google Search job and waits for its result.

import os
import time
import uuid

import httpx

base_url = "https://api.litescrape.com"
# Persist this key if you need to retry the submission after a restart.
idempotency_key = str(uuid.uuid4())

with httpx.Client(
    base_url=base_url,
    headers={"Authorization": f"Bearer {os.environ['LITESCRAPE_API_KEY']}"},
    timeout=30,
) as client:
    submitted = client.post(
        "/api/google/search/async",
        headers={"Idempotency-Key": idempotency_key},
        json={"q": "coffee shops in Austin", "timeout": 60},
    )
    submitted.raise_for_status()
    job_id = submitted.json()["job_id"]
    deadline = time.monotonic() + 3600

    while time.monotonic() < deadline:
        # Status accepts up to 500 IDs; results accepts up to 100 IDs per page.
        polled = client.post("/api/batch/jobs/status", json={"job_ids": [job_id]})
        polled.raise_for_status()
        job = polled.json()["jobs"][0]
        if job.get("error_code"):
            raise RuntimeError(job)
        if job["state"] in {"succeeded", "failed"}:
            result = client.post("/api/batch/jobs/results", json={"job_ids": [job_id]})
            result.raise_for_status()
            job = result.json()["jobs"][0]
            if job.get("result_error"):
                raise RuntimeError(job["result_error"])
            if job["state"] == "failed":
                raise RuntimeError(f"Job failed ({job['failure_reason']}): {job['response']}")
            print(job["response"])
            break
        time.sleep(float(polled.headers.get("Retry-After", "15")))
    else:
        raise TimeoutError(f"Stopped waiting; resume polling job {job_id}")

Stopping your client does not cancel the job. Save the job ID so you can resume polling later.

Submit a job

Append /async to the endpoint path and use POST. For example, POST /api/google/search/async runs a Google Search request in the background. Account operations do not have async variants.

• Send Authorization: Bearer YOUR_API_KEY, Content-Type: application/json, and an Idempotency-Key of 16-128 characters.
• Put the endpoint's request fields in the JSON body. Use its existing parameter documentation for required fields and accepted values.
• Most async endpoints accept only scalar values: strings, numbers, and booleans. Omit unused fields instead of sending null, objects, or arrays.
• Fetch and Screenshot support nested options, Google Shopping Merchant URLs supports its products array, and Microsoft Copilot supports its messages array.
• Optional timeout is greater than zero and at most 90 seconds. Supply it once, in the JSON body or query string. This is an execution timeout, not a deadline for receiving the job result. Set a separate client-side wait limit if needed. Other options belong in the JSON body.

A successful submission returns 202 Accepted and a relative Location header such as /api/batch/jobs/00000000000000000000000000000001. Resolve that path against the API origin.

{
  "job_id": "00000000000000000000000000000001",
  "state": "queued",
  "status_code": null,
  "response": null,
  "attempts": 0,
  "deadline_at": "2026-01-01T01:00:00Z",
  "failure_reason": null,
  "created_at": "2026-01-01T00:00:00Z",
  "completed_at": null,
  "expires_at": null
}

Poll for results

For multiple jobs, send {"job_ids": ["..."]} to POST /api/batch/jobs/status for up to 500 IDs. Fetch completed responses with POST /api/batch/jobs/results for up to 100 IDs. Entries follow input order and unknown or expired IDs have their own errors.

You can also send GET to the returned Location URL with the owning account's current Bearer key. Polling checks existing jobs without submitting new requests. Follow the Retry-After header, currently 15 seconds, while jobs are pending.

A poll for a failed job still returns HTTP 200. Always inspect state and status_code; HTTP success alone does not mean the scraper succeeded. Before completion, status_code, response, completed_at, and expires_at are null.

An unknown job or a job owned by another account returns 404. A successful job's optional feedback URL is inside response.

Retries and idempotency

Generate one unique Idempotency-Key per intended job and keep it with the request. If submission times out, loses its connection, or returns 503, retry the identical request with the same key. Until the result expires, identical retries return the existing job instead of creating another one. After that, reusing the key may create and bill a new job. Do not generate a fresh key inside a retry loop.

Reusing a key with a different endpoint, parameters, or timeout returns 409. Use a new key when you intend to submit a new job.

Continue polling through queued and running until the job reaches a terminal state. If a poll temporarily fails, retry that same polling URL after a delay.

Jobs allow up to 15 execution attempts within a one-hour deadline. Waiting for capacity does not consume an attempt. Inspect attempts, deadline_at, and failure_reason to distinguish an exhausted attempt budget from a missed deadline.

Billing and retention

A successful job costs one API call. Failed jobs and polling are not charged. You need an available call to submit a job; insufficient credits returns 402 at submission.

Results remain available for 24 hours after completion, as shown by expires_at. Save results you need before they expire. Expired jobs return 410, and later 404.

Async submission and polling are unavailable while zero data retention (ZDR) is enabled and return 403. Unfinished jobs fail without charge, and completed results are not deleted but expire on their normal 24-hour schedule. Read the ZDR policy.

For complete schemas and error responses, see the Async group in the API explorer, OpenAPI contract, or Postman collection.