ScrapingCat Scholar API

Authentication

Every /v1 route requires an API key sent in the X-API-Key request header. Keys in the query string are rejected: a request carrying an api_key query parameter fails with 400 regardless of its value.

curl -H "X-API-Key: $SCAT_API_KEY" \
  "https://scrapingcat.boyinzhiheng.com/v1/search?engine=google_scholar&q=deep+learning"

• Missing or invalid key → 401 unauthorized.
• /health and /ready need no key.

General query rules: keys may not be duplicated, unknown keys are rejected, and output (when supplied) must be json.

Errors & status codes

Every error response uses one fixed shape. request_id is unique per request — include it when reporting a problem.

{
  "error": {
    "code": "rate_limited",
    "message": "Request rate limit exceeded",
    "retryable": true
  },
  "request_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}

GET /v1/search

One search endpoint, four engines. The engine parameter selects the target and determines which other parameters are valid.

engine=google_scholar — organic search

Requires at least one of q, cites, or cluster. cluster is exclusive with q and cites.

curl -H "X-API-Key: $SCAT_API_KEY" \
  "https://scrapingcat.boyinzhiheng.com/v1/search?engine=google_scholar&q=attention+is+all+you+need&num=10"

The response mirrors the SerpApi Google Scholar shape: search_metadata, search_parameters, search_information, organic_results[] (with result_id, title, link, publication_info, inline_links.cited_by, …) and pagination when more pages exist.

engine=google_scholar_author — author profile

curl -H "X-API-Key: $SCAT_API_KEY" \
  "https://scrapingcat.boyinzhiheng.com/v1/search?engine=google_scholar_author&author_id=qc6CJjYAAAAJ&num=100"

engine=google_scholar_cite — citation export

q is a result_id taken from an organic search response. Returns the formatted citation styles (MLA, APA, Chicago, Harvard, Vancouver) and export links (BibTeX, EndNote, RefMan, RefWorks).

curl -H "X-API-Key: $SCAT_API_KEY" \
  "https://scrapingcat.boyinzhiheng.com/v1/search?engine=google_scholar_cite&q=KlAl5vgvd-8J"

engine=google_scholar_profiles — profile search

Searches author profiles by name via mauthors. Pagination uses opaque continuation tokens: pass either after_author or before_author (mutually exclusive) from a previous response. This surface is best-effort and may return a retryable upstream error under load.

curl -H "X-API-Key: $SCAT_API_KEY" \
  "https://scrapingcat.boyinzhiheng.com/v1/search?engine=google_scholar_profiles&mauthors=Geoffrey+Hinton"

GET /v1/papers/resolve

Deterministic paper lookup against Google Scholar. Supply exactly one selector:

curl -H "X-API-Key: $SCAT_API_KEY" \
  "https://scrapingcat.boyinzhiheng.com/v1/papers/resolve?doi=10.1038/nature14539"

The response reports an explicit match_status — exact, probable, ambiguous, or not_found — together with the matched result (identifiers, title, link, citation counts) and the candidates that were considered, so callers can apply their own acceptance policy.

Durable batches

Batches process thousands of authors or papers asynchronously with durable state: batch metadata, per-item progress, retries, pause/resume, and crash-safe result archives all survive service restarts. Submit inputs, poll status, then download a compressed NDJSON archive of one terminal record per canonical input.

• Author inputs carry author_id. Paper inputs carry exactly one of doi, pmid, arxiv_id, or title.
• Duplicate inputs are canonicalized; the first occurrence's index is retained.
• An optional Idempotency-Key header makes creation replay-safe: the same key with identical canonical content returns the original batch; the same key with different content returns 409.

Batch endpoints

curl -X POST -H "X-API-Key: $SCAT_API_KEY" -H "Content-Type: application/json" \
  -H "Idempotency-Key: my-batch-2026-07-17" \
  -d '{"items":[{"author_id":"qc6CJjYAAAAJ"},{"author_id":"JicYPdAAAAAJ"}],"options":{"hl":"en"}}' \
  "https://scrapingcat.boyinzhiheng.com/v1/batches/authors"

curl -H "X-API-Key: $SCAT_API_KEY" \
  -o results.ndjson.gz \
  "https://scrapingcat.boyinzhiheng.com/v1/batches/b_01J.../results.ndjson.gz"

Each archive line is one JSON record with the item's original input_index, canonical input, terminal state, and either the full result document or a fixed error classification.

Batch & item states

ingesting → queued → running → completed | completed_with_errors | failed
                       │
                       └→ pausing → paused → queued  (on resume)

Item states: pending, running, retry_wait, succeeded, not_found, failed. A missing entity (e.g. an unknown author_id) terminates as not_found, not failed.

Limits & retention

Health & readiness