Start with what is actually documented
There is a large body of writing explaining exactly how Perplexity and Claude decide what to cite. Almost none of it comes from Perplexity or Anthropic.
The explainers that name an embedding model, a reranker architecture or a scoring formula are third-party reverse-engineering, and a striking number of them are published by competing AI-visibility vendors. They may well be right. They are not documentation, and treating them as documentation is how a technical strategy ends up built on someone's inference.
So this guide draws a hard line. Everything below is marked either published by the vendor, published in the peer-reviewed literature, or not disclosed — and knowing where the third category begins is worth more than any of the guesses that fill it.
Perplexity, as Perplexity describes it
The most substantive public document is the Search API announcement of September 2025. From it: the index covers “hundreds of billions of webpages” and processes tens of thousands of index update requests per second; an AI content-understanding module generates parsing logic rather than relying on fixed extractors; and documents are split into fine-grained sub-document units, each individually scored against the query and pre-ranked before being surfaced.
That third point is the commercially important one. It is the strongest published support for a claim this industry makes constantly: retrieval operates at the level of passages, not whole pages. A page is not fetched and considered as a unit. Pieces of it compete separately.
In June 2026 Perplexity published an Agentic Search SDK description exposing retrieval, filtering, fanout and ranking as callable primitives, with models able to inspect candidate lists and ranking signals.
Insight
What Perplexity has never published: the embedding model, the reranker architecture, the scoring function, the rank-fusion method, or the algorithm binding a sentence in the answer to a specific URL. There is no technical paper. Even the claim that Perplexity runs wholly its own index rather than drawing on third-party search infrastructure is vendor-asserted and has never been independently audited.
Claude, where the documentation is better and narrower
Anthropic documents two distinct things that get conflated constantly.
The Citations API operates on documents you supply. Plain text and PDF documents are automatically chunked into sentences; custom-content documents use developer-supplied chunks. The model emits internal citation markers which are parsed into structured citation objects carrying character locations, page numbers or block indices. Anthropic reports the feature increased recall accuracy by up to 15% in internal testing.
The web search tool is a different system. Claude decides when to search by reasoning about whether the query needs current information. Results return with encrypted content used for citation and multi-turn continuity. Domain allow-lists and block-lists are configurable, as is a maximum number of searches, and a 2026 version adds dynamic filtering in which Claude writes code to filter results before they enter the context window.
What Anthropic has never published: which search backend the web search tool uses, how results are ranked, or the selection logic deciding which sentence gets cited. The Citations API documents chunking granularity and output format, not selection. This is an explicit boundary in the documentation, not an oversight.
The retrieval mechanics underneath
What the vendors do not describe, the literature does — at the level of the general technique rather than any specific production system.
Retrieval-augmented generation was formalised by Lewis et al. in 2020: a parametric model paired with a non-parametric memory, a retriever feeding a generator. Two families of retrieval compete. Lexical methods such as BM25 score documents on term overlap. Dense methods embed query and passage into a vector space and score by proximity, capturing paraphrase that lexical matching misses.
The obvious assumption here is wrong, and it matters. BEIR, the standard zero-shot retrieval benchmark, found dense retrievers frequently underperform BM25 out of domain. No single method dominates. This is why hybrid retrieval became the industry pattern, and why “semantic search replaced keyword matching” is a slogan rather than a finding.
Reranking follows. Dense retrieval is a fast, approximate first pass over a very large index; a cross-encoder reranker scores query and passage jointly rather than separately, which is substantially more accurate and far too expensive to run across an entire index. Hence the two-stage pattern: retrieve a shortlist cheaply, rerank it expensively. Commercial rerank products expose exactly this shape — and, notably, do not disclose their internal scoring either.
Position inside the context window is not neutral. Liu et al. found a U-shaped attention curve: material placed in the middle of a long context is recalled measurably worse than material at either end. Being included is not the same as being used.
Insight
The defensible technical description is this: passages compete, not pages; a cheap first pass selects a shortlist and an expensive second pass reorders it; and where a passage lands in the final context still affects whether it surfaces. Every part of that sentence is sourced. None of it names what Perplexity or Claude actually run.
The claim this industry makes that nobody can support
The standard line is that generative engines select on semantic relevance and source authority, and that backlinks no longer matter. The first half is reasonably supported. The second half is an inference.
Neither Perplexity nor Anthropic has ever published a statement that backlinks, domain authority or link-graph signals are excluded from their ranking. No vendor document says “we do not use backlinks”. That claim entered circulation through SEO commentary and has been repeated until it acquired the texture of fact.
There are good reasons to doubt it. Candidate generation over hundreds of billions of pages needs some prior over quality, and link-graph-derived authority is the cheapest and most mature such prior in existence. It would be surprising if a web-scale index discarded it. And BEIR's result — that lexical methods remain competitive out of domain — cuts against the tidy story that everything moved to embeddings.
The honest version: vendors document passage-level scoring for the features they have disclosed. Whether legacy authority signals still gate the candidate set upstream is unconfirmed in either direction. If a strategy depends on backlinks being irrelevant, it depends on something nobody has demonstrated.
What this does imply for a page
Passages must stand alone. If retrieval scores sub-document units, a paragraph that only makes sense after the three above it scores badly in isolation. Self-contained claims with the subject named explicitly — not “it” or “the company” — survive chunking. That is a writing instruction, not a markup instruction.
Evidence density beats formatting. The KDD 2024 work found quotations, statistics and cited sources each produced double-digit visibility improvements while keyword stuffing produced almost nothing; the 2026 citation-absorption analysis found Q&A formatting on its own slightly negative at −5.7%, and argued structure helps only when it carries real evidence.
Length and heading density correlate with citation — deeply-cited pages carried 11.4 times more words and 12.5 times more headings than rarely-cited ones. That is observational, and the study's authors say so.
Entity naming has to be unambiguous. A reranker scoring a passage against a query about your company can only do so if the passage makes clear which company it is. Consistent naming across your own pages, registries and third-party profiles is retrieval hygiene.
The crawler layer, fully documented and routinely mishandled
This is the best-documented part of the entire stack, and the most common place to find a self-inflicted wound.
The practical consequence: blocking training crawlers and blocking search crawlers are different decisions with different costs. A blanket rule aimed at keeping your content out of model training will also remove you from the retrieval index that answers questions about you. Plenty of companies have made that trade without knowing they made it.
What to take from all this
The engines are less mysterious than the marketing suggests and less transparent than the technical explainers imply. The published record supports passage-level competition, two-stage retrieval, position effects inside the context window, and a measurable advantage for content carrying real evidence.
It does not support any specific claim about ranking formulas, and it does not support the confident assertion that link-based authority has been retired. Build on the documented part. Treat the rest as hypothesis, and measure it rather than believing it.
Perplexity, Introducing the Perplexity Search API (25 Sep 2025) and Rethinking Search as Code Generation (1 Jun 2026) · Perplexity crawler documentation (undated; read 4 Oct 2026) · Anthropic Citations API and web search tool documentation (undated; read 4 Oct 2026) · Anthropic crawler documentation (updated 7 Apr 2026) · Lewis et al., RAG, NeurIPS 2020 (arXiv:2005.11401) · Nogueira & Cho (arXiv:1901.04085) · Thakur et al., BEIR (arXiv:2104.08663) · Liu et al., Lost in the Middle, TACL 2024 (arXiv:2307.03172) · Cohere Rerank documentation (undated; read 4 Oct 2026) · Aggarwal et al., GEO, KDD 2024 · Zhang, He & Yao (arXiv:2604.25707)