Skip to content

Ranking formula

PoiSearchIndex::search (crates/farnsworth-core/src/lib.rs) orders search hits along two independent axes on top of the base text score. This page is the precise math behind it, and where each piece stands relative to the real production formula it's modeled on (see Major finding: ranking is not Farnsworth's job for why exact reproduction only mattered for one of the two axes).

Where each term is applied is not symmetric. The popularity term (popularity_term) runs inside tantivy's own collector, via TopDocs::tweak_score, reading popularity as a FAST field - it decides which candidates tantivy selects in the first place, not just their order afterward. The distance term (apply_distance_skew) runs as a plain post-collection sort over whatever candidates the collector already returned, since lat/lon aren't fast fields (yet). This matters at real scale: a text-score-only collection pool can be too small to even contain the globally popular result (see the worked example below, and the "ranking during collection, not after" note in world-scale.md) - which is exactly why popularity needed the collector-level fix and distance, so far, hasn't.

The combined score - superseded once, keep reading

An earlier version of this doc had popularity sitting additively on top of raw text_score: \(\text{text\_score} + \text{popularity\_skew} \cdot \frac{1}{\max(1,n)} \cdot \text{sat}(p,\pi)\). That's gone. Once the real production fuzzy-score algorithm was ported (fuzzy_score in ranking.rs, replacing tantivy's own BM25 text_score as the thing being ranked), a real bug surfaced live: ?q=rue+de+la+paix&popularity_skew=1&lat=... showed "Square Rue de la Paix" (an imperfect fuzzy match, just popular) outranking the real exact address (fuzzy = 1.0) - popularity's additive bonus pushed the imperfect match's score past the perfect one's ceiling. A bare .min(1.0) clamp on the sum wasn't enough either: it let the imperfect match reach exactly 1.0 too, tying the perfect address and then winning downstream on kind_priority.

Real fix, a gap-closing formula instead of a straight sum:

\[ \text{score} = \text{fuzzy} + (1 - \text{fuzzy}) \cdot \text{popularity\_bonus} \]
\[ \text{popularity\_bonus} = \text{clamp}\!\left(\text{popularity\_skew} \cdot \frac{1}{\max(1, n)} \cdot \text{sat}(p, \pi),\; 0, 1\right) \]

where \(\text{fuzzy} \in [0,1]\) is fuzzy_score's own output (the real, ported production ranking algorithm - see the cascade section below), \(n\) is the query's token count, \(p\) is the candidate's popularity value, and \(\pi\) is the popularity pivot. Popularity can only close part of the remaining distance toward a perfect match - it can break ties and near-ties among otherwise-comparable candidates (its original, tested purpose), but it can never push an imperfect match above, or even level with, a genuinely better one from another kind. popularity_skew = 0.0 disables the bonus entirely, reducing to score = fuzzy.

Distance is unaffected by this change - apply_distance_skew still runs as a separate, post-sort linear penalty on top of this score (see below), not folded into the formula above.

Saturation (popularity axis) - reproduced exactly

\[ \text{sat}(p, \pi) = \frac{p}{p + \pi} \]

A diminishing-returns curve: approaches 1 as \(p \to \infty\), never lets one outlier value dominate outright. This is tantivy code standing in for OpenSearch's own rank_feature saturation function - not approximated, the real shape.

Pivot \(\pi\), per POI index, is the mean popularity across every record that has a value:

\[ \pi = \frac{1}{|\{p_i\}|}\sum_i p_i \]

This matches OpenSearch's real default exactly: RankFeatureSaturation.pivot in production is only ever passed as None, and OpenSearch's own implementation falls back to the field's corpus mean as the pivot when the DSL omits one ({"saturation": {"pivot": self.pivot} if self.pivot else {}} in localities/elasticsearch/builders/dsl.py) - a data-driven default, not a guessed constant.

Query-length dampening, \(\frac{1}{\max(1, n)}\), is copied verbatim from the real weighting in production:

# localities/elasticsearch/builders/poi_query_builder.py
RankFeatureSaturation(field_name="popularity", boost=1 / max(1, token_count))

Popularity matters most for short, ambiguous queries ("par") and fades out as the query gets longer and more specific ("parville center") - the same shape production uses, not an independent invention.

popularity_skew itself is Farnsworth's own knob on top of that reproduced formula: 1.0 reproduces production's weighting exactly, 0.0 disables popularity, >1.0 emphasizes it beyond what production does. Production has no equivalent knob - it always applies at its fixed boost.

fuzzy_score and the real production ranking cascade

fuzzy_score/fuzzy_score_cascade (ranking.rs) are the real production ranking algorithm, reverse-engineered by reading localities/suggestions/processor/ directly, not guessed: DEFAULT_PIPELINE cascades per-property scorers (name -> street_name -> locality -> admin levels -> postal_code...) via a greedy in-order token alignment (rapidfuzz's Indel.normalized_similarity, score_cutoff = 0.7), with a token already matched by an earlier property excluded from being re-scored by a later one (ignore_matches, carried forward cumulatively). The final blended score production actually sorts by is (input_coverage * 3 + properties_similarity_score) / 4, where properties_similarity_score is the sum of all matched properties' raw similarity sums divided by the sum of their token counts - not a mean of per-property averages, a real bug this session got wrong once before fixing it against the real formula.

GeocodeIndex/LocalitiesIndex use fuzzy_score_cascade with a real second corroborating property (street name + locality context, or locality name + admin context) for ranking. PoiSearchIndex::search above still only calls single-property fuzzy_score (name only) - wiring a real corroborating property into POI ranking the same way is a natural next step, not done here.

name_coverage, not blended fuzzy_score, is the real exemption-gate signal. Both GeocodeIndex and LocalitiesIndex use an exact-or-near-exact name match to bypass their geo radius/population floors (see world-scale.md and localities.md). Gating that bypass on the blended fuzzy_score was tried first and found to be a real bug: input_coverage's *3 weight trivially maxes out for any short query that's a textual subset of a longer name - "boulangerie" against "Rue de la Boulangerie" scored a deceptively high 0.81 blended despite covering only 1 of 4 real name tokens. Fixed by splitting a separate name_coverage out of fuzzy_score_components and gating the exemption on name_coverage >= 0.9 ("near-complete") instead - a stricter, differently-shaped signal than the 0.7 per-token cutoff used elsewhere in the cascade.

Distance term - a deliberate simplification, not a reproduction

Farnsworth's distance term is a plain linear penalty:

\[ -\,\text{distance\_skew} \cdot d \]

Production's real geo clause, DistanceFeature (dsl.py), is a different shape entirely - OpenSearch's own distance_feature decay function:

\[ \text{relevance} = \text{boost} \cdot \frac{\text{pivot}}{\text{pivot} + d} \]

(the same saturation-family shape as popularity's, just applied to distance instead of a static value - and complicated in production's ES 7.9 target by a real boost-squaring bug, worked around by wrapping the clause in a bool and applying boost there instead of letting distance_feature square it itself).

Farnsworth does not reproduce this. Per the ranking finding: production's OpenSearch _score - including distance_feature's own contribution - is discarded entirely for final ordering. Real relevance ranking happens downstream in Python (DEFAULT_PIPELINE / LocationScorer.geo_bias_score), engine-independent. distance_feature's only real job in production is recall shaping, not precision-matching a decay curve. Farnsworth's linear penalty is good enough for that same job and was never rewritten to match the decay shape - unlike popularity, which got the "read the real code" correction specifically because it is meant to demonstrate production's actual weighting, not just recall.

Worked example

Two POIs, query "par" (\(n=1\)), both an equally imperfect fuzzy match against their own name (say \(\text{fuzzy} = 0.4\) for both - the point here is popularity breaking a near-tie, not fuzzy quality), no bias point, popularity_skew = 1.0, corpus pivot \(\pi = 50.5\) (mean of 1 and 100):

popularity \(p\) \(\text{sat}(p,\pi)\) popularity_bonus \(\text{score} = \text{fuzzy} + (1-\text{fuzzy})\cdot\text{bonus}\)
Parville (small town) 1 \(1/51.5 \approx 0.019\) \(0.019\) \(0.4 + 0.6 \cdot 0.019 \approx 0.411\)
Paris 100 \(100/150.5 \approx 0.664\) \(0.664\) \(0.4 + 0.6 \cdot 0.664 \approx 0.798\)

With fuzzy otherwise tied, Paris's popularity contribution dominates and it sorts first - with zero location context, which a distance-only skew could never do. Note the ceiling the gap-closing formula enforces: however popular Parville got, its score could never cross 1.0, and could never exceed a genuinely perfect (fuzzy = 1.0) match elsewhere. See popularity_skew_surfaces_the_globally_important_result_with_no_location_bias and popularity_skew_cannot_push_an_imperfect_match_above_a_perfect_one in lib.rs for the executable versions of this.

Real-scale worked example - why in-collector ranking matters

"buckingham" matches 150 real POIs in the full world dataset. Buckingham Palace has real popularity 140 against a real corpus mean pivot of 2.43:

\[ \text{sat}(140, 2.43) = \frac{140}{142.43} \approx 0.983 \]

An overwhelming popularity contribution - and yet the first implementation (popularity resorted after TopDocs::with_limit(overfetch), overfetch = \max(50, \text{limit} \cdot 10)) never surfaced it at limit = 5. tantivy's own text-score tie-breaking picked its top 50 out of 150 tied-score candidates with no knowledge of popularity, and Buckingham Palace wasn't among them - the resort had nothing to promote. \(0.983\) beats every other candidate's contribution, but only for candidates the collector already decided to keep.

Fixed by moving popularity_term inside TopDocs::tweak_score (popularity as a tantivy FAST field): the collector now ranks all 150 candidates by \(\text{text\_score} + \text{popularity\_term}\) directly (still real BM25 text_score at this stage - the collector's own job is just keeping the right 150-candidate pool, fuzzy_score's gap-closing re-rank happens afterward, same split described in the combined-score section above), so Buckingham Palace is visible at collection time, not just to a pool that already excluded it.

The overfetch window - a separate, token-count-scaled real cost

All three indices overfetch a real BM25-ordered candidate pool before re-ranking it by fuzzy_score/fuzzy_score_cascade (ranking::overfetch_window(limit, token_count) in ranking.rs) - the adversarial case that justifies a large pool ("rue de la paix paris" needing ~2000 candidates before its true best match, excluded from BM25's own top ranks by an unrelated decoy's EXACT_MATCH_BOOST, was even reachable) only showed up at 5 tokens. A flat 2000 paid that cost on every query, including a single-token one like "paris" with none of the adversarial shape. Real profiled cost (samply, this session): doc-store decompression alone was ~11-14% of real query time, paid per candidate fetched - scaling the floor down for short queries (1 token -> 100, 2 -> 500, 3+ -> 2000, all still .max(limit*100)) measurably helped: ?q=paris&limit=8&popularity_skew=1 dropped from ~250ms to ~170ms on the real running server, with the 5-token decoy case still resolving correctly.