The MeSH ranker uses one row for each query-candidate pair. The stable row key remains unchanged while feature producers append values. Training and inference call the same feature functions.
500 candidates per query
53 feature columns
7 construction operations
Row contract: Stable identity with additive feature columns
Construction path: Seven ordered operations
- Read record fields and expected labels.
- Tokenize with the deterministic query tokenizer.
- Retrieve 500 frequency candidates.
- Join vocabulary and hierarchy attributes.
- Run semantic and hierarchy encoders.
- Append named feature-stage outputs.
- Write features, relevance labels, and query groups.
Training and inference parity: Producers stay identical
| Concern | Training | Inference |
|---|---|---|
| Record | Historical labeled record | Live query record |
| Candidates | Frequency top 500 | Frequency top 500 |
| Features | Shared producers | Shared producers |
| Output | Row plus label | Row for scoring |
Feature producers: Six sources supply every value
| Producer | Supplies |
|---|---|
| Record loader | PubMed titles and abstracts; trial titles, summaries, conditions, interventions |
| Frequency index | 500 candidates, association scores, token counts, source support, original rank |
| MeSH catalog | Names, synonyms, scope notes, identifiers, hierarchy paths |
| Corpus aggregates | Document frequencies, prevalence, acronym maps, ambiguity |
| Batch encoders | Source MiniLM, hard-negative MiniLM, SapBERT, hierarchy roots |
| Cross-encoder | Joint scores for the strongest 50 candidates |
Appended columns: Values 31 through 53
| Columns | Values | Generation |
|---|---|---|
| 31–35 | Preferred overlap, synonym overlap, keyword match, acronym count, character similarity | Per candidate from stored vocabulary values |
| 36–39 | Separate record-field overlaps | Preserve caller-supplied fields |
| 40–44 | Two source MiniLM scores, hard-negative score, two prevalence values | Batch models plus stored aggregates |
| 45–48 | Query hierarchy probabilities, mismatch, relative summaries | Once per query and candidate group |
| 49–50 | Cross-encoder and SapBERT scores | Top 50 joint scoring plus stored vectors |
| 51–53 | Name ambiguity, raw rank, reciprocal rank | Catalog aggregate and retriever order |
Leakage control: Fit every producer on training records only
- Preserve the stable record hash and 10,021 untouched test groups.
- Build dictionaries, aggregates, keyword statistics, and encoders from training records.
- Run producers on all splits without reading expected labels.
- Add binary relevance only after all 53 features exist.
- Train the ranking model, which learns to order candidates within each query group.