Lexical MeSH linking converts query text into exact MeSH candidates. It combines deterministic phrase windows, exact vocabulary names, and observed token-to-MeSH associations from indexed biomedical records.
unigrambigramtrigramexact termassociation weight
Phrase construction: Overlapping windows preserve local meaning
| Window | Example output | Role |
|---|---|---|
| Unigram | heart · attack · risk · stent | Broad standalone evidence |
| Bigram | heart attack · attack risk · risk stent | Compact medical concepts |
| Trigram | heart attack risk · attack risk stent | Narrow contextual evidence |
Candidate evidence: Exact names precede learned associations
query → normalized tokens → phrase windows → exact MeSH names → corpus associations → weighted candidate ranking
Output contract: Ranked exact identifiers with inspectable evidence
- Return exact MeSH identifiers rather than generated labels.
- Retain the phrase, source count, and match class behind every score.
- Add a sparse model only when calibration improves held-out accuracy.
Offline aggregation: One record contributes one vote per pair
Corpus aggregation occurs before live retrieval. Each record joins its searchable text with assigned MeSH identifiers. Repetition inside one record does not multiply support.
- Join title, abstract, conditions, and intervention fields.
- Run the deterministic tokenizer once.
- Collect each unique token and assigned identifier.
- Create every unique token-identifier pair.
- Increment each pair once for that record.
Association table: Support, prevalence, and corpus size
| Value | Meaning | Example |
|---|---|---|
| Phrase | Normalized word or phrase | heart attack |
| Phrase length | One, two, or three words | two words |
| MeSH identifier | The exact vocabulary term | D009203 |
| Source | Which corpus the association came from | PubMed |
| Records with both | Records containing the phrase and the label | 842,193 |
| Records with the phrase | Records containing the phrase | 896,400 |
| Records with the label | Records carrying the label | 1,104,200 |
| Records in total | Usable source records | 26,000,000 |
Ranking: Specific phrases outrank raw popularity
- Reward exact descriptor and official synonym matches.
- Reward longer supported phrases.
- Discount labels occurring across most records.
- Reward agreement between PubMed and ClinicalTrials evidence.
Evaluation: Predict labels hidden before counting
Records are split before association counts are built. Test labels cannot influence the mapping table. The linker sees only normalized text and predicts labels already assigned by experts.
| Metric | Decision answered |
|---|---|
| Exact top-one | How often the first identifier matches a hidden label |
| Recall at five | How many hidden labels appear in the first five results |
| Coverage | How many test records receive a confident result |
| Latency | Time for tokenization, lookup, and ranking |