Finding evidence in Academica moves through three clear stages. Original files are kept in an archive. A searchable database holds the clean fields and full text. A separate search index ranks results and returns pointers back to those records.
original sourceclean recorddatabasesearch indexfull text
Each stage has one job
original releases and feeds → read and clean → records in the database → search index → ranked matches → full text from the database
| Stage | Owns | Does not own |
|---|---|---|
| Original source | The unchanged file | Clean, queryable fields |
| Cleaning | Reading and tidying records | Deciding what ranks first |
| Database | Fields, full text, and analysis | Ranking free-text search |
| Search index | Ranking and keyword scoring | The full, authoritative text |
Rank first, then fetch the full text
free-text query → search index ranks the top matches → their identifiers → full records from the database → evidence
Keep what matters, drop the rest before indexing
- Keep stable identifiers, text, entities, dates, and a few useful filters.
- Standardize publication dates before storing them.
- Keep time-based analysis in the database.
- Rebuild the search index after the records change shape.
Every source reduces to a few useful fields
| Field | Examples | Purpose |
|---|---|---|
| Identity | Article ID, event ID, filing number | Joining records and removing duplicates |
| Title | Article, event, filing, or notice title | Display and keyword ranking |
| Text | Abstract, narrative, description, filing body | The evidence itself |
| Entity | Drug, device, sponsor, company | Filtering and linking |
| Date | The real publication or event date | Time filtering and analysis |
| Extra fields | A few source-specific facts | Preserve essential structured detail |
Most sources already carry their own text
| Source | Main text | Fields kept | Cleaning |
|---|---|---|---|
| PubMed | Abstract | about 7 | Light |
| MAUDE device reports | Report narrative | about 10 | Heavy |
| FAERS adverse events | Patient narrative | about 8 | Heavy |
| ClinicalTrials | Detailed description | about 12 | Heavy |
| DailyMed labels | Indications and warnings | about 12 | Heavy |
| Device registry | Device description | about 10 | Heavy |
| Post body | about 7 | Light |