Disambiguation
When a word has more than one reformed spelling, something has to choose. Euspell reads the surrounding words and decides — with grammar rules for the common cases and a trained model for the hardest one.
The context window
Text is tokenized per block-level element, so a sentence split across inline markup (I <em>record</em> this) is one stream. Each rule sees a fixed five-slot view — two tokens before, the target, two after — built by contextWindow(tokens, idx). Slots past a sentence boundary arrive as a frozen BOUNDARY sentinel, so a missing neighbor is itself a signal.
[ w-2 , w-1 , TARGET , w+1 , w+2 ] ▲ ▲ past a sentence edge -> BOUNDARY (tag ZB)
Each token carries its full candidate CLAWS7 tag set from the lexicon (e.g. records → NN2|VVZ), not a single resolved tag. Rules test those sets with prefix and exact matching.
Three tiers of resolution
The converter's route()dispatches on the entry's encoding and part-of-speech pair to one of three mechanisms.
1 · General part-of-speech predicates
A small set of hand-written predicates decide whole grammatical classes — not one function per word. They vote over the token window:
is_VVZ_svm(tokens, idx) // NN2|VVZ: plural noun vs third-person verb, via the trained SVM is_verbal_s(tokens, idx) // genitive 's vs contracted is/has 'z is_verb_VV0(tokens, idx) // noun/adjective vs verb reading is_plural_noun(tokens, idx) is_past_tense(tokens, idx) is_past_participle(tokens, idx) is_adjective(tokens, idx)
2 · The NN2|VVZ model
The dominant hard case — plural noun vs third-person-singular verb (records, anchors) — is a trained linear SVM, not a hand rule. It reads the window and scores noun-vs-verb; retrain it with npm run gen:svm, which writes src/disambig/vvz-svm.js.
3 · Per-word semantic rules
About 30 words split on pronunciation, not grammar — the two readings share a part of speech, so no POS rule can separate them. Each gets its own file:
/**
* @returns {'reed' | 'red' | null} null -> use the fallback spelling
*/
export function disambiguate_read(tokens, idx) { … }read heteronym family (read, reread, misread, proofread, copyread, foreread, outread, sightread) shares one context engine, read-verb.js; each word maps its base/past/null result to its own euspellings.Which mechanism fires
| Case | Example | Resolved by |
|---|---|---|
| NN2 | VVZ | records → recordz, leaves → leavz | Trained SVM (is_VVZ_svm) |
Clitic 's | cat's genitive vs he's | is_verbal_s predicate |
| Past vs participle vs adjective | 022 | POS predicates over the window |
| Semantic sense split | read, bow, tear | Per-word semantic/ rule |
| Number pair | 702 | Route by encoding to the plural |
Contractions carry sequences
A contraction is one surface piece but encodes a sequence of grammatical words. anybody's is analyzed as PN1 GE|PN1 VBZ|PN1 VHZ; the tokenizer pushes one pseudo-token per position, so a neighbor to the left of he's sees PPHS1 and one to the right sees VBZ|VHZ — correct adjacency, with the rules unchanged.