Spelling Variant Patterns - ES (Edit Distance and Sorted Distance)

I. Introduction

After using the normalization and MES (Metaphone, Edit distance & Sorted Distance) more terms (98.5% in Lexicon.2014) are identified and grouped as spelling variants. However, spelling variants such as:

• Abkhasian|Abkhazian
• aptha|aphtha
• ensoul|insoul
• toxic edema|toxic oedema
• vinleucinol|vinleukinol
• wholly|wholely
• zygapophyseal|zygapophysial
• ...

are not identified in the previous steps (1-2). The following algorithm are used for further spelling variants identifications.

• EditDistance.java
• SortedDistance.java
• GroupSpVarByES.java

II. Algorithm Details

For those terms are not identified in any spelling variants group, ES algorithm is used to checks two properties:

• Limited Edit distance (1: 80.26%, 2:95.41%)
• The smallest sorted distance:
  • All terms are sorted in the alphabetical order. The difference in indexes of two terms is the sorted distance.
  • The closer the sorted distance of two terms is, the more chance that these terms are spelling variants

III. Algorithm Examples

Example term	Norm	Metaphone	Edit Distance
Edit Distance = 1
Abkhasian|Abkhazian	abkhasian|abkhazian	ABKHXN|ABKHSN	1
aptha|aphtha	aptha|aphtha	AP0|AF0	1
ensoul|insoul	ensoul|insoul	ENSL|INSL	1
toxic edema|toxic oedema	toxicedema|toxicoedema	TKSSTM|TKSKTM	1
vinleucinol|vinleukinol	vinleucinol|vinleukinol	FNLSNL|FNLKNL	1
wholly|wholely	wholly|wholely	WL|WLL	1
zincemia|zincaemia	zincemia|zincaemia	SNSM|SNKM	1
zygapophyseal|zygapophysial	zygapophyseal|zygapophysial	SKPFSL|SKPFXL	1

Example term	Norm	Metaphone	Edit Distance
Edit Distance = 2
disconnexion|disconnection	disconnexion|disconnection	TSKNKSN|TSKNKXN	2
racketball|racquetball	racketball|racquetball	RKTBL|RKKTBL	2
subtly|subtlely	subtly|subtlely	SBTL|SBTLL	2
type 4 collagenase|type IV collagenase	typefourcollagenase|typeivcollagenase	TPKLJNS|TPFKLJN	2
woadwax|woadwaxen	woadwax|woadwaxen	WTWKS|WTWKSN	2