SpVar - Phonetic Tests

I. Introduction

SpVarNorm is used to find spVars from a corpus by normalizing lexical information. The other chracteristic of spVars is having same pronunciation. Serveral Phonetics algorithms are tests for this task.

• Lexical Tools - Metaphone

• Apache Commons Codec 1.10
  • Metaphone
  • Double metaphone
  • Soundex
  • Refined Soundex
  • Caverphone 1.0
  • Caverphone 2.0
  • Cologne Phonetic
• Metaphone 3.0

II. Processes

• Most of phonetic algorithmis very agreesive and result in high recall and low precision.
• We perfromed two types of tests: unit test and application test.

III. Results

• Unit Tests:
  We tested all above algorithms. Soundex, RefinedSoundex, Caverphone 1.0, and Cologne are too aggresive and results in poor precision.

• Application Tests
  • First Test:

    Step	Methods	Edit Distance	Sample No.	ret-rel	ret-irrel	notRet-rel	notRet-irrel	Precision	Recall	F1	Accuracy	Notes
    0	GoldStd	N/A	867,728	379,269	0	0	488,459	1.0000	1.0000	1.0000	1.0000	1 min.
    1	Norm	N/A	867,728	305,309	3,495	73,960	484,964	0.9887	0.8050	0.8874	0.9107	1 min.
    2	M2ES	2	867,728	364,396	64,622	14,873	423,837	0.8494	0.9608	0.9017	0.9084	2 hr.
    2	M3ES	2	867,728	364,870	64,371	14,399	424,088	0.8500	0.9620	0.9026	0.9092	2 hr.
    2	C2ES	2	867,728	366,788	96,116	12,481	392,343	0.7924	0.9671	0.8711	0.8748	2 hr.
    2	M2CES	2	867,728	363,609	57,259	15,660	431,200	0.8640	0.9587	0.9089	0.9160	2 hr.
    2	M3CES	2	867,728	363,593	56,953	15,676	431,506	0.8646	0.9587	0.9092	0.9163	2 hr.

  • 2nd Test (from new spVarNorm):

    Step	Methods	Edit Distance	Sample No.	ret-rel	ret-irrel	notRet-rel	notRet-irrel	Precision	Recall	F1	Accuracy	Notes
    0	GoldStd	N/A	867,728	379,269	0	0	488,459	1.0000	1.0000	1.0000	1.0000	1 min.
    1	Norm	N/A	867,728	305,309	3,495	73,960	484,964	0.9887	0.8050	0.8874	0.9107	1 min.
    2	M2ES	2	867,728	363,956	62,979	15,313	425,480	0.8525	0.9596	0.9029	0.9098	113 min.
    2	M3ES	2	867,728	364,447	62,726	14,822	425,733	0.8532	0.9609	0.9038	0.9106	114 min.
    2	C2ES	2	867,728	366,574	94,654	12,695	393,805	0.7948	0.9665	0.8723	0.8763	133 min.
    2	M2CES	2	867,728	363,222	55,618	16,047	432,841	0.8672	0.9577	0.9102	0.9174	96 min.
    2	M3CES	2	867,728	363,203	55,310	16,066	433,149	0.8678	0.9567	0.9105	0.9177	97 min.

  • M2CES and M3CES have very similar results. Considering the recall and software maturity, M2CES model is used to on MEDLINE n-gram distillled set for retrieve LMS candidates.