Because of a lapse in government funding, the information on this website may not be up to date, transactions submitted via the website may not be processed, and the agency may not be able to respond to inquiries until appropriations are enacted. The NIH Clinical Center (the research hospital of NIH) is open. For more details about its operating status, please visit cc.nih.gov. Updates regarding government operating status and resumption of normal operations can be found at OPM.gov.

The SPECIALIST Lexicon

Test on Lexicon: for AMIA Initial Submission

Norm, MES, and ES are used in a sequential order to retrieve the most spelling variant groups. This model is tested on Lexicon (inflVars.data) and LRSPL for the recall, precisino, F1, and accuracy. The details are shown as follows:

  • Setup:
    • Test name: LexTest.Amia.1.Init
    • Input File: (Lexicon.2015)
      • inflVars.data
      • LRSPL
    • Software:
      • GoldStd: GetGoldStdFromLex.java.1.AmiaInit
      • Norm: SpVarNorm.java.1.AmiaInit
      • MES: GroupSpVarByMES.java
      • ES: GroupSpVarByES.java

  • Results:

    2015 (Used in AMIA paper initial submission)

    StepMethodsEdit DistanceSample No.ret-relret-irrelnotRet-relnotRet-irrelPrecisionRecallF1AccuracyNotes
    0GoldStdN/A867,728363,21700504,5111.00001.00001.00001.00001 min.
    1NormN/A867,728306,38719,37456,830485,1370.94050.84350.88940.91222 min.
    2MES2867,728355,423173,6477,794330,8640.67180.97850.79670.79096 hr.
    3ES1867,728360,599286,9322,618217,5790.55690.99280.71350.666324 hr.
    4MES3867,728360,956301,0972,261203,4140.54520.99380.70410.65048 min.
    5ES2867,728362,082353,5121,135150,9990.50600.99690.67130.591327 hr.
    6MES4867,728362,159356,1561,058148,3550.50420.99710.66970.58832 min.

  • Discussion:
    • Step 6 is the final results we use for the matcher. Use it as example for calculation check:

      Check ItemCheck numbers
      Total sample no867,728 = 362,159 + 356,156 + 1,058 + 148,355
      Precision 0.5042 = 362,159 / (362,159 + 356,156)
      Recall0.9971 = 362,159 / (362,159 + 1,058)
      F10.6697 = (2 * 0.5042 * 0.9971) / (0.5042 + 0.9971)
      Accuracy0.5883 = (362,159 + 148,355) / 867,728

    • The recall reaches 99.71% while precision, F1, and accuracy are relatively low. Also, the performance is very low. Thus, we have to reduce the size of n-gram by increasing the WC from 30 to 150. Even so, the entire process took more than 14 days to run. Thus, a better model with improve performnace, precision, F1, and accuracy should be developed.