資源描述：
《Scaling Conditional Random Fields for Natural Language Processing》由會(huì)員上傳分享，免費(fèi)在線閱讀，更多相關(guān)內(nèi)容在行業(yè)資料-天天文庫。
1、ScalingConditionalRandomFieldsforNaturalLanguageProcessingTrevorA.CohnSubmittedintotalful?lmentoftherequirementsofthedegreeofDoctorofPhilosophyJanuary,2007DepartmentofComputerScienceandSoftwareEngineeringFacultyofEngineeringUniversityofMelbourneAbstractThisthesisdealswiththeuseofConditionalRandom
2、Fields(CRFs;La?ertyetal.(2001))forNaturalLanguageProcessing(NLP).CRFsareprobabilisticmodelsforsequencelabellingwhichareparticularlywellsuitedtoNLP.Theyhavemanycompellingadvan-tagesoverotherpopularmodelssuchasHiddenMarkovModelsandMaximumEntropyMarkovModels(Rabiner,1990;McCallumetal.,2001),andhaveb
3、eenappliedtoanum-berofNLPtaskswithconsiderablesuccess(e.g.,ShaandPereira(2003)andSmithetal.(2005)).Despitetheirapparentsuccess,CRFssu?erfromtwomainfailings.Firstly,theyoftenover-?tthetrainingsample.Thisisaconsequenceoftheirconsiderableexpres-sivepower,andcanbelimitedbyaprioroverthemodelparameters
4、(ShaandPereira,2003;PengandMcCallum,2004).TheirsecondfailingisthatthestandardmethodsforCRFtrainingareoftenveryslow,sometimesrequiringweeksofprocessingtime.Thise?ciencyproblemislargelyignoredincurrentliterature,althoughinpractisethecostoftrainingpreventstheapplicationofCRFstomanynewmorecomplextask
5、s,andalsopreventstheuseofdenselyconnectedgraphs,whichwouldallowformuchricherfeaturesets.Thisthesisaddressestheissueoftraininge?ciency.Firstly,wedemonstratethattheasymptotictimecomplexityofstandardtrainingforalinearchainCRFisquadraticinthesizeofthelabelset,linearinthenumberoffeaturesandalmostquadr
6、aticinthesizeofthetrainingsample.Thecostofinferenceincyclicgraphs,suchaslatticestructuredDynamicCRFs(Suttonetal.,2004),isevengreater.ThecomplexityoftraininglimitstheapplicationofCRFstolargeandcomplextasks.Wecomparetheaccuracyofanumberofpopularapproximatetrainingtechniques,whichcangreatlyreducethe
7、trainingcost.However,formosttasksthissavingiscoupledwithasubstantiallossinaccuracy.Forthisreasonweproposetwonoveltrainingmethods,whichbothreducetheresourcerequirementsandimprovethescalabilityoftraining,suchthatCRFscanb
