資源描述：
《Neural Turing Machines》由會員上傳分享，免費在線閱讀，更多相關(guān)內(nèi)容在學術(shù)論文-天天文庫。
1、NeuralTuringMachinesAlexGravesgravesa@google.comGregWaynegregwayne@google.comIvoDanihelkadanihelka@google.comGoogleDeepMind,London,UKAbstractWeextendthecapabilitiesofneuralnetworksbycouplingthemtoexternalmemoryre-sources,whichtheycaninteractwithbyattentionalprocesses.Thecombinedsyst
2、emisanalogoustoaTuringMachineorVonNeumannarchitecturebutisdifferentiableend-to-end,allowingittobeef?cientlytrainedwithgradientdescent.Preliminaryresultsdemon-stratethatNeuralTuringMachinescaninfersimplealgorithmssuchascopying,sorting,andassociativerecallfrominputandoutputexamples.1I
3、ntroductionComputerprogramsmakeuseofthreefundamentalmechanisms:elementaryoperations(e.g.,arithmeticoperations),logical?owcontrol(branching),andexternalmemory,whicharXiv:1410.5401v2[cs.NE]10Dec2014canbewrittentoandreadfrominthecourseofcomputation(VonNeumann,1945).De-spiteitswide-rang
4、ingsuccessinmodellingcomplicateddata,modernmachinelearninghaslargelyneglectedtheuseoflogical?owcontrolandexternalmemory.Recurrentneuralnetworks(RNNs)standoutfromothermachinelearningmethodsfortheirabilitytolearnandcarryoutcomplicatedtransformationsofdataoverextendedperiodsoftime.More
5、over,itisknownthatRNNsareTuring-Complete(SiegelmannandSontag,1995),andthereforehavethecapacitytosimulatearbitraryprocedures,ifproperlywired.Yetwhatispossibleinprincipleisnotalwayswhatissimpleinpractice.Wethereforeenrichthecapabilitiesofstandardrecurrentnetworkstosimplifythesolutiono
6、falgorithmictasks.Thisenrichmentisprimarilyviaalarge,addressablememory,so,byanalogytoTuring’senrichmentof?nite-statemachinesbyanin?nitememorytape,we1dubourdevicea“NeuralTuringMachine”(NTM).UnlikeaTuringmachine,anNTMisadifferentiablecomputerthatcanbetrainedbygradientdescent,yieldinga
7、practicalmechanismforlearningprograms.Inhumancognition,theprocessthatsharesthemostsimilaritytoalgorithmicoperationisknownas“workingmemory.”Whilethemechanismsofworkingmemoryremainsome-whatobscureatthelevelofneurophysiology,theverbalde?nitionisunderstoodtomeanacapacityforshort-termsto
8、rageofinformationan
