資源描述：
《convolutional neural networks for sentence》由會(huì)員上傳分享，免費(fèi)在線閱讀，更多相關(guān)內(nèi)容在教育資源-天天文庫。
1、ConvolutionalNeuralNetworksforSentenceClassi cationConvolutionalNeuralNetworksforSentenceClassi cationYoonKimNewYorkUniversity1/34ConvolutionalNeuralNetworksforSentenceClassi cationAgendaWordEmbeddingsClassi cationRecursiveNeuralTensorNetworksConvolutionalNeuralNetwor
2、ksExperimentsConclusion2/34ConvolutionalNeuralNetworksforSentenceClassi cationWordEmbeddingsDeeplearninginNaturalLanguageProcessingIDeeplearninghasachievedstate-of-the-artresultsincomputervision(Krizhevskyetal.,2012)andspeech(Gravesetal.,2013).INLP:fastbecoming(alread
3、yis)ahotareaofresearch.IMuchoftheworkinvolveslearningwordembeddingsandperformingcompositionoverthelearnedembeddingsforNLPtasks.3/34ConvolutionalNeuralNetworksforSentenceClassi cationWordEmbeddingsWordEmbeddings(orWordVectors)ITraditionalNLP:Wordsaretreatedasindices(or
4、one-hot"vectorsinRV)IEverywordisorthogonaltooneanother.Iwmotherwfather=0ICanweembedwordsinRDwithDVsuchthatsemanticallyclosewordsarelikewise`close'inRD?(i.e.wmotherwfather>0)IYes!IDon't(necessarily)needdeeplearningforthis:LatentSemanticAnalysis,LatentDirichletAlloc
5、ation,orsimplecontextcountsallgivedenserepresentations.4/34ConvolutionalNeuralNetworksforSentenceClassi cationWordEmbeddingsNeuralLanguageModels(NLM)IAnotherwaytoobtainwordembeddings.IWordsareprojectedfromRVtoRDviaahiddenlayer.IDisahyperparametertobetuned.IVariousarch
6、itecturesexist.Simpleonesarepopularthesedays(right).IVeryfast
7、cantrainonbillionsoftokensinonedaywithasinglemachine.Figure1:SkipgramarchitectureofMikolovetal.(2013)5/34ConvolutionalNeuralNetworksforSentenceClassi cationWordEmbeddingsLinguisticregularitiesintheobtainede
8、mbeddingsIThelearnedembeddingsencodesemanticandsyntacticregularities:Iwbig
