資源描述:
《convolutional neural networks for sentence》由會(huì)員上傳分享,免費(fèi)在線閱讀,更多相關(guān)內(nèi)容在教育資源-天天文庫。
1、ConvolutionalNeuralNetworksforSentenceClassicationConvolutionalNeuralNetworksforSentenceClassicationYoonKimNewYorkUniversity1/34ConvolutionalNeuralNetworksforSentenceClassicationAgendaWordEmbeddingsClassicationRecursiveNeuralTensorNetworksConvolutionalNeuralNetwor
2、ksExperimentsConclusion2/34ConvolutionalNeuralNetworksforSentenceClassicationWordEmbeddingsDeeplearninginNaturalLanguageProcessingIDeeplearninghasachievedstate-of-the-artresultsincomputervision(Krizhevskyetal.,2012)andspeech(Gravesetal.,2013).INLP:fastbecoming(alread
3、yis)ahotareaofresearch.IMuchoftheworkinvolveslearningwordembeddingsandperformingcompositionoverthelearnedembeddingsforNLPtasks.3/34ConvolutionalNeuralNetworksforSentenceClassicationWordEmbeddingsWordEmbeddings(orWordVectors)ITraditionalNLP:Wordsaretreatedasindices(or
4、one-hot"vectorsinRV)IEverywordisorthogonaltooneanother.Iwmotherwfather=0ICanweembedwordsinRDwithDVsuchthatsemanticallyclosewordsarelikewise`close'inRD?(i.e.wmotherwfather>0)IYes!IDon't(necessarily)needdeeplearningforthis:LatentSemanticAnalysis,LatentDirichletAlloc
5、ation,orsimplecontextcountsallgivedenserepresentations.4/34ConvolutionalNeuralNetworksforSentenceClassicationWordEmbeddingsNeuralLanguageModels(NLM)IAnotherwaytoobtainwordembeddings.IWordsareprojectedfromRVtoRDviaahiddenlayer.IDisahyperparametertobetuned.IVariousarch
6、itecturesexist.Simpleonesarepopularthesedays(right).IVeryfast
7、cantrainonbillionsoftokensinonedaywithasinglemachine.Figure1:SkipgramarchitectureofMikolovetal.(2013)5/34ConvolutionalNeuralNetworksforSentenceClassicationWordEmbeddingsLinguisticregularitiesintheobtainede
8、mbeddingsIThelearnedembeddingsencodesemanticandsyntacticregularities:Iwbig