在vm下hadoop集群搭建

《在vm下hadoop集群搭建》由會員上傳分享，免費在線閱讀，更多相關(guān)內(nèi)容在教育資源-天天文庫。

1、.（原創(chuàng)）VM下搭建hadoop集群一．前期準(zhǔn)備1.安裝VMware_Workstation_wmb2.安裝三臺CentOS-6.3-i386-bin-DVD1Master；192.168.66.174Slave1：192.168.66.171Slave2：9:1.168.66.173二．安裝步驟：（在安裝centos時就把pc的名字改好，免得后面改hostname）1.在每臺pc上的/etc/hosts中加入：127.0.0.1localhost192.168.66.174master192.168.66.171slave1192.168.66.173slave22.在每臺pc安裝j

2、ava：在/etc/profile中加入：exportJAVA_HOME=/usr/local/java/jdk1.6.0_45exportPATH=:$JAVA_HOME/bin:/sbin:/usr/bin:/usr/sbin:/binexportCLASSPATH=.:$JAVA_HOME/lib3.配置SSH無密碼登錄：...在每臺pc上面：ssh-keygen-trsacat~/.ssh/id_rsa.pub>>~/.ssh/authorized_keyschmod700.sshchmod644authorized_keys編輯sshd配置文件/etc/ssh/sshd_co

3、nfig，把#AuthorizedKeysFile.ssh/authorized_keys前面的注釋取消掉。在master上面：ssh-copy-id-i~/.ssh/id_rsa.pubroot@slave1ssh-copy-id-i~/.ssh/id_rsa.pubroot@slave2重啟servicesshdrestart在slave1上面：ssh-copy-id-i~/.ssh/id_rsa.pubroot@masterssh-copy-id-i~/.ssh/id_rsa.pubroot@slave2重啟servicesshdrestart在slave2上面：ssh-copy

4、-id-i~/.ssh/id_rsa.pubroot@slave1ssh-copy-id-i~/.ssh/id_rsa.pubroot@master重啟servicesshdrestart再sshslave1;sshslave2;sshmaster驗證...**遇到Agentadmittedfailuretosignusingthekey解決方式使用ssh-add指令將私鑰加進來（根據(jù)個人的密匙命名不同更改id_rsa）#ssh-add~/.ssh/id_rsa**遇到ssh:connecttohostmasterport22:Noroutetohost是由于ip地址問題，檢查/etc

5、/hosts4.配置hadoop下載hadoop后scphadoop-1.2.0.tar.gzroot@slave1:/usr/localtarxzvfhadoop-1.2.0.tar.gzmvhadoop-1.2.0/usr/local/hadoop在profile和./bashrc中加入：exportHADOOP_HOME=/usr/local/hadoopexportPATH=$PATH:$HADOOP_HOME/bin配置hadoop-env.sh設(shè)置JAVA_HOME配置core-site.xmlfs.default.name

6、alue>hdfs://master:9000true...hadoop.tmp.dir/home/hadoop/hadoop/data/hdfs/tmp配置mapred-site.xmlmapred.job.trackermaster:9001配置hdfs-site.xml

7、ame>dfs.name.dir/home/hadoop/hadoop/data/hdfs/nametruedfs.data.dir/home/hadoop/hadoop/data/hdfs/datatrue