hadoop Safe mode is ON 的解决办法

16343阅读 3评论2012-03-07 playmud
分类:云计算

搭了一个hadoop集群环境,近期总是出现读写文件错误的情况,查看name node的日志显示 (Safe mode is ON)
  1. Safe mode is ON. The ratio of reported blocks 0.2000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
再往上翻
  1. org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /home/hadoop/tmp/mapred/system. Name node is in safe mode.
google 解决方案有三种
1)执行命令:bin/hadoop dfsadmin -safemode leave
dfsadmin -safemode value 参数value的说明如下:
enter - 进入安全模式
leave - 强制NameNode离开安全模式
get -  返回安全模式是否开启的信息
wait - 等待安全模式结束。
2)重新格式化文件系统 =。=
3)bin/hadoop fsck /

第一种方法,需要每次都执行一遍,很纠结~
第二种方法,format的意思你懂得。。。。
第三种方法,如果数据多,那执行起来会很慢,没办法,慢慢等吧。

  1. hadoop@mhadoop:~/hadoop-1.0.0$ bin/hadoop fsck /
  2. FSCK started by hadoop from /192.168.1.110 for path / at Wed Mar 07 17:57:43 CST 2012
  3. .
  4. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0001/job.jar: CORRUPT block blk_-8710406465335110666
  5. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0001/job.jar: MISSING 1 blocks of total size 142465 B..
  6. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0002/job.jar: CORRUPT block blk_8343735010139595189
  7. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0002/job.jar: MISSING 1 blocks of total size 142465 B..
  8. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0004/job.jar: CORRUPT block blk_4796918520991551976
  9. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0004/job.jar: MISSING 1 blocks of total size 142465 B..
  10. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0005/job.jar: CORRUPT block blk_8732649167016211914
  11. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0005/job.jar: MISSING 1 blocks of total size 142465 B..
  12. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0006/job.jar: CORRUPT block blk_2190395249671123528
  13. /tmp/hadoop/tmp/mapred/staging/hadoop/.staging/job_201202290728_0006/job.jar: MISSING 1 blocks of total size 142465 B..
  14. /tmp/hadoop/tmp/mapred/system/jobtracker.info: Under replicated blk_3010497672022840180_1511. Target Replicas is 2 but found 1 replica(s).
  15. .
  16. /user/hadoop/input/README.txt: Under replicated blk_1278803782511847062_1510. Target Replicas is 2 but found 1 replica(s).
  17. ..
  18. /user/hadoop/output/_logs/history/job_201202290728_0007_1330473591943_hadoop_word+count: CORRUPT block blk_-165491155081655193
  19. /user/hadoop/output/_logs/history/job_201202290728_0007_1330473591943_hadoop_word+count: MISSING 1 blocks of total size 13581 B..
  20. /user/hadoop/output/_logs/history/job_201202290728_0007_conf.xml: CORRUPT block blk_359906738089887519
  21. /user/hadoop/output/_logs/history/job_201202290728_0007_conf.xml: MISSING 1 blocks of total size 20344 B..
  22. /user/hadoop/output/part-r-00000: CORRUPT block blk_-3883649133311907523
  23. /user/hadoop/output/part-r-00000: MISSING 1 blocks of total size 158081 B.Status: CORRUPT
  24. Total size: 905701 B
  25. Total dirs: 20
  26. Total files: 11
  27. Total blocks (validated): 10 (avg. block size 90570 B)
  28. ********************************
  29. CORRUPT FILES: 8
  30. MISSING BLOCKS: 8
  31. MISSING SIZE: 904331 B
  32. CORRUPT BLOCKS: 8
  33. ********************************
  34. Minimally replicated blocks: 2 (20.0 %)
  35. Over-replicated blocks: 0 (0.0 %)
  36. Under-replicated blocks: 2 (20.0 %)
  37. Mis-replicated blocks: 0 (0.0 %)
  38. Default replication factor: 2
  39. Average block replication: 0.2
  40. Corrupt blocks: 8
  41. Missing replicas: 2 (100.0 %)
  42. Number of data-nodes: 1
  43. Number of racks: 1
  44. FSCK ended at Wed Mar 07 17:57:43 CST 2012 in 11 milliseconds
  45. The filesystem under path '/' is CORRUPT




上一篇:对于一致性哈希(Consistent Hashing )的学习
下一篇:Hadoop & Twitter storm QQ群 8443695

文章评论