启动spark-sql
spark-sql --master yarn-client --executor-memory 1G
报错：
16/03/30 14:45:32 WARN BlockReaderLocal: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
使用命令查看databases
spark-sql> show databases;
16/03/30 14:46:01 ERROR GPLNativeCodeLoader: Could not load native gpl library
java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1088)
at com.hadoop.compression.lzo.GPLNativeCodeLoader.(GPLNativeCodeLoader.java:32)
at com.hadoop.compression.lzo.LzoCodec.(LzoCodec.java:71)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1834)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1799)
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
at org.apache.hadoop.io.compress.CompressionCodecFactory.(CompressionCodecFactory.java:175)
at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16/03/30 14:46:01 ERROR LzoCodec: Cannot load native-lzo without native-hadoop
解决方法：
spark-env.sh中增加
export SPARK_SUBMIT_LIBRARY_PATH=/usr/lib/hadoop/lib/native/:$SPARK_SUBMIT_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/hadoop/lib/native/
export SPARK_LIBRARY_PATH=/usr/lib/hadoop/lib/native/:$SPARK_LIBRARY_PATH
export SPARK_CLASSPATH=/usr/local/spark/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar:$CLASSPATH
后来尝试了下，保留以下两个即可：
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/hadoop/lib/native/
export SPARK_CLASSPATH=/usr/local/spark/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar:$CLASSPATH
参考：
http://find.searchhub.org/document/a128707a98fe4ec6
http://lotso.blog.51cto.com/3681673/1441737
http://guoyunsky.iteye.com/blog/1237327
