Get all extended Hive tables with location in HDFS

for file in $(hive -e "show table extended like \`*\`" | grep location: | awk 'BEGIN { FS = ":" };{printf("hdfs:%s:%s\n",$3,$4)}'); do hdfs dfs -du -h $file; done;
Output:

Time taken: 2.494 seconds
12.6m  hdfs://hadoop1:8020/hive/tpcds/customer/customer.dat
5.2m  hdfs://hadoop1:8020/hive/tpcds/customer_address/customer_address.dat
76.9m  hdfs://hadoop1:8020/hive/tpcds/customer_demographics/customer_demographics.dat
9.8m  hdfs://hadoop1:8020/hive/tpcds/date_dim/date_dim.dat
148.1k  hdfs://hadoop1:8020/hive/tpcds/household_demographics/household_demographics.dat
4.8m  hdfs://hadoop1:8020/hive/tpcds/item/item.dat
36.4k  hdfs://hadoop1:8020/hive/tpcds/promotion/promotion.dat
3.1k  hdfs://hadoop1:8020/hive/tpcds/store/store.dat
370.5m  hdfs://hadoop1:8020/hive/tpcds/store_sales/store_sales.dat
4.9m  hdfs://hadoop1:8020/hive/tpcds/time_dim/time_dim.dat
0      hdfs://hadoop1:8020/user/alexander/transactions/_SUCCESS
95.0k  hdfs://hadoop1:8020/user/alexander/transactions/_logs
3.1m   hdfs://hadoop1:8020/user/alexander/transactions/part-m-00000
3.1m   hdfs://hadoop1:8020/user/alexander/transactions/part-m-00001
3.1m   hdfs://hadoop1:8020/user/alexander/transactions/part-m-00002
3.1m   hdfs://hadoop1:8020/user/alexander/transactions/part-m-00003
1.9m  hdfs://hadoop1:8020/user/hive/warehouse/zipcode_incomes_plain/DEC_00_SF3_P077_with_ann_noheader.csv

Comments

Popular posts from this blog

Deal with corrupted messages in Apache Kafka

Hive query shows ERROR "too many counters"

Embedded Linux won't reboot - how to fix and repair