Apache Hive

The Apache Hive ? is a distributed, fault-tolerant data warehouse system that enables analytics at a massive scale and facilitates reading, writing, and managing petabytes of data residing in distributed storage using SQL.

What is Hive?

Apache Hive is a distributed, fault-tolerant data warehouse system that enables analytics at a massive scale. Hive Metastore(HMS) provides a central repository of metadata that can easily be analyzed to make informed, data driven decisions, and therefore it is a critical component of many data lake architectures. Hive is built on top of Apache Hadoop and supports storage on S3, adls, gs etc though hdfs. Hive allows users to read, write, and manage petabytes of data using SQL.

Key Features

beeline -u "jdbc:hive2://host:10001/default" Connected to: Apache Hive jdbc:hive2://host:10001/>select count(*) from test_t1;

HiveServer2 (HS2)

HS2 supports multi-client concurrency and authentication. It is designed to provide better support for open API clients like JDBC and ODBC.

Hive Metastore Server (HMS)

The Hive Metastore (HMS) is a central repository of metadata for Hive tables and partitions in a relational database, and provides clients (including Hive, Impala and Spark) access to this information using the metastore service API. It has become a building block for data lakes that utilize the diverse world of open-source software, such as Apache Spark and Presto. In fact, a whole ecosystem of tools, open-source and otherwise, are built around the Hive Metastore, some of which this diagram illustrates.

Hive ACID

Hive provides full ACID support for ORC tables and insert only support to all other formats.

Hive Data Compaction

Query-based and MR-based data compactions are supported out-of-the-box.

jdbc:hive2://> alter table test_t1 compact "MAJOR"; Done! jdbc:hive2://> alter table test_t1 compact "MINOR"; Done! jdbc:hive2://> show compactions;

Hive Iceberg

Hive provides out of the box support for Apache Iceberg Tables, a cloud-native, high-performance open table format, via Hive StorageHandler.

Security and Observability

Apache Hive supports kerberos auth and integrates with Apache Ranger and Apache Atlas for security and observability.

Hive LLAP

Apache Hive enables interactive and subsecond SQL through Low Latency Analytical Processing (LLAP), introduced in Hive 2.0 that makes Hive faster by using persistent query infrastructure and optimized data caching

Query planner and Cost based Optimizer

Hive uses Apache Calcite's cost based query optimizer (CBO) and query execution framework to optimize sql queries.

jdbc:hive2://> explain cbo select ss.ss_net_profit, sr.sr_net_loss from store_sales ss join store_returns sr on (ss.ss_item_sk=sr.sr_item_sk) limit 5 ; +---------------------------------------------+ Explain +---------------------------------------------+ CBO PLAN: HiveSortLimit(fetch=[5]) HiveProject(ss_net_profit=[$1], sr_net_loss=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner]) HiveProject(ss_item_sk=[$2], ss_net_profit=[$22]) HiveFilter(condition=[IS NOT NULL($2)]) HiveTableScan(table=[[tpcds_text_10, store_sales]], table:alias=[ss]) HiveProject(sr_item_sk=[$2], sr_net_loss=[$19]) HiveFilter(condition=[IS NOT NULL($2)]) HiveTableScan(table=[[tpcds_text_10, store_returns]], table:alias=[sr]) +---------------------------------------------+

jdbc:hive2://> repl dump src with ( . . .> 'hive.repl.dump.version'= '2', . . .> 'hive.repl.rootdir'= 'hdfs://<host>:<port>/user/replDir/d1' . . .> ); Done! jdbc:hive2://> repl load src into tgt with ( . . .> 'hive.repl.rootdir'= 'hdfs://<host>:<port>/user/replDir/d1' . . .> ); Done!

Hive Replication

Hive supports bootstap and incremental replication for backup and recovery.

动车与高铁有什么区别	左卵巢囊性结构是什么意思	左侧卵巢内无回声是什么意思	孕妇吃什么鱼对胎儿好	空调外机很响是什么原因
为什么有些	乏力没精神容易疲劳是什么原因	袁崇焕为什么杀毛文龙	世界上最大的昆虫是什么	24号来月经什么时候是排卵期
烦躁是什么意思	核磁共振什么时候出结果	18k金是什么材质	尿道炎症状吃什么药	10月生日是什么星座
什么的山谷	白细胞偏高有什么危害	胃肠道感冒吃什么药	宝批龙什么意思	凶神宜忌是什么意思

肩胛骨发麻是什么原因hcv9jop3ns5r.cn	杀鸡吓什么hanqikai.com	十二指肠霜斑样溃疡是什么意思hcv9jop0ns1r.cn	女生排卵期是什么意思hcv8jop3ns3r.cn	食少便溏是什么意思hcv8jop4ns5r.cn
女人缺铁性贫血吃什么好hcv9jop4ns9r.cn	蚂蟥是什么xinjiangjialails.com	肥波是什么品种的猫hcv9jop1ns9r.cn	打喷嚏是什么意思hcv7jop4ns5r.cn	大姨妈来吃什么好hcv7jop5ns2r.cn
整个后背疼是什么原因hcv9jop3ns0r.cn	便秘是什么症状hcv9jop7ns4r.cn	烧心吃什么食物好得快hcv7jop6ns5r.cn	皇帝的新装是什么意思hcv7jop6ns0r.cn	各就各位是什么意思hcv9jop0ns0r.cn
画画用什么铅笔hcv8jop7ns6r.cn	肝内小钙化灶是什么意思hcv8jop8ns2r.cn	西南方向是什么方位hcv8jop8ns0r.cn	学字五行属什么bjhyzcsm.com	碘化银什么颜色hcv9jop1ns0r.cn