Project: StarRocks (App Definition and Development / Database)
File: TPC_DS_Benchmark.md

displayed_sidebar: "English" TPC-DS is a decision support benchmark developed by the Transaction Processing Performance Council (TPC). It uses more comprehensive test datasets and complex SQL queries than TPC-H. TPC-DS models several generally applicable aspects of a decision support system, including queries and data maintenance. TPC-DS aims to provide a comprehensive and realistic workload for testing and evaluating the performance of database systems in a retail environment. The TPC-DS benchmark simulates the sales and return data of three sales channels (stores, Internet, and catalog) in a retail enterprise. In addition to creating tables for sales and return data models, it also includes a simple inventory system and a promotion system. This benchmark tests a total of 99 complex SQL queries against 24 tables whose data size ranges from 1 GB to 3 GB. The main performance metric is the response time of each query, which is the duration between the time a query is submitted to the time the result is returned. We perform a test on 99 queries against a TPC-DS 100 GB dataset. The following figure shows the test result. In the test, StarRocks queries data from both its native storage and Hive external tables. StarRocks and Trino query the same copy of data from Hive external tables. Data is LZ4-compressed and stored in the Parquet format. The latency for StarRocks to query data from its native storage is 174s, that for StarRocks to query Hive external tables is 239s, that for StarRocks to query Hive external tables with the Data Cache feature enabled is 176s, and that for Trino to query Hive external tables is 892s. | Machine | 4 cloud hosts | | -- | - | | CPU | 8core Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz | | Memory | 32 GB | | Network bandwidth | 5 Gbit/s | | Disk | ESSD | StarRocks and Trino are deployed on machines with the same configuration. StarRocks has 1 FE and 3 BEs deployed. Trino has 1 Coordinator and 3 Workers deployed. Kernel version: Linux 3.10.0-1127.13.1.el7.x86_64 OS version: CentOS Linux released 7.8.2003 Software version: StarRocks Community Edition 3.1, Trino-419, Hive-3.1.2 The StarRocks FE can be separately deployed or hybrid deployed with BEs, which does not affect the test results. | Table | Records | | - | | | call_center | 30 | | catalog_page | 20400 | | catalog_returns | 14404374 | | catalog_sales | 143997065 | | customer_address | 1000000 | | customer_demographics | 1920800 | | customer | 2000000 | | date_dim | 73049 | | household_demographics | 7200 | | income_band | 20 | | inventory | 399330000 | | item | 204000 | | promotion | 1000 | | reason | 55 | | ship_mode | 20 | | store | 402 | | store_returns | 28795080 | | store_sales | 287997024 | | time_dim | 86400 | | warehouse | 15 | | web_page | 2040 | | web_returns | 7197670 | | web_sales | 72001237 | | web_site | 24 | NOTE The unit of query latency in the following table is ms. `StarRocks-3.0.5-native` indicates StarRocks native storage, `StarRocks-3.0-Hive external` indicates StarRocks queries Hive external tables through Hive Catalog, `StarRocks-3.0-Hive external-Cache` indicates StarRocks queries Hive external tables through Hive Catalog with Data Cache

Aggregate pushdown is enabled for StarRocks (`SET global cbopushdownaggregatemode = 0`). | Query | StarRocks-3.0.5-native | StarRocks-3.0-Hive external |

external-Cache | Trino-419 | | | -- | - | - | - | | SUM | 174157 | 238590 | 175970 | 891892 | | Q1 | 274 | 780 | 254 | 1681 | | Q2 | 338 | 676 | 397 | 10200 | | Q3 | 455 | 1156 | 628 | 3156 | | Q4 | 16180 | 13229 | 12623 | 48176 | | Q5 | 1162 | 773 | 506 | 4490 | | Q6 | 397 | 606 | 165 | 1349 | | Q7 | 898 | 1707 | 724 | 2300 | | Q8 | 532 | 447 | 141 | 2330 | | Q9 | 2113 | 7998 | 6336 | 17734 | | Q10 | 588 | 847 | 285 | 2498 | | Q11 | 6465 | 5086 | 4665 | 31333 | | Q12 | 149 | 302 | 135 | 728 | | Q13 | 1573 | 2661 | 1349 | 4370 | | Q14 | 7928 | 7811 | 5955 | 69729 | | Q15 | 323 | 461 | 199 | 1522 | | Q16 | 639 | 1278 | 661 | 3282 | | Q17 | 1157 | 898 | 682 | 4102 | | Q18 | 540 | 1746 | 521 | 2471 | | Q19 | 667 | 639 | 230 | 1701 | | Q20 | 209 | 369 | 144 | 849 | | Q21 | 466 | 586 | 306 | 1591 | | Q22 | 3876 | 4704 | 4536 | 17422 | | Q23 | 24500 | 24746 | 21707 | 145850 | | Q24 | 1256 | 5220 | 3219 | 21234 | | Q25 | 1037 | 792 | 542 | 3702 | | Q26 | 393 | 834 | 360 | 1737 | | Q27 | 742 | 1303 | 696 | 2396 | | Q28 | 1864 | 8600 | 6564 | 15837 | | Q29 | 1097 | 1134 | 888 | 4024 | | Q30 | 194 | 669 | 242 | 1922 | | Q31 | 1149 | 1070 | 834 | 5431 | | Q32 | 222 | 718 | 104 | 1706 | | Q33 | 922 | 735 | 327 | 2048 | | Q34 | 544 | 1392 | 576 | 3185 | | Q35 | 974 | 897 | 574 | 3050 | | Q36 | 630 | 1009 | 464 | 3056 | | Q37 | 246 | 791 | 273 | 3258 | | Q38 | 2831 | 2017 | 1695 | 10913 | | Q39 | 1057 | 2312 | 1324 | 10665 | | Q40 | 331 | 560 | 209 | 2678 | | Q41 | 57 | 148 | 79 | 776 | | Q42 | 463 | 559 | 106 | 1213 | | Q43 | 885 | 602 | 342 | 2914 | | Q44 | 506 | 3783 | 2306 | 9705 | | Q45 | 439 | 777 | 309 | 1012 | | Q46 | 868 | 1746 | 1037 | 4766 | | Q47 | 1816 | 2979 | 2684 | 19111 | | Q48 | 635 | 2038 | 1202 | 3635 | | Q49 | 1440 | 2754 | 1168 | 3435 | | Q50 | 836 | 2053 | 1305 | 4375 | | Q51 | 3966 | 5258 | 4466 | 14283 | | Q52 | 483 | 436 | 100 | 1126 | | Q53 | 698 | 802 | 391 | 1648 | | Q54 | 794 | 970 | 534 | 5146 | | Q55 | 463 | 540 | 97 | 963 | | Q56 | 874 | 695 | 240 | 2110 | | Q57 | 1717 | 2723 | 2372 | 10203 | | Q58 | 554 | 727 | 242 | 2053 | | Q59 | 2764 | 1581 | 1368 | 15697 | | Q60 | 1053 | 557 | 389 | 2421 | | Q61 | 1353 | 1026 | 439 | 2334 | | Q62 | 453 | 659 | 427 | 2422 | | Q63 | 709 | 943 | 374 | 1624 | | Q64 | 3209 | 6968 | 6175 | 31994 | | Q65 | 2147 | 3043 | 2451 | 9334 | | Q66 | 688 | 805 | 437 | 2598 | | Q67 | 15486 | 23743 | 21975 | 58091 | | Q68 | 965 | 1702 | 776 | 2710 | | Q69 | 600 | 703 | 263 | 2872 | | Q70 | 2376 | 2217 | 1588 | 10272 | | Q71 | 702 | 691 | 348 | 3074 | | Q72 | 1764 | 2733 | 2305 | 13973 | | Q73 | 576 | 1145 | 484 | 1899 | | Q74 | 4615 | 3884 | 3776 | 18749 | | Q75 | 2661 | 3479 | 3137 | 10858 | | Q76 | 450 | 2001 | 1014 | 5297 | | Q77 | 1109 | 743 | 317 | 2810 | | Q78 | 6540 | 7198 | 5890 | 19671 | | Q79 | 1116 | 1953 | 1121 | 4406 | | Q80 | 2290 | 1973 | 1480 | 5865 | | Q81 | 247 | 1024 | 317 | 1729 | | Q82 | 392 | 929 | 407 | 3605 | | Q83 | 134 | 313 | 158 | 1209 | | Q84 | 107 | 820 | 228 | 2448 | | Q85 | 460 | 2045 | 621 | 4311 | | Q86 | 433 | 999 | 387 | 1693 | | Q87 | 2873 | 2159 | 1779 | 10709 | | Q88 | 3616 | 7076 | 5432 | 26002 | | Q89 | 735 | 785 | 454 | 1997 | | Q90 | 174 | 898 | 232 | 2585 | | Q91 | 113 | 495 | 139 | 1745 | | Q92 | 203 | 627 | 91 | 1016 | | Q93 | 529 | 2508 | 1422 | 12265 | | Q94 | 475 | 811 | 598 | 2153 | | Q95 | 1059 | 1993 | 1526 | 8058 | | Q96 | 395 | 1197 | 681 | 3976 | | Q97 | 3000 | 3459 | 2860 | 6818 | | Q98 | 419 | 486 | 344 | 2090 | | Q99 | 755 | 1070 | 740 | 4332 | Download the tpcds-poc toolkit and generate the standard TPC-DS test dataset `scale factor=100`. ```Bash wget https://starrocks-public.oss-cn-zhangjiakou.aliyuncs.com/tpcds-poc-1.0.zip unzip tpcds-poc-1.0 cd tpcds-poc-1.0 sh bin/gendata/gen-tpcds.sh 100 data100 ``` Modify the configuration file `conf/starrocks.conf` and specify the cluster address. Pay attention to `mysqlhost` and `mysqlport`. ```Bash sh bin/createdbtable.sh ddl_100 ``` ```Bash sh bin/streamload.sh data100 ``` ```Bash sh bin/benchmark.sh -p -d tpcds ``` Create external tables in Hive, whose storage format is Parquet and compression format is LZ4. For detailed CREATE TABLE statements, see . StarRocks and Trino query data from these external tables. Load the CSV data generated in to the specified HDFS path of Hive. This example uses `/user/tmp/csv/` as the HDFS path. Create CSV Hive external tables in Hive and store the tables in `/user/tmp/csv/`. For detailed CREATE TABLE statements, see . Use INSERT INTO to load data from CSV external tables to Parquet external tables. This generates LZ4-compressed Parquet data. ```SQL use tpcds100gparquet_lz4; insert into callcenter select * from tpcds100gcsv.callcenter order by cccallcenter_sk; insert into catalogpage select * from tpcds100gcsv.catalogpage order by cpcatalogpage_sk; insert into catalogreturns select * from tpcds100gcsv.catalogreturns order by crreturneddatesk, critem_sk; insert into datedim select * from tpcds100gcsv.datedim order by DDATESK; insert into householddemographics select * from tpcds100gcsv.householddemographics order by HDDEMOSK; insert into incomeband select * from tpcds100gcsv.incomeband order by IBINCOMEBAND_SK; insert into item select * from tpcds100gcsv.item order by IITEMSK; insert into promotion select * from tpcds100gcsv.promotion order by PPROMOSK; insert into reason select * from tpcds100gcsv.reason a order by a.RREASONSK; insert into shipmode select * from tpcds100gcsv.shipmode order by SMSHIPMODE_SK; insert into store select * from tpcds100gcsv.store order by SSTORESK; insert into timedim select * from tpcds100gcsv.timedim order by TTIMESK; insert into warehouse select * from tpcds100gcsv.warehouse order by WWAREHOUSESK; insert into webpage select * from tpcds100gcsv.webpage order by WPWEBPAGE_SK; insert into website select * from tpcds100gcsv.website order by WEBSITESK; insert into customerdemographics select * from tpcds100gcsv.customerdemographics order by CDDEMOSK; insert into customer select * from tpcds100gcsv.customer order by CCUSTOMERSK; insert into customeraddress select * from tpcds100gcsv.customeraddress order by CAADDRESSSK; insert into websales select * from tpcds100gcsv.websales order by WSSOLDDATESK, WSITEM_SK; insert into webreturns select * from tpcds100gcsv.webreturns order by WRRETURNEDDATESK, WRITEM_SK; insert into inventory select * from tpcds100gcsv.inventory order by INVDATESK, INVITEMSK; insert into catalogsales select * from tpcds100gcsv.catalogsales order by CSSOLDDATESK, CSITEM_SK; insert into storereturns select * from tpcds100gcsv.storereturns order by SRRETURNEDDATESK, SRITEM_SK; insert into storesales select * from tpcds100gcsv.storesales order by SSSOLDDATESK, SSITEM_SK; ``` StarRocks v3.0 does not support collecting statistics from external tables. Therefore, we use Hive to collect column-level

```SQL use tpcds100gparquet_lz4; analyze table call_center compute statistics FOR COLUMNS; analyze table catalog_page compute statistics FOR COLUMNS; analyze table catalog_returns compute statistics FOR COLUMNS; analyze table catalog_sales compute statistics FOR COLUMNS; analyze table customer compute statistics FOR COLUMNS; analyze table customer_address compute statistics FOR COLUMNS; analyze table customer_demographics compute statistics FOR COLUMNS; analyze table date_dim compute statistics FOR COLUMNS; analyze table household_demographics compute statistics FOR COLUMNS; analyze table income_band compute statistics FOR COLUMNS; analyze table inventory compute statistics FOR COLUMNS; analyze table item compute statistics FOR COLUMNS; analyze table promotion compute statistics FOR COLUMNS; analyze table reason compute statistics FOR COLUMNS; analyze table ship_mode compute statistics FOR COLUMNS; analyze table store compute statistics FOR COLUMNS; analyze table store_returns compute statistics FOR COLUMNS; analyze table store_sales compute statistics FOR COLUMNS; analyze table time_dim compute statistics FOR COLUMNS; analyze table warehouse compute statistics FOR COLUMNS; analyze table web_page compute statistics FOR COLUMNS; analyze table web_returns compute statistics FOR COLUMNS; analyze table web_sales compute statistics FOR COLUMNS; analyze table web_site compute statistics FOR COLUMNS; ``` StarRocks uses to query Hive external tables. If is enabled for StarRocks during the test, the following configuration is recommended for the Data Cache feature. ```SQL blockcachemem_size = 5368709120 blockcachedisk_size = 193273528320 ``` For more information about the 99 SQL queries, see

```SQL create table call_center ( cccallcenter_sk integer not null, cccallcenter_id char(16) not null, ccrecstart_date date , ccrecend_date date , cccloseddate_sk integer , ccopendate_sk integer , cc_name varchar(50) , cc_class varchar(50) , cc_employees integer , ccsqft integer , cc_hours char(20) , cc_manager varchar(40) , ccmktid integer , ccmktclass char(50) , ccmktdesc varchar(100) , ccmarketmanager varchar(40) , cc_division integer , ccdivisionname varchar(50) , cc_company integer , cccompanyname char(50) , ccstreetnumber char(10) , ccstreetname varchar(60) , ccstreettype char(15) , ccsuitenumber char(10) , cc_city varchar(60) , cc_county varchar(30) , cc_state char(2) , cc_zip char(10) , cc_country varchar(20) , ccgmtoffset decimal(5,2) , cctaxpercentage decimal(5,2) ) duplicate key (cccallcenter_sk) distributed by hash(cccallcenter_sk) buckets 1 properties( "replication_num" = "1" ); create table catalog_page ( cpcatalogpage_sk integer not null, cpcatalogpage_id char(16) not null, cpstartdate_sk integer , cpenddate_sk integer , cp_department varchar(50) , cpcatalognumber integer , cpcatalogpage_number integer , cp_description varchar(100) , cp_type varchar(100) ) duplicate key (cpcatalogpage_sk) distributed by hash(cpcatalogpage_sk) buckets 1 properties( "replication_num" = "1" ); create table catalog_returns ( critemsk integer not null, crordernumber integer not null, crreturneddate_sk integer , crreturnedtime_sk integer , crrefundedcustomer_sk integer , crrefundedcdemo_sk integer , crrefundedhdemo_sk integer , crrefundedaddr_sk integer , crreturningcustomer_sk integer , crreturningcdemo_sk integer , crreturninghdemo_sk integer , crreturningaddr_sk integer , crcallcenter_sk integer , crcatalogpage_sk integer , crshipmode_sk integer , crwarehousesk integer , crreasonsk integer , crreturnquantity integer , crreturnamount decimal(7,2) , crreturntax decimal(7,2) , crreturnamtinctax decimal(7,2) , cr_fee decimal(7,2) , crreturnship_cost decimal(7,2) , crrefundedcash decimal(7,2) , crreversedcharge decimal(7,2) , crstorecredit decimal(7,2) , crnetloss decimal(7,2) ) duplicate key (critemsk, crordernumber) distributed by hash(critemsk, crordernumber) buckets 16 properties( "replication_num" = "1" ); create table catalog_sales ( csitemsk integer not null, csordernumber integer not null, cssolddate_sk integer , cssoldtime_sk integer , csshipdate_sk integer , csbillcustomer_sk integer , csbillcdemo_sk integer , csbillhdemo_sk integer , csbilladdr_sk integer , csshipcustomer_sk integer , csshipcdemo_sk integer , csshiphdemo_sk integer , csshipaddr_sk integer , cscallcenter_sk integer , cscatalogpage_sk integer , csshipmode_sk integer , cswarehousesk integer , cspromosk integer , cs_quantity integer , cswholesalecost decimal(7,2) , cslistprice decimal(7,2) , cssalesprice decimal(7,2) , csextdiscount_amt decimal(7,2) , csextsales_price decimal(7,2) , csextwholesale_cost decimal(7,2) , csextlist_price decimal(7,2) , csexttax decimal(7,2) , cscouponamt decimal(7,2) , csextship_cost decimal(7,2) , csnetpaid decimal(7,2) , csnetpaidinctax decimal(7,2) , csnetpaidincship decimal(7,2) , csnetpaidincship_tax decimal(7,2) , csnetprofit decimal(7,2) ) duplicate key (csitemsk, csordernumber) distributed by hash(csitemsk) buckets 192 properties( "replication_num" = "1" ); create table customer_address ( caaddresssk integer not null, caaddressid char(16) not null, castreetnumber char(10) , castreetname varchar(60) , castreettype char(15) , casuitenumber char(10) , ca_city varchar(60) , ca_county varchar(30) , ca_state char(2) , ca_zip char(10) , ca_country varchar(20) , cagmtoffset decimal(5,2) , calocationtype char(20) ) duplicate key(caaddresssk) distributed by hash(caaddresssk) buckets 10 properties( "replication_num" = "1" ); create table customer_demographics ( cddemosk integer not null, cd_gender char(1) not null, cdmaritalstatus char(1) not null, cdeducationstatus char(20) not null, cdpurchaseestimate integer not null, cdcreditrating char(10) not null, cddepcount integer not null, cddepemployed_count integer not null, cddepcollege_count integer not null ) duplicate key (cddemosk) distributed by hash(cddemosk) buckets 10 properties( "replication_num" = "1" ); create table customer ( ccustomersk integer not null, ccustomerid char(16) not null, ccurrentcdemo_sk integer , ccurrenthdemo_sk integer , ccurrentaddr_sk integer , cfirstshiptodatesk integer , cfirstsalesdatesk integer , c_salutation char(10) , cfirstname char(20) , clastname char(30) , cpreferredcust_flag char(1) , cbirthday integer , cbirthmonth integer , cbirthyear integer , cbirthcountry varchar(20) , c_login char(13) , cemailaddress char(50) , clastreview_date char(10) ) duplicate key (ccustomersk) distributed by hash(ccustomersk) buckets 10 properties( "replication_num" = "1" ); create table date_dim ( ddatesk integer not null, ddateid char(16) not null, d_date date not null, dmonthseq integer not null, dweekseq integer not null, dquarterseq integer not null, d_year integer not null, d_dow integer not null, d_moy integer not null, d_dom integer not null, d_qoy integer not null, dfyyear integer not null, dfyquarter_seq integer not null, dfyweek_seq integer not null, ddayname char(9) not null, dquartername char(6) not null, d_holiday char(1) not null, d_weekend char(1) not null, dfollowingholiday char(1) not null, dfirstdom integer not null, dlastdom integer not null, dsameday_ly integer not null, dsameday_lq integer not null, dcurrentday char(1) not null, dcurrentweek char(1) not null, dcurrentmonth char(1) not null, dcurrentquarter char(1) not null, dcurrentyear char(1) not null ) duplicate key (ddatesk) distributed by hash(ddatesk) buckets 5 properties( "replication_num" = "1" ); create table household_demographics ( hddemosk integer not null, hdincomeband_sk integer not null, hdbuypotential char(15) not null, hddepcount integer not null, hdvehiclecount integer not null ) duplicate key (hddemosk) distributed by hash(hddemosk) buckets 1 properties( "replication_num" = "1" ); create table income_band ( ibincomeband_sk integer not null, iblowerbound integer , ibupperbound integer ) duplicate key (ibincomeband_sk) distributed by hash(ibincomeband_sk) buckets 1 properties( "replication_num" = "1" ); create table inventory ( invitemsk integer not null, invdatesk integer not null, invwarehousesk integer not null, invquantityon_hand integer ) duplicate key (invitemsk, invdatesk, invwarehousesk) distributed by hash(invitemsk) buckets 32 properties( "replication_num" = "1" ); create table item ( iitemsk integer not null, iitemid char(16) not null, irecstart_date date , irecend_date date , iitemdesc varchar(200) , icurrentprice decimal(7,2) , iwholesalecost decimal(7,2) , ibrandid integer , i_brand char(50) , iclassid integer , i_class char(50) , icategoryid integer , i_category char(50) , imanufactid integer , i_manufact char(50) , i_size char(20) , i_formulation char(20) , i_color char(20) , i_units char(10) , i_container char(10) , imanagerid integer , iproductname char(50) ) duplicate key (iitemsk) distributed by hash(iitemsk) buckets 10 properties( "replication_num" = "1" ); create table promotion ( ppromosk integer not null, ppromoid char(16) not null, pstartdate_sk integer , penddate_sk integer , pitemsk integer , p_cost decimal(15,2) , presponsetarget integer , ppromoname char(50) , pchanneldmail char(1) , pchannelemail char(1) , pchannelcatalog char(1) , pchanneltv char(1) , pchannelradio char(1) , pchannelpress char(1) , pchannelevent char(1) , pchanneldemo char(1) , pchanneldetails varchar(100) , p_purpose char(15) , pdiscountactive char(1) ) duplicate key (ppromosk) distributed by hash(ppromosk) buckets 1 properties( "replication_num" = "1" ); create table reason ( rreasonsk integer not null, rreasonid char(16) not null, rreasondesc char(100) ) duplicate key (rreasonsk) distributed by hash(rreasonsk) buckets 1 properties( "replication_num" = "1" ); create table ship_mode ( smshipmode_sk integer not null, smshipmode_id char(16) not null, sm_type char(30) , sm_code char(10) , sm_carrier char(20) , sm_contract char(20) ) duplicate key (smshipmode_sk) distributed by hash(smshipmode_sk) buckets 1 properties( "replication_num" = "1" ); create table store_returns ( sritemsk integer not null, srticketnumber integer not null, srreturneddate_sk integer , srreturntime_sk integer , srcustomersk integer , srcdemosk integer , srhdemosk integer , sraddrsk integer , srstoresk integer , srreasonsk integer , srreturnquantity integer , srreturnamt decimal(7,2) , srreturntax decimal(7,2) , srreturnamtinctax decimal(7,2) , sr_fee decimal(7,2) , srreturnship_cost decimal(7,2) , srrefundedcash decimal(7,2) , srreversedcharge decimal(7,2) , srstorecredit decimal(7,2) , srnetloss decimal(7,2) ) duplicate key (sritemsk, srticketnumber) distributed by hash(sritemsk, srticketnumber) buckets 10 properties( "replication_num" = "1" ); create table store_sales ( ssitemsk integer not null, ssticketnumber integer not null, sssolddate_sk integer , sssoldtime_sk integer , sscustomersk integer , sscdemosk integer , sshdemosk integer , ssaddrsk integer , ssstoresk integer , sspromosk integer , ss_quantity integer , sswholesalecost decimal(7,2) , sslistprice decimal(7,2) , sssalesprice decimal(7,2) , ssextdiscount_amt decimal(7,2) , ssextsales_price decimal(7,2) , ssextwholesale_cost decimal(7,2) , ssextlist_price decimal(7,2) , ssexttax decimal(7,2) , sscouponamt decimal(7,2) , ssnetpaid decimal(7,2) , ssnetpaidinctax decimal(7,2) , ssnetprofit decimal(7,2) ) duplicate key (ssitemsk, ssticketnumber) distributed by hash(ssitemsk) buckets 192 properties( "replication_num" = "1" ); create table store ( sstoresk integer not null, sstoreid char(16) not null, srecstart_date date , srecend_date date , scloseddate_sk integer , sstorename varchar(50) , snumberemployees integer , sfloorspace integer , s_hours char(20) , s_manager varchar(40) , smarketid integer , sgeographyclass varchar(100) , smarketdesc varchar(100) , smarketmanager varchar(40) , sdivisionid integer , sdivisionname varchar(50) , scompanyid integer , scompanyname varchar(50) , sstreetnumber varchar(10) , sstreetname varchar(60) , sstreettype char(15) , ssuitenumber char(10) , s_city varchar(60) , s_county varchar(30) , s_state char(2) , s_zip char(10) , s_country varchar(20) , sgmtoffset decimal(5,2) , staxprecentage decimal(5,2) ) duplicate key (sstoresk) distributed by hash(sstoresk) buckets 1 properties( "replication_num" = "1" ); create table time_dim ( ttimesk integer not null, ttimeid char(16) not null, t_time integer not null, t_hour integer not null, t_minute integer not null, t_second integer not null, tampm char(2) not null, t_shift char(20) not null, tsubshift char(20) not null, tmealtime char(20) ) duplicate key (ttimesk) distributed by hash(ttimesk) buckets 5 properties( "replication_num" = "1" ); create table warehouse ( wwarehousesk integer not null, wwarehouseid char(16) not null, wwarehousename varchar(20) , wwarehousesq_ft integer , wstreetnumber char(10) , wstreetname varchar(60) , wstreettype char(15) , wsuitenumber char(10) , w_city varchar(60) , w_county varchar(30) , w_state char(2) , w_zip char(10) , w_country varchar(20) , wgmtoffset decimal(5,2) ) duplicate key (wwarehousesk) distributed by hash(wwarehousesk) buckets 1 properties( "replication_num" = "1" ); create table web_page ( wpwebpage_sk integer not null, wpwebpage_id char(16) not null, wprecstart_date date , wprecend_date date , wpcreationdate_sk integer , wpaccessdate_sk integer , wpautogenflag char(1) , wpcustomersk integer , wp_url varchar(100) , wp_type char(50) , wpcharcount integer , wplinkcount integer , wpimagecount integer , wpmaxad_count integer ) duplicate key (wpwebpage_sk) distributed by hash(wpwebpage_sk) buckets 1 properties( "replication_num" = "1" ); create table web_returns ( writemsk integer not null, wrordernumber integer not null, wrreturneddate_sk integer , wrreturnedtime_sk integer , wrrefundedcustomer_sk integer , wrrefundedcdemo_sk integer , wrrefundedhdemo_sk integer , wrrefundedaddr_sk integer , wrreturningcustomer_sk integer , wrreturningcdemo_sk integer , wrreturninghdemo_sk integer , wrreturningaddr_sk integer , wrwebpage_sk integer , wrreasonsk integer , wrreturnquantity integer , wrreturnamt decimal(7,2) , wrreturntax decimal(7,2) , wrreturnamtinctax decimal(7,2) , wr_fee decimal(7,2) , wrreturnship_cost decimal(7,2) , wrrefundedcash decimal(7,2) , wrreversedcharge decimal(7,2) , wraccountcredit decimal(7,2) , wrnetloss decimal(7,2) ) duplicate key (writemsk, wrordernumber) distributed by hash(writemsk, wrordernumber) buckets 10 properties( "replication_num" = "1" ); create table web_sales ( wsitemsk integer not null, wsordernumber integer not null, wssolddate_sk integer , wssoldtime_sk integer , wsshipdate_sk integer , wsbillcustomer_sk integer , wsbillcdemo_sk integer , wsbillhdemo_sk integer , wsbilladdr_sk integer , wsshipcustomer_sk integer , wsshipcdemo_sk integer , wsshiphdemo_sk integer , wsshipaddr_sk integer , wswebpage_sk integer , wswebsite_sk integer , wsshipmode_sk integer , wswarehousesk integer , wspromosk integer , ws_quantity integer , wswholesalecost decimal(7,2) , wslistprice decimal(7,2) , wssalesprice decimal(7,2) , wsextdiscount_amt decimal(7,2) , wsextsales_price decimal(7,2) , wsextwholesale_cost decimal(7,2) , wsextlist_price decimal(7,2) , wsexttax decimal(7,2) , wscouponamt decimal(7,2) , wsextship_cost decimal(7,2) , wsnetpaid decimal(7,2) , wsnetpaidinctax decimal(7,2) , wsnetpaidincship decimal(7,2) , wsnetpaidincship_tax decimal(7,2) , wsnetprofit decimal(7,2) ) duplicate key (wsitemsk, wsordernumber) distributed by hash(wsitemsk, wsordernumber) buckets 192 properties( "replication_num" = "1" ); create table web_site ( websitesk integer not null, websiteid char(16) not null, webrecstart_date date , webrecend_date date , web_name varchar(50) , webopendate_sk integer , webclosedate_sk integer , web_class varchar(50) , web_manager varchar(40) , webmktid integer , webmktclass varchar(50) , webmktdesc varchar(100) , webmarketmanager varchar(40) , webcompanyid integer , webcompanyname char(50) , webstreetnumber char(10) , webstreetname varchar(60) , webstreettype char(15) , websuitenumber char(10) , web_city varchar(60) , web_county varchar(30) , web_state char(2) , web_zip char(10) , web_country varchar(20) , webgmtoffset decimal(5,2) , webtaxpercentage decimal(5,2) ) duplicate key (websitesk) distributed by hash(websitesk) buckets 1 properties( "replication_num" = "1" ); ``` ```SQL use tpcds100gparquet_lz4; CREATE EXTERNAL TABLE IF NOT EXISTS customer_address ( caaddresssk int ,caaddressid varchar(16) ,castreetnumber varchar(10) ,castreetname varchar(60) ,castreettype varchar(15) ,casuitenumber varchar(10) ,ca_city varchar(60) ,ca_county varchar(30) ,ca_state varchar(2) ,ca_zip varchar(10) ,ca_country varchar(20) ,cagmtoffset decimal(5,2) ,calocationtype varchar(20) ) stored as PARQUET LOCATION '/user/tmp/parquet/customer_address/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS customer_demographics ( cddemosk int ,cd_gender varchar(1) ,cdmaritalstatus varchar(1) ,cdeducationstatus varchar(20) ,cdpurchaseestimate int ,cdcreditrating varchar(10) ,cddepcount int ,cddepemployed_count int ,cddepcollege_count int ) stored as PARQUET LOCATION '/user/tmp/parquet/customer_demographics/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS date_dim ( ddatesk int ,ddateid varchar(16) ,d_date date ,dmonthseq int ,dweekseq int ,dquarterseq int ,d_year int ,d_dow int ,d_moy int ,d_dom int ,d_qoy int ,dfyyear int ,dfyquarter_seq int ,dfyweek_seq int ,ddayname varchar(9) ,dquartername varchar(6) ,d_holiday varchar(1) ,d_weekend varchar(1) ,dfollowingholiday varchar(1) ,dfirstdom int ,dlastdom int ,dsameday_ly int ,dsameday_lq int ,dcurrentday varchar(1) ,dcurrentweek varchar(1) ,dcurrentmonth varchar(1) ,dcurrentquarter varchar(1) ,dcurrentyear varchar(1) ) stored as PARQUET LOCATION '/user/tmp/parquet/date_dim/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS warehouse ( wwarehousesk int ,wwarehouseid varchar(16) ,wwarehousename varchar(20) ,wwarehousesq_ft int ,wstreetnumber varchar(10) ,wstreetname varchar(60) ,wstreettype varchar(15) ,wsuitenumber varchar(10) ,w_city varchar(60) ,w_county varchar(30) ,w_state varchar(2) ,w_zip varchar(10) ,w_country varchar(20) ,wgmtoffset decimal(5,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/warehouse/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS ship_mode ( smshipmode_sk int ,smshipmode_id varchar(16) ,sm_type varchar(30) ,sm_code varchar(10) ,sm_carrier varchar(20) ,sm_contract varchar(20) ) stored as PARQUET LOCATION '/user/tmp/parquet/ship_mode/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS time_dim ( ttimesk int ,ttimeid varchar(16) ,t_time int ,t_hour int ,t_minute int ,t_second int ,tampm varchar(2) ,t_shift varchar(20) ,tsubshift varchar(20) ,tmealtime varchar(20) ) stored as PARQUET LOCATION '/user/tmp/parquet/time_dim/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS reason ( rreasonsk int ,rreasonid varchar(16) ,rreasondesc varchar(100) ) stored as PARQUET LOCATION '/user/tmp/parquet/reason/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS income_band ( ibincomeband_sk integer , iblowerbound integer , ibupperbound integer ) stored as PARQUET LOCATION '/user/tmp/parquet/income_band/'

CREATE EXTERNAL TABLE IF NOT EXISTS item ( iitemsk int ,iitemid varchar(16) ,irecstart_date date ,irecend_date date ,iitemdesc varchar(200) ,icurrentprice decimal(7,2) ,iwholesalecost decimal(7,2) ,ibrandid int ,i_brand varchar(50) ,iclassid int ,i_class varchar(50) ,icategoryid int ,i_category varchar(50) ,imanufactid int ,i_manufact varchar(50) ,i_size varchar(20) ,i_formulation varchar(20) ,i_color varchar(20) ,i_units varchar(10) ,i_container varchar(10) ,imanagerid int ,iproductname varchar(50) ) stored as PARQUET LOCATION '/user/tmp/parquet/item/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS store ( sstoresk int ,sstoreid varchar(16) ,srecstart_date date ,srecend_date date ,scloseddate_sk int ,sstorename varchar(50) ,snumberemployees int ,sfloorspace int ,s_hours varchar(20) ,s_manager varchar(40) ,smarketid int ,sgeographyclass varchar(100) ,smarketdesc varchar(100) ,smarketmanager varchar(40) ,sdivisionid int ,sdivisionname varchar(50) ,scompanyid int ,scompanyname varchar(50) ,sstreetnumber varchar(10) ,sstreetname varchar(60) ,sstreettype varchar(15) ,ssuitenumber varchar(10) ,s_city varchar(60) ,s_county varchar(30) ,s_state varchar(2) ,s_zip varchar(10) ,s_country varchar(20) ,sgmtoffset decimal(5,2) ,staxprecentage decimal(5,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/store/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS call_center ( cccallcenter_sk int ,cccallcenter_id varchar(16) ,ccrecstart_date date ,ccrecend_date date ,cccloseddate_sk int ,ccopendate_sk int ,cc_name varchar(50) ,cc_class varchar(50) ,cc_employees int ,ccsqft int ,cc_hours varchar(20) ,cc_manager varchar(40) ,ccmktid int ,ccmktclass varchar(50) ,ccmktdesc varchar(100) ,ccmarketmanager varchar(40) ,cc_division int ,ccdivisionname varchar(50) ,cc_company int ,cccompanyname varchar(50) ,ccstreetnumber varchar(10) ,ccstreetname varchar(60) ,ccstreettype varchar(15) ,ccsuitenumber varchar(10) ,cc_city varchar(60) ,cc_county varchar(30) ,cc_state varchar(2) ,cc_zip varchar(10) ,cc_country varchar(20) ,ccgmtoffset decimal(5,2) ,cctaxpercentage decimal(5,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/call_center/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS customer ( ccustomersk int ,ccustomerid varchar(16) ,ccurrentcdemo_sk int ,ccurrenthdemo_sk int ,ccurrentaddr_sk int ,cfirstshiptodatesk int ,cfirstsalesdatesk int ,c_salutation varchar(10) ,cfirstname varchar(20) ,clastname varchar(30) ,cpreferredcust_flag varchar(1) ,cbirthday int ,cbirthmonth int ,cbirthyear int ,cbirthcountry varchar(20) ,c_login varchar(13) ,cemailaddress varchar(50) ,clastreview_date varchar(10) ) stored as PARQUET LOCATION '/user/tmp/parquet/customer/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS web_site ( websitesk int ,websiteid varchar(16) ,webrecstart_date date ,webrecend_date date ,web_name varchar(50) ,webopendate_sk int ,webclosedate_sk int ,web_class varchar(50) ,web_manager varchar(40) ,webmktid int ,webmktclass varchar(50) ,webmktdesc varchar(100) ,webmarketmanager varchar(40) ,webcompanyid int ,webcompanyname varchar(50) ,webstreetnumber varchar(10) ,webstreetname varchar(60) ,webstreettype varchar(15) ,websuitenumber varchar(10) ,web_city varchar(60) ,web_county varchar(30) ,web_state varchar(2) ,web_zip varchar(10) ,web_country varchar(20) ,webgmtoffset decimal(5,2) ,webtaxpercentage decimal(5,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/web_site/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS store_returns ( sritemsk int ,srticketnumber int ,srreturneddate_sk int ,srreturntime_sk int ,srcustomersk int ,srcdemosk int ,srhdemosk int ,sraddrsk int ,srstoresk int ,srreasonsk int ,srreturnquantity int ,srreturnamt decimal(7,2) ,srreturntax decimal(7,2) ,srreturnamtinctax decimal(7,2) ,sr_fee decimal(7,2) ,srreturnship_cost decimal(7,2) ,srrefundedcash decimal(7,2) ,srreversedcharge decimal(7,2) ,srstorecredit decimal(7,2) ,srnetloss decimal(7,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/store_returns/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS household_demographics ( hddemosk int ,hdincomeband_sk int ,hdbuypotential varchar(15) ,hddepcount int ,hdvehiclecount int ) stored as PARQUET LOCATION '/user/tmp/parquet/household_demographics/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS web_page ( wpwebpage_sk int ,wpwebpage_id varchar(16) ,wprecstart_date date ,wprecend_date date ,wpcreationdate_sk int ,wpaccessdate_sk int ,wpautogenflag varchar(1) ,wpcustomersk int ,wp_url varchar(100) ,wp_type varchar(50) ,wpcharcount int ,wplinkcount int ,wpimagecount int ,wpmaxad_count int ) stored as PARQUET LOCATION '/user/tmp/parquet/web_page/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS promotion ( ppromosk int ,ppromoid varchar(16) ,pstartdate_sk int ,penddate_sk int ,pitemsk int ,p_cost decimal(15,2) ,presponsetarget int ,ppromoname varchar(50) ,pchanneldmail varchar(1) ,pchannelemail varchar(1) ,pchannelcatalog varchar(1) ,pchanneltv varchar(1) ,pchannelradio varchar(1) ,pchannelpress varchar(1) ,pchannelevent varchar(1) ,pchanneldemo varchar(1) ,pchanneldetails varchar(100) ,p_purpose varchar(15) ,pdiscountactive varchar(1) ) stored as PARQUET LOCATION '/user/tmp/parquet/promotion/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS catalog_page ( cpcatalogpage_sk int ,cpcatalogpage_id varchar(16) ,cpstartdate_sk int ,cpenddate_sk int ,cp_department varchar(50) ,cpcatalognumber int ,cpcatalogpage_number int ,cp_description varchar(100) ,cp_type varchar(100) ) stored as PARQUET LOCATION '/user/tmp/parquet/catalog_page/'

CREATE EXTERNAL TABLE IF NOT EXISTS inventory ( invdatesk integer , invitemsk integer , invwarehousesk integer , invquantityon_hand integer ) stored as PARQUET LOCATION '/user/tmp/parquet/inventory/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS catalog_returns ( critemsk int ,crordernumber int ,crreturneddate_sk int ,crreturnedtime_sk int ,crrefundedcustomer_sk int ,crrefundedcdemo_sk int ,crrefundedhdemo_sk int ,crrefundedaddr_sk int ,crreturningcustomer_sk int ,crreturningcdemo_sk int ,crreturninghdemo_sk int ,crreturningaddr_sk int ,crcallcenter_sk int ,crcatalogpage_sk int ,crshipmode_sk int ,crwarehousesk int ,crreasonsk int ,crreturnquantity int ,crreturnamount decimal(7,2) ,crreturntax decimal(7,2) ,crreturnamtinctax decimal(7,2) ,cr_fee decimal(7,2) ,crreturnship_cost decimal(7,2) ,crrefundedcash decimal(7,2) ,crreversedcharge decimal(7,2) ,crstorecredit decimal(7,2) ,crnetloss decimal(7,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/catalog_returns/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS web_returns ( writemsk int ,wrordernumber int ,wrreturneddate_sk int ,wrreturnedtime_sk int ,wrrefundedcustomer_sk int ,wrrefundedcdemo_sk int ,wrrefundedhdemo_sk int ,wrrefundedaddr_sk int ,wrreturningcustomer_sk int ,wrreturningcdemo_sk int ,wrreturninghdemo_sk int ,wrreturningaddr_sk int ,wrwebpage_sk int ,wrreasonsk int ,wrreturnquantity int ,wrreturnamt decimal(7,2) ,wrreturntax decimal(7,2) ,wrreturnamtinctax decimal(7,2) ,wr_fee decimal(7,2) ,wrreturnship_cost decimal(7,2) ,wrrefundedcash decimal(7,2) ,wrreversedcharge decimal(7,2) ,wraccountcredit decimal(7,2) ,wrnetloss decimal(7,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/web_returns/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS web_sales ( wsitemsk int ,wsordernumber int ,wssolddate_sk int ,wssoldtime_sk int ,wsshipdate_sk int ,wsbillcustomer_sk int ,wsbillcdemo_sk int ,wsbillhdemo_sk int ,wsbilladdr_sk int ,wsshipcustomer_sk int ,wsshipcdemo_sk int ,wsshiphdemo_sk int ,wsshipaddr_sk int ,wswebpage_sk int ,wswebsite_sk int ,wsshipmode_sk int ,wswarehousesk int ,wspromosk int ,ws_quantity int ,wswholesalecost decimal(7,2) ,wslistprice decimal(7,2) ,wssalesprice decimal(7,2) ,wsextdiscount_amt decimal(7,2) ,wsextsales_price decimal(7,2) ,wsextwholesale_cost decimal(7,2) ,wsextlist_price decimal(7,2) ,wsexttax decimal(7,2) ,wscouponamt decimal(7,2) ,wsextship_cost decimal(7,2) ,wsnetpaid decimal(7,2) ,wsnetpaidinctax decimal(7,2) ,wsnetpaidincship decimal(7,2) ,wsnetpaidincship_tax decimal(7,2) ,wsnetprofit decimal(7,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/web_sales/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS catalog_sales ( csitemsk int ,csordernumber int ,cssolddate_sk int ,cssoldtime_sk int ,csshipdate_sk int ,csbillcustomer_sk int ,csbillcdemo_sk int ,csbillhdemo_sk int ,csbilladdr_sk int ,csshipcustomer_sk int ,csshipcdemo_sk int ,csshiphdemo_sk int ,csshipaddr_sk int ,cscallcenter_sk int ,cscatalogpage_sk int ,csshipmode_sk int ,cswarehousesk int ,cspromosk int ,cs_quantity int ,cswholesalecost decimal(7,2) ,cslistprice decimal(7,2) ,cssalesprice decimal(7,2) ,csextdiscount_amt decimal(7,2) ,csextsales_price decimal(7,2) ,csextwholesale_cost decimal(7,2) ,csextlist_price decimal(7,2) ,csexttax decimal(7,2) ,cscouponamt decimal(7,2) ,csextship_cost decimal(7,2) ,csnetpaid decimal(7,2) ,csnetpaidinctax decimal(7,2) ,csnetpaidincship decimal(7,2) ,csnetpaidincship_tax decimal(7,2) ,csnetprofit decimal(7,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/catalog_sales/' tblproperties("parquet.compression"="Lz4"); CREATE EXTERNAL TABLE IF NOT EXISTS store_sales ( ssitemsk int ,ssticketnumber int ,sssolddate_sk int ,sssoldtime_sk int ,sscustomersk int ,sscdemosk int ,sshdemosk int ,ssaddrsk int ,ssstoresk int ,sspromosk int ,ss_quantity int ,sswholesalecost decimal(7,2) ,sslistprice decimal(7,2) ,sssalesprice decimal(7,2) ,ssextdiscount_amt decimal(7,2) ,ssextsales_price decimal(7,2) ,ssextwholesale_cost decimal(7,2) ,ssextlist_price decimal(7,2) ,ssexttax decimal(7,2) ,sscouponamt decimal(7,2) ,ssnetpaid decimal(7,2) ,ssnetpaidinctax decimal(7,2) ,ssnetprofit decimal(7,2) ) stored as PARQUET LOCATION '/user/tmp/parquet/store_sales/'

``` ```SQL use tpcds100gcsv; CREATE EXTERNAL TABLE IF NOT EXISTS customer_address ( caaddresssk int ,caaddressid varchar(16) ,castreetnumber varchar(10) ,castreetname varchar(60) ,castreettype varchar(15) ,casuitenumber varchar(10) ,ca_city varchar(60) ,ca_county varchar(30) ,ca_state varchar(2) ,ca_zip varchar(10) ,ca_country varchar(20) ,cagmtoffset decimal(5,2) ,calocationtype varchar(20) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/customer_address/'; CREATE EXTERNAL TABLE IF NOT EXISTS customer_demographics ( cddemosk int ,cd_gender varchar(1) ,cdmaritalstatus varchar(1) ,cdeducationstatus varchar(20) ,cdpurchaseestimate int ,cdcreditrating varchar(10) ,cddepcount int ,cddepemployed_count int ,cddepcollege_count int ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/customer_demographics/'; CREATE EXTERNAL TABLE IF NOT EXISTS date_dim ( ddatesk int ,ddateid varchar(16) ,d_date date ,dmonthseq int ,dweekseq int ,dquarterseq int ,d_year int ,d_dow int ,d_moy int ,d_dom int ,d_qoy int ,dfyyear int ,dfyquarter_seq int ,dfyweek_seq int ,ddayname varchar(9) ,dquartername varchar(6) ,d_holiday varchar(1) ,d_weekend varchar(1) ,dfollowingholiday varchar(1) ,dfirstdom int ,dlastdom int ,dsameday_ly int ,dsameday_lq int ,dcurrentday varchar(1) ,dcurrentweek varchar(1) ,dcurrentmonth varchar(1) ,dcurrentquarter varchar(1) ,dcurrentyear varchar(1) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/date_dim/'; CREATE EXTERNAL TABLE IF NOT EXISTS warehouse ( wwarehousesk int ,wwarehouseid varchar(16) ,wwarehousename varchar(20) ,wwarehousesq_ft int ,wstreetnumber varchar(10) ,wstreetname varchar(60) ,wstreettype varchar(15) ,wsuitenumber varchar(10) ,w_city varchar(60) ,w_county varchar(30) ,w_state varchar(2) ,w_zip varchar(10) ,w_country varchar(20) ,wgmtoffset decimal(5,2) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/warehouse/'; CREATE EXTERNAL TABLE IF NOT EXISTS ship_mode ( smshipmode_sk int ,smshipmode_id varchar(16) ,sm_type varchar(30) ,sm_code varchar(10) ,sm_carrier varchar(20) ,sm_contract varchar(20) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/ship_mode/'; CREATE EXTERNAL TABLE IF NOT EXISTS time_dim ( ttimesk int ,ttimeid varchar(16) ,t_time int ,t_hour int ,t_minute int ,t_second int ,tampm varchar(2) ,t_shift varchar(20) ,tsubshift varchar(20) ,tmealtime varchar(20) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/time_dim/'; CREATE EXTERNAL TABLE IF NOT EXISTS reason ( rreasonsk int ,rreasonid varchar(16) ,rreasondesc varchar(100) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/reason/'; CREATE EXTERNAL TABLE IF NOT EXISTS income_band ( ibincomeband_sk integer , iblowerbound integer , ibupperbound integer ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/income_band/'; CREATE EXTERNAL TABLE IF NOT EXISTS item ( iitemsk int ,iitemid varchar(16) ,irecstart_date date ,irecend_date date ,iitemdesc varchar(200) ,icurrentprice decimal(7,2) ,iwholesalecost decimal(7,2) ,ibrandid int ,i_brand varchar(50) ,iclassid int ,i_class varchar(50) ,icategoryid int ,i_category varchar(50) ,imanufactid int ,i_manufact varchar(50) ,i_size varchar(20) ,i_formulation varchar(20) ,i_color varchar(20) ,i_units varchar(10) ,i_container varchar(10) ,imanagerid int ,iproductname varchar(50) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/item/'; CREATE EXTERNAL TABLE IF NOT EXISTS store ( sstoresk int ,sstoreid varchar(16) ,srecstart_date date ,srecend_date date ,scloseddate_sk int ,sstorename varchar(50) ,snumberemployees int ,sfloorspace int ,s_hours varchar(20) ,s_manager varchar(40) ,smarketid int ,sgeographyclass varchar(100) ,smarketdesc varchar(100) ,smarketmanager varchar(40) ,sdivisionid int ,sdivisionname varchar(50) ,scompanyid int ,scompanyname varchar(50) ,sstreetnumber varchar(10) ,sstreetname varchar(60) ,sstreettype varchar(15) ,ssuitenumber varchar(10) ,s_city varchar(60) ,s_county varchar(30) ,s_state varchar(2) ,s_zip varchar(10) ,s_country varchar(20) ,sgmtoffset decimal(5,2) ,staxprecentage decimal(5,2) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/store/'; CREATE EXTERNAL TABLE IF NOT EXISTS call_center ( cccallcenter_sk int ,cccallcenter_id varchar(16) ,ccrecstart_date date ,ccrecend_date date ,cccloseddate_sk int ,ccopendate_sk int ,cc_name varchar(50) ,cc_class varchar(50) ,cc_employees int ,ccsqft int ,cc_hours varchar(20) ,cc_manager varchar(40) ,ccmktid int ,ccmktclass varchar(50) ,ccmktdesc varchar(100) ,ccmarketmanager varchar(40) ,cc_division int ,ccdivisionname varchar(50) ,cc_company int ,cccompanyname varchar(50) ,ccstreetnumber varchar(10) ,ccstreetname varchar(60) ,ccstreettype varchar(15) ,ccsuitenumber varchar(10) ,cc_city varchar(60) ,cc_county varchar(30) ,cc_state varchar(2) ,cc_zip varchar(10) ,cc_country varchar(20) ,ccgmtoffset decimal(5,2) ,cctaxpercentage decimal(5,2) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/call_center/'; CREATE EXTERNAL TABLE IF NOT EXISTS customer ( ccustomersk int ,ccustomerid varchar(16) ,ccurrentcdemo_sk int ,ccurrenthdemo_sk int ,ccurrentaddr_sk int ,cfirstshiptodatesk int ,cfirstsalesdatesk int ,c_salutation varchar(10) ,cfirstname varchar(20) ,clastname varchar(30) ,cpreferredcust_flag varchar(1) ,cbirthday int ,cbirthmonth int ,cbirthyear int ,cbirthcountry varchar(20) ,c_login varchar(13) ,cemailaddress varchar(50) ,clastreview_date varchar(10) ) row format delimited fields terminated by '|' LOCATION '/user/tmp/csv/customer/'; CREATE EXTERNAL TABLE IF NOT EXISTS