Doubts with Hive Integration


Hadoop Eco System Forums Hadoop Discussion Forum Doubts with Hive Integration

Tagged: , ,

This topic contains 8 replies, has 2 voices, and was last updated by Profile photo of Siva Siva 4 years, 10 months ago.

Viewing 9 posts - 1 through 9 (of 9 total)
  • Author
    Posts
  • #1675 Reply
    Profile photo of Subramaniyabharathi
    Subramaniyabharathi
    Participant

    Hi,
    After executing the ‘Insert overwrite command’ and load the data into hbase table. the data resides in both hbase and hive warehouse. Since these two tables are mapped which each other. If i delete/drop the table in hive, will it affect the table in Hbase also?

    #1677 Reply
    Profile photo of Siva
    Siva
    Keymaster

    Its a nice question. Yes, it will affect the Hbase table as well. if you drop table in hive, it will also drop the table in HBase as well.

    #1678 Reply
    Profile photo of Subramaniyabharathi
    Subramaniyabharathi
    Participant

    Hi,
    Thanks Mr.Siva. So is there any other possible way? Because of both the tables are there it takes lots of memory, Thus making my cluster running out of memory.

    #1679 Reply
    Profile photo of Siva
    Siva
    Keymaster

    As far as my knowledge, i don’t think it will maintain two copies of data. Only one copy of data is referenced both from Hive and HBase.
    On which basis, you are saying that it is maintaining two copies of data. Did you create External table or Managed table in Hive?

    #1680 Reply
    Profile photo of Siva
    Siva
    Keymaster

    As far as my knowledge, i don’t think it will maintain two copies of data. Only one copy of data is referenced both from Hive and HBase.
    On which basis, you are saying that it is maintaining two copies of data.

    #1681 Reply
    Profile photo of Subramaniyabharathi
    Subramaniyabharathi
    Participant

    Hi,
    I checked it in the WebUI of hadoop. when i loaded the data into hive table. dfs used was 30.02gb. But after executing “insert overwrite command”, it showed dfs used as 59.34gb. Dont know why?. I didn’t create any external table. i just followed the steps in “Hive Integration with Hive” explained in this website. Thanks.

    #1684 Reply
    Profile photo of Siva
    Siva
    Keymaster

    Oh..You should have created an External Table in Hive with the help of HBaseStorageHandler then it will not keep two instances of data. Please try by creating an external table instead of managed table.

    #1685 Reply
    Profile photo of Siva
    Siva
    Keymaster

    Hey…now i got solution to ur question why its size is doubled… can you call on 9704231873

    #1687 Reply
    Profile photo of Siva
    Siva
    Keymaster

    As discussed, you can safely remove the test table created in Hive, from which you selected rows to insert into Hbase table.

Viewing 9 posts - 1 through 9 (of 9 total)
Reply To: Doubts with Hive Integration
Your information: