tag:blogger.com,1999:blog-860423771829255614.post8073228651920917483..comments2023-06-22T01:52:58.037-07:00Comments on Lineland: HBase Architecture 101 - StorageLars Georgehttp://www.blogger.com/profile/13990677998590435541noreply@blogger.comBlogger42125tag:blogger.com,1999:blog-860423771829255614.post-50396450122774848622015-11-26T01:32:23.635-08:002015-11-26T01:32:23.635-08:00Great article... Thanx for sharing... Can you plea...Great article... Thanx for sharing... Can you please upload more????<br /><br /><br /><br /><a href="http://www.newyearimages2016.com/" rel="nofollow">Happy New Year 2016</a>Anonymoushttps://www.blogger.com/profile/06186924297990357520noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-9062836775961359622014-01-08T15:17:39.480-08:002014-01-08T15:17:39.480-08:00Yes, merge is using B+tree
in its own RegionServer...Yes, merge is using B+tree<br />in its own RegionServer<br />Yes.Pihttps://www.blogger.com/profile/09459270445676730997noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-88147928093348967662014-01-08T15:13:11.242-08:002014-01-08T15:13:11.242-08:00In HMatser
From ZoopkeeperIn HMatser <br />From ZoopkeeperPihttps://www.blogger.com/profile/09459270445676730997noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-14139187218935312082014-01-08T15:01:05.676-08:002014-01-08T15:01:05.676-08:00HFile.
you mean memstore, yes.
Hbase will split in...HFile.<br />you mean memstore, yes.<br />Hbase will split into another region in same regionserverPihttps://www.blogger.com/profile/09459270445676730997noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-14601782504679310992014-01-08T14:53:53.220-08:002014-01-08T14:53:53.220-08:00per RegionServerper RegionServerPihttps://www.blogger.com/profile/09459270445676730997noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-58182880451148497122013-06-05T21:55:24.774-07:002013-06-05T21:55:24.774-07:00Yes you are right.Yes you are right.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-46641824392443305112013-06-05T21:54:21.530-07:002013-06-05T21:54:21.530-07:00This article is talking HFile v1. In V2 this has c...This article is talking HFile v1. In V2 this has changed to multi level indexing with B+tree like data structure for efficient processing and the index is moved at block level. Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-45810661359892482712013-03-29T23:08:13.864-07:002013-03-29T23:08:13.864-07:00One question: In the first big picture, the HLog s...One question: In the first big picture, the HLog should be per RegionServer, not per Region ?Anonymoushttps://www.blogger.com/profile/12185377828997169368noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-9023661256233294532013-03-08T07:47:25.711-08:002013-03-08T07:47:25.711-08:00Hi Lars,
Thank you for this article. I am working...Hi Lars,<br /><br />Thank you for this article. I am working on building a Hadoop System and there will be HBase too. We are planning to server multiple purposes with the same data. There will be HBase, but there will be Mapreduce as well. You mention use of HFiles, does that mean I will have to plan space for both HFiles and Mapfiles for the same data (like doubling the storage volume), if I were to use Hbase and Mapreduce on the same Hadoop cluster.<br /><br />Thank you,<br /><br />OnerOakhttps://www.blogger.com/profile/08196532970542175787noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-6806332693428082592012-12-07T04:36:29.089-08:002012-12-07T04:36:29.089-08:00I am working on Hbase. I have query regarding how ...I am working on Hbase. I have query regarding how Hbase store the data in sorted order by LSM Tree.<br /><br />As per my understanding, Hbase use LSM Tree for data transfer in large scale data processing. when Data comes from client, it store in-memory sequentially first and than sort and store as B-Tree as Store file. Than it is merging the Store file with Disk B-Tree(of key). is it correct ? Am I missing something ?<br /><br />If Yes, than in cluster env. there are multiple RegionServers who take the client request. On that case, How all the Hlogs (of each regionServer) merge with disk B-Tree ?<br /><br />Is it like Hlog only merge the data with Hfile of same regionServer ?Anonymoushttps://www.blogger.com/profile/14482644241256122217noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-76186532240472591992012-06-26T00:10:03.304-07:002012-06-26T00:10:03.304-07:00Thanks a lot! Its a really great blog! Any details...Thanks a lot! Its a really great blog! Any details on the magic part of the header?Anonymoushttps://www.blogger.com/profile/01221815252348791056noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-74848575317192780032012-05-18T14:19:34.596-07:002012-05-18T14:19:34.596-07:00If a record is like {
Name : "ABC"
Addre...If a record is like {<br />Name : "ABC"<br />Address : "XYZ"<br />Number : "123"<br />}<br /><br />Does HBase store the name of the column in the row with every row or does it store it once for the table and reference it in the individual rows?Yash Ganthehttps://www.blogger.com/profile/07800211441627230676noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-38873130608665219562012-04-22T13:27:11.017-07:002012-04-22T13:27:11.017-07:00Lars,
i am having hard time understanding the valu...Lars,<br />i am having hard time understanding the value of a data block. What would HBase lose if concept of block is removed?<br />You could compress whole file (in fact compression would be better), you could build an index for whole file. You can make file available by replicating it.Unknownhttps://www.blogger.com/profile/10880114070254033807noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-33667802833223652362012-04-20T09:40:10.477-07:002012-04-20T09:40:10.477-07:00Lars,
Is the green block named "data" in...Lars,<br />Is the green block named "data" in your diagram part of one physical file? I am new to hadoop, and my understanding is that in hadoop, there is a concept of data block, which is represented by one file and replicated over multiple data nodes. Is there one to one relationship between a datablock in and a T-file?Unknownhttps://www.blogger.com/profile/10880114070254033807noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-4869951741920352462012-04-12T01:45:51.266-07:002012-04-12T01:45:51.266-07:00hey everyone...i am using hadoop-0.20.2 with hbase...hey everyone...i am using hadoop-0.20.2 with hbase-0.92.0....when i start the hbase with slaves in hbase/regionservers file.there is this error of (no descriptor for .Meta)..n hbase doesnot start but with only master in regionserver file it starts jus perfectly..can anyone telme wats da problem here n how to start all the slaves as regionseervers...>????....<br />please reply asap...Tahreemhttps://www.blogger.com/profile/02591357999463647898noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-22957712434199665552012-01-23T03:03:19.677-08:002012-01-23T03:03:19.677-08:00Thanks for very informative article.Thanks for very informative article.Surendra Mhttps://www.blogger.com/profile/18322362933631041603noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-26664034044036587032012-01-13T09:11:38.606-08:002012-01-13T09:11:38.606-08:00Thanks for this article and the book. Please can y...Thanks for this article and the book. Please can you detail "encoded-regionname" notion, and how it is created?linus novahttps://www.blogger.com/profile/00925931599883439191noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-52323669636120507092012-01-13T09:06:45.349-08:002012-01-13T09:06:45.349-08:00This comment has been removed by the author.linus novahttps://www.blogger.com/profile/00925931599883439191noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-27971670151712443782011-12-14T07:02:17.947-08:002011-12-14T07:02:17.947-08:00Thanks for a great article! Have there been any ma...Thanks for a great article! Have there been any major changes in hbase architecture for the last couple of years?vanyatkahttps://www.blogger.com/profile/17304669927201680470noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-80427199348235650692011-10-24T14:11:01.930-07:002011-10-24T14:11:01.930-07:00Thank you. Well detailed article. It helped to und...Thank you. Well detailed article. It helped to understand the internalsRaja Thiruvathuruhttps://www.blogger.com/profile/12284823676835264792noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-10012300566396859452011-08-16T01:38:41.431-07:002011-08-16T01:38:41.431-07:00Hi Larse,
In Hbase , what is the actual location ...Hi Larse, <br />In Hbase , what is the actual location where ROOT and META TABLEs are stored? where can i see that ROOt and meta data in the disk.vamshionbloghttps://www.blogger.com/profile/09265209239545977297noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-12233051019719464742011-08-05T05:24:28.751-07:002011-08-05T05:24:28.751-07:00Thanks for information provided! Very useful.
I ...Thanks for information provided! Very useful. <br /><br />I have some questions:<br />Where does HBase store the row key? Is it in the memory of each region server? If my row keys are very large, what will happen? <br /><br />Thanks!Leonhttps://www.blogger.com/profile/00727879434623224282noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-85006855533702609522011-06-11T20:42:18.446-07:002011-06-11T20:42:18.446-07:00Thanks for this article! It really helps!Thanks for this article! It really helps!anirudhtodihttps://www.blogger.com/profile/17355991738972107826noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-62320741562834249072011-04-02T15:02:56.323-07:002011-04-02T15:02:56.323-07:00One correction:
"The first set of files are t...One correction:<br />"The first set of files are the log files handled by the HLog instances and which are created in a directory called .logs underneath the HBase root directory. Then there is another subdirectory for each HRegionServer and then a log for each HRegion. “<br /><br />It should be:<br />"Then there is another subdirectory for each HRegionServer. In that subdirectory, there are several HLog files (because of log rotation). All regions from that region server share the same HLog files."Zheng Shaohttps://www.blogger.com/profile/05495751131627874757noreply@blogger.comtag:blogger.com,1999:blog-860423771829255614.post-74900551194502905592011-03-14T05:55:33.127-07:002011-03-14T05:55:33.127-07:00Thanks for letting me know! Interesting, this site...Thanks for letting me know! Interesting, this site has ripped of off other blogs too.Lars Georgehttps://www.blogger.com/profile/05191379407576369008noreply@blogger.com