tag:blogger.com,1999:blog-7813991035505148508.post2316967725467629865..comments2024-03-24T19:00:19.546-07:00Comments on Woonsan on Open Source Software: Can't we store huge amount of binary data in JCR?Woonsan Kohttp://www.blogger.com/profile/00788991540896347936noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-7813991035505148508.post-84049562007047743272022-06-09T01:23:10.911-07:002022-06-09T01:23:10.911-07:00Great to hear that! Thanks for the drop in!
Indeed...Great to hear that! Thanks for the drop in!<br />Indeed, S3DataStore becomes more and more accessible and useful than SFTP or WebDav.<br />Happy hacking and cheers,<br />WoonsanWoonsan Kohttps://www.blogger.com/profile/00788991540896347936noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-16493521172866257732022-06-08T22:24:49.560-07:002022-06-08T22:24:49.560-07:00Hi!
Just wanted to drop in and say "Thanks!...Hi! <br /><br />Just wanted to drop in and say "Thanks!" for your blogs on the S3DataStore, it really made moving across from the default DbDataStore very easy. I got it to work on brXM 14.6.3, your migration tool made all this very straightforward.<br /><br />Thanks again!Marnix Kokhttps://www.blogger.com/profile/08898898610094970268noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-28758862884955278102022-04-14T06:02:26.863-07:002022-04-14T06:02:26.863-07:00Sorry for my too late response. Just FWIW, it is s...Sorry for my too late response. Just FWIW, it is still applicable at least in v14, which is the last version I played with as a contributor. If your DataStore configuration wasn't picked up, then it probably means that your custom repository.xml wasn't picked up for some reason.<br />And, AFAICT, I don't believe Apache Jackrabbit OAK's BlobStore would work in Apache Jackrabbit 2.x implementation which is the base of Bloomreach v14.Woonsan Kohttps://www.blogger.com/profile/00788991540896347936noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-54077002663951095212021-03-22T14:40:40.576-07:002021-03-22T14:40:40.576-07:00Hi Woonsan,
I was wondering whether to what exten...Hi Woonsan,<br /><br />I was wondering whether to what extend this is still applicable to Bloomreach v14 and beyond? After some initial experimentation it seems my data store is simply ignored.<br /><br />Also, besides `jackrabbit-aws-ext` there are various alternative DataStore implementations developed under JackRabbit Oak. For example: Microsoft Azure Blob Storage. Do you think these implementations are pluggable in Bloomreach as well?<br /><br />Thanks!Anonymoushttps://www.blogger.com/profile/06120431600908589182noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-33901355160149181282019-09-24T22:33:01.406-07:002019-09-24T22:33:01.406-07:00For huge data storage I recommend to use dataroom....For huge data storage I recommend to use <a href="https://data-room.co/" rel="nofollow">dataroom</a>. It is the best to seure any data.Social Signalshttps://www.blogger.com/profile/04830911033254982863noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-26599595772325819322019-03-20T14:51:04.572-07:002019-03-20T14:51:04.572-07:00Yes, see https://woonsanko.blogspot.com/2016/10/pl...Yes, see https://woonsanko.blogspot.com/2016/10/playing-with-apache-jackrabbit.html. Set the target with S3DataStore instead of VFSDataStore, and then you will be able to migrate DbDataStore to S3DataStore. Once migrated and your repository.xml is updated to use S3DataStore, you may remove the DATASTORE table as the table is used only by the old DbDataStore. Backup will be the last resort, of course.Woonsan Kohttps://www.blogger.com/profile/00788991540896347936noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-13963903295825981912019-03-20T11:46:39.445-07:002019-03-20T11:46:39.445-07:00Hi Woonsan, I currently hace a Hippo instance usin...Hi Woonsan, I currently hace a Hippo instance using default data store (mysql db), I want to use S3 datastore, there is a way to migrate all the existing binaries in mysql to S3 after add S3 datastore?Leanhttps://www.blogger.com/profile/16306807551294466263noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-40192220896568711112018-10-08T06:26:21.850-07:002018-10-08T06:26:21.850-07:00Great! :-)
Also, feel free to post any questions o...Great! :-)<br />Also, feel free to post any questions or suggests in Apache Jackrabbit Lists if you meet anything later: http://jackrabbit.apache.org/jcr/mailing-lists.html<br />Cheers,<br />WoonsanWoonsan Kohttps://www.blogger.com/profile/00788991540896347936noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-23032291262917458312018-10-08T04:39:22.076-07:002018-10-08T04:39:22.076-07:00Hi Woonsan, it was actually as easy as I expected,...Hi Woonsan, it was actually as easy as I expected, but we had some problems because apparently you cannot use '@' in your SFTP password. You need to escape those! :S<br />Thanks anyway. We've got it running :)<br />Cheers,<br />Jesper Anonymoushttps://www.blogger.com/profile/14055137159187787344noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-60841514174130346472018-10-05T10:15:59.572-07:002018-10-05T10:15:59.572-07:00Hi Jesper,
You can find repository.xml examples fo...Hi Jesper,<br />You can find repository.xml examples for different database envs here:<br />- https://www.onehippo.org/library/deployment/configuring/databases.html<br />Just replace DataStore element there with VFSDataStore of S3DataStore following the examples.<br />Cheers,<br />WoonsanWoonsan Kohttps://www.blogger.com/profile/00788991540896347936noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-75698710078684539712018-10-05T06:53:35.266-07:002018-10-05T06:53:35.266-07:00Hey Woonsan, thanks! I'm trying to make a POC ...Hey Woonsan, thanks! I'm trying to make a POC with this and our project right now, only I'm having some problems combining SFTP with MySQL in the repository.xml. Do you maybe have an example for a situation like this? Since H2 is only used for local development.<br />Looking forward to your response.<br />Cheers,<br />JesperAnonymoushttps://www.blogger.com/profile/14055137159187787344noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-92010118019508164852018-10-01T08:31:24.057-07:002018-10-01T08:31:24.057-07:00Hi Jesper,
The reason why you were able to still d...Hi Jesper,<br />The reason why you were able to still download binaries while the backend being down is that VFSDataStore and S3DataStore extends CachingDataStore (see p6 and p7 in my slides) which maintains a binary cache in the local file system for performance. So, once cached, binaries are served from local cache if found. As binaries are immutable in Jackrabbit, that makes sense to keep cache without a big issue. CachingDataStore also has its own garbage collector to remove any item not referred by anything else periodically.Woonsan Kohttps://www.blogger.com/profile/00788991540896347936noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-51228079147214835382018-10-01T08:03:59.489-07:002018-10-01T08:03:59.489-07:00It depends on your DataStore configuration in repo...It depends on your DataStore configuration in repository.xml. It can be S3, VFS (SFTP or WebDAV) or Database or local file system. I was talking about VFS options as well as S3 in this article. But you can choose whatever you want from those options, changing the element configuration in repository.xml.Woonsan Kohttps://www.blogger.com/profile/00788991540896347936noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-68906363510732190242018-10-01T03:26:02.540-07:002018-10-01T03:26:02.540-07:00Hi Woonsan,
Great blog and thanks for keeping it ...Hi Woonsan, <br />Great blog and thanks for keeping it up to date :)<br /><br />I've got your demo project running in combination with a SFTP server (from https://hub.docker.com/r/atmoz/sftp/ -> super easy, I recommend using this for a quick setup). It works perfectly fine, although I have a question about the storage of the binairy files. I uploaded different types of assets (pdf,docx,png,xls,txt) and also different images. They all seems to appear on my SFTP server and also appear in my /console with a reference number (Content Identity number) to the actual asset stored on the SFTP server. Although, when I turn off the SFTP server I am still able to view, download and even upload new files. Nothing changes, although I do get exceptions because it's unable to upload them to the SFTP server. But nothing breaks, so I'm quite curious about how this works. I still see a reference number which does not make sense, because the SFTP server is down, and I am still able to use this asset as before. Is this a fallback meganism? Is this just a caching layer? Or are the files stored twice (sftp & repository)?<br /><br />Looking forward to your response!<br />Cheers,<br />JesperAnonymoushttps://www.blogger.com/profile/14055137159187787344noreply@blogger.comtag:blogger.com,1999:blog-7813991035505148508.post-7302474632137409462018-08-30T03:23:09.018-07:002018-08-30T03:23:09.018-07:00Hi Woonsam, its very nice and good explanation.
I...Hi Woonsam, its very nice and good explanation. <br />I am very new to jackrabbit.<br />I have a question may be its silly or wrong.<br />When i am trying to store a file (not a big file) using jackrabbit where the actual content will store, will it be in the s3 storage (if i configured DataStore using S3DataStore) or in the database where i configured for persistenceManager.Anonymoushttps://www.blogger.com/profile/04973507177033683638noreply@blogger.com