I’am planning to use Pimcore 5 for a project with a amount of objects and assets reaching the billions mark.
Pimcore will be running in a multi-environment setup (dedicated database, multiple frontends, NFS, S3, …) and the objects will be organized in 10-20 classes – of course with some relations to each other (also to the assets).
Apart from optimizing the database and using corresponding hardware, are there any suggestions for a setup with this huge amount of objects and assets? How should I start for having as few as possible effort with horizontal scaling?
Are there any projects out there running with that amount of objects and assets without problems? If yes, how they are set up? Does it make sense, to divide the frontends (with user and session data) from the other data and work with kind of a api between the instances? Or is it better (for the beginning) to keep everything in one database/cluster?
Another question is, are there any suggestions for the object and also the asset folder structure – e.g. separating in a lot of folders so it’s possible to use the advantage of lazy loading? Will the internal backend search handle billions of objects and millions of assets with an acceptable speed? Of course, it should be possible to comfortably work with those objects and assets in the backend.
I would be really happy to get some answers I can work with – of course with keeping you up-to-date on this.
Thanks in advance,