• Welcome to the Chevereto user community!

    Here users from all over the world gather around to learn the latest about Chevereto and contribute with ideas to improve the software.

    Please keep in mind:

    • 😌 This community is user driven. Be polite with other users.
    • 👉 Is required to purchase a Chevereto license to participate in this community (doesn't apply to Pre-sales).
    • 💸 Purchase a Pro Subscription to get access to active software support and faster ticket response times.
  • Chevereto Support CLST

    Support response

    Support checklist

Problem with Mysql replication

lovedigit

👽 Chevereto Freak
Hello,

I am trying to set up a geo-steered server instances in multiple locations globally. I am still new to this, but I have played with this system recently. There are few websites that I have been hosting without any problem with this setup, however, Chevereto seems to act a little awkward.

Goal: I want to setup the frontend of the chevereto website in Tokyo, Singapore, Amsterdam, Australia, and US in order to minimize the latency. I am using Amazon's geo-dns to send users to the nearest server location based on their location.

There is a Master-Master Replications across the databases of all these locations individually. i.e. each location has their database synced at all the times, so that when a change is made in one location, it can be replicated to all the datacenters. Moreover, the files are also being synced bidirectionally with the help of unison and ssh key chain.

Result: Everything works perfectly. The system works as it should. I can verify it by using VPN service to change my location and setup the servers to return subdomain of the server location browser is being connected to. For example, if I set my VPN to US, the server returns en.domain.com. Simillarly, when connected to Asia, the webpage is servered by asia.domain.com, Australia by au,domain.com, Europe by eu.domain.com and Japan by jp.domain.com. I tried carrying out natural navigation, and everything worked as expected. Logged in to my account, Created a new account from signup page, deleted an image, renamed an image and so on. I tested whether or not these actions were replicated on other datacenters, and they did.

However, the real and biggest issue with this system is that chevereto seems to start from the same #ID, as if it stores the last id of the image somewhere outside the database. As a result, if I upload an image at Eu server and another image at US server at the same time, they end up sharing the same image ID, which means we have two or more images at the same image ID when accessed from the different locations. For Example, en.domain.com/image/CXjlK will have different image than eu.domain.com/image/CXjlK.

I really can't figure out whether it is something related to Chevereto, this script simply isn't designed to handle the database replications or perhaps I am doing something wrong?

I would like to mention that there are 3 other highly active websites which is working fine with this setup. One of them is a file hosting website.

Any pointers will be a great help to me, as I really want to scale my chevereto website to handle anticipated traffic over the next few months.
 
it stores the last id of the image somewhere outside the database

Nope, it store it in the database at chv_id_reservations (if you use id filenaming).

I am doing something wrong?

Are you using id filenaming? When you use that, Chevereto must reserve the ID before completing the process because it needs to use encoded ID as the filename. So when someone uploads something this ID gets reserved in let's say USA but if the replication isn't made instantly someone in Asia will reserve the same ID.

Since you are doing merging, what happens in one database is merged to the whole so the id reservation stuff isn't working, either you replicate after insert a reservation or you don't merge your database.

Usually you replicate a database to have a failover or mirror just in case something goes wrong.

You have to remember that your client doesn't connect directly to your database so have one database vs multiple databases doesn't make a big difference in terms of response times if you setup the remote server properly and if the connection bewteen these is fast enough.
 
Nope, it store it in the database at chv_id_reservations (if you use id filenaming).
Oh, so it is reserving the id somewhere. The fact that it does so in the database somewhere doesn't mean it can be avoided during replication. The website will always get the reference from local database reserved ID, and since it is Master-Master replication, the database that contains newer changes will get replicated, which results in conflict of IDs. I am not sure What I can do here. If I exclude the replication of chv_id_reservations, it will still have error, since there may be 10 uploads on server location 1 and 5 uploads on server location 2. And I believe they use the same incremental algorithm to assign the ID to each image upload. Which means, next time there are 5 uploads on server location 2, it will have same ID as those 5 images at server location 1, where we uploaded 10 images initially.

Are you using id filenaming? When you use that, Chevereto must reserve the ID before completing the process because it needs to use encoded ID as the filename. So when someone uploads something this ID gets reserved in let's say USA but if the replication isn't made instantly someone in Asia will reserve the same ID.

Since you are doing merging, what happens in one database is merged to the whole so the id reservation stuff isn't working, either you replicate after insert a reservation or you don't merge your database.

No, I am not using ID filenaming. I am actually using the Random Filenaming method instead.
62c19038bfa0f2c2d4e7cb70cafbc96c.jpg



Usually you replicate a database to have a failover or mirror just in case something goes wrong.

You have to remember that your client doesn't connect directly to your database so have one database vs multiple databases doesn't make a big difference in terms of response times if you setup the remote server properly and if the connection bewteen these is fast enough.

Yes, that's true. I have used the replication to have real time up-to-date backup of databases with Master-Slave replication. It is really helpful and ensures that you never loose your information, especially when you have those backup servers located in different location altogether.

However, in my case, connection through Tokyo or Australia will still incur some latency even if the connection is fast enough. Keeping that in mind, I thought it would be better to have a local database replication. If everything fails, I will eventually have to move to single database system as a last resort. But it beats the purpose somewhat if not completely.
 
Back
Top