Must read Building the next-gen Chevereto

Rodolfo

The Chevere Guru
Chevereto Staff
Saw some of the things your planning for 4.0 and I'm super excited! Definitely going to upgrade as soon as it's released.
Thank you!

I haven't been able to commit any updates on V4 as at this very moment I'm designing how the application will actually run. Internally, the new system is simple as input > plugins > output, the actual caveat is that in this new thing everything is distributed and tuned for performance. The boost is just insane, going from ~700 req/s up to 12K... In the very same machine, with the same code.

To be honest, I wasn't expecting such huge performance. Combined with the unique flexibility of this thing and its cleanliness, I'm sure that V4 will be all what I ever wanted for Chevereto. Add on that that I want to create a lot of other software, I just can't wait to start shipping new stuff for you guys.
 

Rodolfo

The Chevere Guru
Chevereto Staff
2020-07-20

Hey there,

The documenting for reference is complete and all the public reference is now available!

It took many systems and time to get there, in fact, I had to build entirely the system that parses the code and presents it in a human-readable format, with support for searching and all the neat stuff that you can see. 👍🏾 It was very important to have this reference available, as it sets the framework standard and it will be used everyday by developers wanting to work with Chevereto.

It took longer than expected, but the result is just next level and I'm very happy with it. You will find a hard time trying to find another reference of this level anywhere else because it is all in-house development.

Also docs, the system I built for this is just awesome stuff to work with. I've to keep documenting the remaining components and to fix the old stuff I wrote previously, but it shouldn't be that complicated as is just describe the how-to, I don't have to write a parser or anything. As the reference is now complete, the documenting is even easier from now on and it just flows naturally.

Code coverage has been pushed to 100%, and I've fixed many odd stuff that slipped by. At some point I realized that I was actually removing features so that's how I can tell that the hardest part of the job is already done. At this point, the framework is usable (and kind of safe) for production, but I will only now how good it is until I start using it for real.

I will start crafting V4 public API within this week, from there, I will keep pushing updates to docs and examples powered by the work on V4. The designing approach that I'm taking for V4 is centered in the API user, as the public API is what will define the application, it sounds more than reasonable to center the design around it.

Most of the application logic already exists (V3, remember?) but the code not only has to be adapted for the new V4 runtime logic, it must be deeply reviewed and refactored. At some point this process will became very mechanic and I will be doing it at a very insane speed, but is just too much code, it needs time.

I will keep you guys updated, thanks for the interest in the development process.
 

Rodolfo

The Chevere Guru
Chevereto Staff
@Rodolfo do you have any estimate for v4 to be released?
Seems you missed that part.


ETA

Based in my horrible way to measure time, I could say that "it will take a while" to have it ready for being used in real projects. I guess that the time needed only to complete the replacement of V3 to V4 could take up to 24 months. Of course, this is accounting all what V3 currently does.

The base edition, similar to the current Chevereto-Free feature set, should be available within 12 months being very optimistical and remembering that is just the basic feature set.

No surprises here, it is a 13 year old project, is quite big for one person.

Please note that I'm assuming zero collaboration in my timing measurements. It will be ready way faster if I more people can help me out!​
 

Rodolfo

The Chevere Guru
Chevereto Staff
2020-07-31

Another week in development and I've great news as I have already designed the base application model 🎉 and it will be based on message queue. This implementation will allow to separate each process as a collection of steps.

The current process in V3 to upload an image into your account is a long procedure that starts checking the login cookie, the image integrity/validity, checks for dupes, generate medium and thumb sized versions, calculate stats, update database and then it replies back. All these processes are hard wired directly in the application code, making impossible to fine tune the control flow. As you may imagine, the process in V3 is sequential and it happens in just one channel and it is very annoying to bring alterations or any improvement really in these instructions. Did you tried? Share your experience so I can make a point here.

V4 is different as it introduces the usage of workflows to define the application procedures. A workflow is a collection of orchestated steps, which are atomic functions that perform a single task. In V4, the process to upload an image to your account is a workflow, and stuff like the duplicate check is one of its steps.

A workflow works similar to a pipeline, and you can pass values from previous steps. It also supports to add steps at a named position with the intention of allowing third-party plugins to add more proceses. For example, you may want to add checking the image hash before doing anything and a plugin could provide such functionality.

It also allows to deliver a more reliable application as if something fails, the worflow can be resumed at any step. Perhaps B2 breaks (like it never happens) so rather than just throwing an error and fail, the workflow run will be avaibale for inspection and to resume the process once the incident have been addressed.

This application model is perfect for extending the software as it will really allow us to have very deep customized applications. It is also safer to run, to monitor, etc.

Hope you like the heads up!
 
Top