Is Odoo scalable?
#13 - How We Became Experts in Odoo Large Data for Midmarket and Corporate Projects
Hi,
Welcome to issue #13 of Odoo Trailblazer, our journey to become the #1 Odoo partner!
Today, the Tokyo skyline is graced by a beautiful day, setting the perfect stage to explore the realms of data management and performance with business software.
We often proclaim that software systems are scalable, as they process 10, 100, 1000, or even 10,000 tasks automatically with almost the same operational cost. But do these numbers have a ceiling? Can software handle 1,000,000 or 1,000,000,000 tasks?
The answer: it's complicated. When deploying a business software for a medium to large-scale enterprise, considering these scaling factors is crucial. Today, we'll use Odoo as a case study, elucidating its limitations and how we tackle them.
Some relevant resources:
Our article on Odoo API solutions
Our article on Large data
Our product: data management solutions for large Odoo projects
Agenda
What are the limits?
When is the right time to start?
How do we solve it
Our data solution for Odoo
1. What are the limits?
Integrated business software might find handling large data challenging due to three primary factors:
Volume of transactions
Complexity of the workflow and flexibility of the software
Scope & design of the implementation
1.1 Volume of transactions
Since our inception in 2015, we've dealt with large data projects. Our most significant client back then processed 100,000 sales order lines daily.
The metric of sales order lines is a quick gauge to estimate the client's size, taking into account the daily order volume and the average order size.
To put this into context, processing 100,000 sales order lines per day equates to roughly 30 million annually. In comparison, an Excel Spreadsheet maxes out at 1,048,576 rows, requiring a new Excel file creation every 10 days if you were to manage your business using Microsoft Excel.
1.2 Complexity of the workflows and flexibility of the software
Software of this nature often allows the processing of numerous business rules. E.g. verifying data input against many conditions, applying business logic to compute price, checking compliance, or user access rights.
As the software becomes more complex, it demands more computing power. Such software is usually very adaptable, catering to the diverse needs of various industries and business processes. This adaptability can lead to a lack of optimization for specific configurations.
With older versions of Odoo, even validating a sales order of 100 items was taking over a minute, because it had to generate delivery orders, booking the stock, etc. Today it is much faster, but you still need to optimize your Odoo installation.
1.3 Scope & design of the implementation.
The implementation scope significantly impacts your system's speed and its limitations. For instance, organizing your delivery orders in 1 step or 3 steps (picking, shipping, delivery) will have a profound effect on the generation time of your sales and delivery orders, and on the size of your stock transaction table.
Over time, any table in your database (such as sales order line, accounting entries, stock transactions) will grow, and the larger the table, the slower it will be to query.
1.4 Combine all parameters
The limit isn't a standalone figure. It's a composite of your data, the complexity of the apps, and the implementation design. Remember, a PostgreSQL table can start to decelerate if it surpasses a million rows. However, the likelihood of performance issues in your Odoo installation increases significantly at 100 million.
2. When is the right time to start?
If large data challenges aren't properly addressed in your enterprise software installation, the problems compound over time as your database expands. The repercussions for users aren't just slower software, but it also becomes increasingly unusable and unreliable.
When should we start considering data volume and performance when implementing business software or Odoo?
"Earlier" is the answer.
In our early projects, we often overlooked the potential of performance issues. Identifying such challenges upfront allows you to plan your requirements more effectively or even simplify your scope. In some instances, it might even influence the choice of software.
3. How did we solve it?
Our ability to handle large data volumes stems from three key areas:
Odoo's annual new version releases
A dedicated infrastructure and engineering team
Data management solutions
Odoo consistently releases newer and faster versions each year, with Odoo 16 far outpacing its predecessor, Odoo 15, and so forth.
In 2020, we established a dedicated infrastructure team, followed by a data engineering team in 2021. The former ensures a robust backbone for large data projects in terms of servers, network, database, security, etc. The latter is responsible for developing complex data workflows between Odoo and other systems, and implementing data solutions for larger projects.
Simultaneously, we started to build a data management solution for Odoo, named DACO (DAta COnnector), that we use to implement large numbers of stores or POS together:
Or manage complex and large data workflows in big Odoo clients:
Conclusion & Recommendations
Never underestimate the performance or large data risks when implementing any business software for a mid-size or large enterprise. Seek expert help, ideally before embarking on your implementation project.
Useful links:
To follow me on Linkedin
To implement Odoo with us
If you are a partner, and you need experts in large data project