1. Mailing Lists
  2. Contributors
  3. Re: 30 Checkout Operations per Minute A job for Odoo or not ?

Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Re: 30 Checkout Operations per Minute A job for Odoo or not ?

Avatar

Re: 30 Checkout Operations per Minute A job for Odoo or not ?

by
Vauxoo, Moisés López Calderón
- 14/05/2021 17:17:33


El jue, 6 may 2021 a las 5:02, Frederik Kramer (<frederik.kramer@initos.com>) escribió:
Hi Moises, 

Am Freitag, den 30.04.2021, 16:02 +0000 schrieb Moises Lopez:


> Hi Frederik


> 


> A customer requested to support 5k concurrent sale.order (1 or 2


> lines of products) in Odoo.


> The process that we followed were: test, measure, fix, test...

haha, yes thats what works well if you have a customer that well
endorses this "agile" and "pivoting" style of work. Apart from all
economic considerations this is probably also something that isolated
leads to quite solid solutions

Well, we are talking about 5k concurrent VUs in this special case.
The cost vs benefit demands this.


> 


> The first time that we tested the checkout process supported 15


> concurrent virtual users (VUs)


> The measure said that it had heavy custom functions in the same


> transaction of checkout but it could be executed in background (cron,


> queue)


> and there were slow queries because of indexes issues or too many


> queries executed because of python code

did you use any specific profiling tools here? Or the ones that rcollet
often proposes in his talks about the matter ?

Watch my talk in SFO
I used the same tools


> 


> Note: Be careful with the base_automation if you can migrate them to 


> code it is better.

Thanks for the hint, i think we experienced that also quite some times
in the past and learned that lesson already ;-)



> 


> After fix them the next iteration supported 400 VUs


> So good but so far from the focus.


> 


> The next iteration we found that there were methods that could be


> using LRU-cache (e.g. prices, website that don't change)


> Now we needed to check where the cache was clearing too frequently


> and fix it.

How did you measure LRU cache hit rates vs. code parts in a good way.
Any tooling suggestion here ?

For the case of the price in the website our reasoning was:
- The value don't change in a long time for this customer (each 6 months)
- It couldn't be a stored field in the database because it depends on:
  - date
  - quantity
  - currency rate (if your have multi-currency)
  - partner
  - Supplier price (if your price depends on it)
  - product category
- Adding LRU-Cache with these keys will work fine and faster
Even the website the most of request came from public user

I share you the output of the following command in production:
 - kill -SIGUSR1 WORKER-PID

It shows the hit rates:
 - 19715 hit 91.5% ratio for website._get_cached_values
 - 3507 hit 75.8% ratio for website.get_pricelist_available_cached



> 


> We implemented CDN


> e.g. The sentence "/web/image/company.logo" runs 10 queries


> (prefetching fields, translating fields...) for each user opening the


> page


> After using CDN 0 queries are executed.

But you just used Odoos standard CDN function or any specific measure
here ?


> 


> We removed soft process that running with many users it is heavy


> e.g. the random image that odoo assign when you create a res.partner


> e.g. computed fields moved to cron

Also here which analysis tools do you prefer to do so ?



> 


> After that we detected concurrent update issues


> It is hell for performance matter.


> e.g. updating the same record at the same time from many users,


> e.g. allowing pressing the same button too many times


> e.g. sequences using no-gap in the main process, and so on


> 


> After finishing all the hints of performance issues in the code and


> database the next steps were to improve the hardware.


> 


> Now, It supports 5k concurrent VUs doing checkout process (choose 2


> products, type address, payment card info and confirm)

In what time frame ?



> 


> So, Odoo will support those users but you will need to test, measure,


> fix, test...


> Stress testing tools helps us a lot here


> python profilers and query analyzers too


> 


> Maybe you are interest to watch our talk in SFO about:


>  - https://youtu.be/kiREZqg3JQg

I will definitely have a look at this

Thanks a lot for your extensive insight.

Cheers from rainy Germany to sunny Mexico ;-)

Frederik



> 


> El mié, 28 abr 2021 a las 15:35, Frederik Kramer (<


> frederik.kramer@initos.com>) escribió:


> > Hi Folks, 


> > 


> > i have a project proposal on my desk that will ultimately end up in


> > approx. 30 checkout transactions per minute. Has anybody got a


> > project


> > with that order volume done with a relatively recent version of


> > Odoo


> > and provide a good reference. Please bear in mind that the project


> > ahead involves a similarly high volume of stock moves.


> > 


> > Any suggestion / comment like (but not limited to) 


> > 


> > Odoo is the wrong technology for that, forget it, because....


> > 


> > over


> > 


> > You can do it but, a), b), c) ....


> > 


> > to


> > 


> > If you really want that beast flying involve us over here because


> > we


> > are among the few ones that already did soemthing similar


> > 


> > would be appreciated. 


> > 


> > From our load intensive projects at initOS i know that 10000 orders


> > a


> > day can be handled but almost 50000 is a different story and thats


> > why


> > i am asking the most experienced resource out there ;-)


> > 


> > Thanks in advance for your replay


> > 


> > Cheers Frederik


> > 


> > 


> > -- 


> > Dr.-Ing. Frederik Kramer


> > Geschäftsführer


> >         


> > initOS GmbH


> > An der Eisenbahn 1


> > 21224 Rosengarten


> >         


> > Phone:  +49 4105 56156-12


> > Fax:    +49 4105 56156-10


> > Mobil:  +49 179 3901819


> >         


> > Email: frederik.kramer@initos.com


> > Web:   www.initos.com


> >         


> > Geschäftsführung:


> > Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke


> > 


> > Sitz der Gesellschaft: Rosengarten – Klecken


> > Amtsgericht Tostedt, HRB 205226


> > Steuer-Nr: 15/200/53247


> > USt-IdNr.: DE815580155


> > 


> > _______________________________________________


> > Mailing-List: https://odoo-community.org/groups/contributors-15


> > Post to: mailto:contributors@odoo-community.org


> > Unsubscribe: https://odoo-community.org/groups?unsubscribe


> 


> 


-- 
Dr.-Ing. Frederik Kramer
Geschäftsführer
        
initOS GmbH
An der Eisenbahn 1
21224 Rosengarten
        
Phone:  +49 4105 56156-12
Fax:    +49 4105 56156-10
Mobil:  +49 179 3901819
        
Email: frederik.kramer@initos.com
Web:   www.initos.com
        
Geschäftsführung:
Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke

Sitz der Gesellschaft: Rosengarten – Klecken
Amtsgericht Tostedt, HRB 205226
Steuer-Nr: 15/200/53247
USt-IdNr.: DE815580155

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe



--
Moisés López Calderón
Mobile: (+521) 477-752-22-30
Twitter: @moylop260
hangout: moylop260@vauxoo.com
http://www.vauxoo.com - Odoo Gold Partner
Twitter: @vauxoo

Reference