1. Mailing Lists
  2. Contributors
  3. Re: Product variant import (performance issue)

Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Re: Product variant import (performance issue)

Re: Product variant import (performance issue)

Avatar

Re: Product variant import (performance issue)

by
MoaHub, Graeme Gellatly
- 26/01/2016 05:34:54
I did loads of work on this years ago for the old product variant.

Our largest product variant was 22,000 products and would take a very long time.  500,000 variants in all.  It would go at roughly 3000 per hour.  Creation and updates.

On examination there were many things to speed the process and by the end we got to 500/minute for creation and 5000 per minute for updates which was acceptable.  Of course none of this applies to v8 variants as implemented by Odoo which is quite broken, but the general principles remain.

What we did in essence was
use ORM for the Create calls.  This ensured required fields, log_access fields and defaults were correctly filled.  It also returned the ids.
We then just ran an UPSERT SQL statement essentially for all the records.

But in your case, I'd advise heavily against the SQL method or even csv and go with the usual interface if it is just one template.  I've done a bit of investigation and the Odoo implementation is very very limited.  Even csv is going to suck, but at least you don't have to deal with calculating stored functional fields, m2m relations etc.  Even then you are going to have 16,000 create calls, and probably 16,000 write calls as well.  And if you don't get the variant ordering exactly right, any change to the template will delete them all anyway.  Some of my comments are in product-attribute repo pull requests which show what is not working and no way to override.  (Sorting, sequencing, ordering, optional variants to name a few)

I know Pedro's module as part of OdooMRP, but it really doesn't work for many use cases outside of it, especially for people who do need all the variants.  OdooMRP is kind of this whole system of modules which work around other limitations, but really works best as a whole system rather than individual modules, mainly due to the Odoo function itself create_variant_ids.

I'd just start a second server, not in worker mode, on a different port so you don't hit timeouts, memory limits etc and let it go overnight.  8000 is not so many, will only take a few hours, and a lot less time than writing the csv files.  Normally you can monitor the process just using top anyway and see when its done after browser timed out. 



On Tue, Jan 26, 2016 at 11:53 AM, Maxime Chambreuil <maxime.chambreuil@savoirfairelinux.com> wrote:
Hello,

I have to create/import one product with 8692 variants. I left the creation runs for more than hour, no success.

Before I create a csv file with all the different combinations, I would like to know if you have a better suggestion.

Thanks!
--
Maxime Chambreuil
Ring ID: 239e01f6262afbf3c7d276d2c661173ddddc0340

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


Reference