1. Mailing Lists
  2. Contributors
  3. Re: OCA Addons Packaging next step

Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Re: OCA Addons Packaging next step

Re: OCA Addons Packaging next step

Avatar

Re: OCA Addons Packaging next step

by
Vauxoo, Nhomar Hernández
- 13/04/2016 20:55:06


On Wed, Apr 13, 2016 at 12:38 PM, <Bidoul@pad.odoo-community.org> wrote:
Hi Nhomar,

Thanks for your interest! It's never too late. 

There are a lot of different topics in your e-mail, let me try to answer to the most importants:

> pbr

We don't use pbr at all in setuptools-odoo. It happens to be used by OCA/maintainer-tools but that's unrelated. I had a look at pbr and it would not help here.

OK, we should rethink that then, but clear OT this point.
 

> Plain list of packages vs PyPi:

I agree pushing to pypi is better. It's the goal but there are a few thinks to resolve first, see my previous post in the same thread. 
We are moving one step at a time, gaining valuable experience at each step. 
Trust me, doing it all in one giant leap is going to fail. Only by doing can we see all implications, and some are subtle.

what I tried to say is that we can have odoopypi.odoo-com.org alá pypi withour program a single line, just thinking in the deployment process (may be build a docker file or a vagrant is enought) because the pypi tool is freely available already.

More Info Here:


It means no need to do it plainly. 


> one package per addon

This is by far the most flexible approach, given the automatic dependencies we gain. You install only what you need. One package per OCA repo would create a dependency mess and/or depend on OCA-specific conventions (such as oca_dependencies.txt). Nevertheless there are meta-packages per repository, see for instance the end of my post: https://odoo-community.org/blog/oca-news-1/post/installing-oca-addons-the-easy-way-32.

In odoo's word that's not true.

the dependency mixtore horizontally and bertically in odoo is huge.

Basically deploy a customer in that way is like live with Linux beta kernel version always.... this package approach is valid ONLY if we manage properly a relase proces s which in not viable at all in a production environment.

Version depends of:

1.- Commit of your own repo.
2.- Commit of other repositories.

And it mus affect:

1.- Migrations
2.- Updates of views and data.
3.- Change on business process (which is human).

And the patches must be managed and contributed:

By Module.
By Commit.

Basically this is how kernel maintainers do the job but even the packaging process on the kernel is not done by the kernel itself is mixed up by distributiios (which at the end of the day has its own packages systems speratelly).

It means... it do not encourage the contributios it encourage the split of effort.

But BTW is conceptually nice. More comments in the proper project on Github.

 

> odoo-autodiscover 

is very simple, it just adds the odoo_addons namespace package directories to addons-path. It's indeed optional if you fear sideeffects (there are none known, and we have it battle-tested for several months now).

Thats the point, are you injecting in which order? the order matter and that's part of the deployment process. But nice if it is **optional.
 

> odoo-autodiscover has no tests

Indeed. It's not easy to devise a test strategy for that, I'm open to ideas (and PR's :)
setuptools-odoo has 93% test coverage.

I was talking about the other one. But yers I need to see details to understand some whys.
 

> branding

It's not Acsone branded at all. Acsone just happens to be the author :) We can move it to OCA anytime of course. At this stage it's still beta and there are no other contributors. If a team can be formed around this, I'm happy to do the move.

Can you start this job please?

Because for us is important contribute to OCA's repositories not custom repositories for deals like this importance.
 

> The setuptools_odoo is excellent BTW, I would do the same, 

Glad you like it. 

> I am worried a little is in the support of the more pracmatic tests environments:

I'm not sure what you mean by that. setuptools-odoo if fully tested (just run tox to see).
For the rest, improving the test mechanisms for Odoo addons is a whole new project in iteself. And Odoo seems to have interesting stuff in preparation on master on that matter. So I'd rather not invest into that right now.

I am talking about:

$wget url.module.
$cd module
$virtualenv
$pip install --editable .
$setup.py tests #<< This is a fundamental step for a package without this is basically uselss a deployment per package process.

or

$tox # This should be better (did you know tox has as many tools and power than a MakeFile?)
 
$cat setup.py | __version__  # this (or other) command must have all the coordinates to enable the developer mode (version, revision, sha, repo, whatever)


Be care about the enable process for virtualenv in your helps trying to show it is **simple:

Famous problem with lxml:¿? < and this is just 1


Then if we oversale that install something with odoo - oca is as simple as that we are wrong and if it is only to share individual code the app store is doing the job or even it become as simple as:

$ unzinp repo.zip
$ cp repo/module folder_addons_path(or even /home/user/.local/share/Odoo/addons/VERSION.0)

Automate that can be as simple of index in json (or whatever other tool) the list of modules.

Make all the setup.py stuff just to copy a folder without all the impact in the context evironment IMHO is a bad approach the **packaging process is not just copy the code.

Be care of that. 

For easier discussion, I'd encourage you to create issues on the setuptools-odoo project so we can track each topic independently.


Agree, Let's see there 

Cheers,

-sbi

On Wed, Apr 13, 2016 at 11:38 AM, Nhomar Hernandez <nhomar@vauxoo.com> wrote:
Hello.

Sorry I came late to the party, but I have several questions/comments which I hope stay properly in this time of the work.

1.- I see you use pbr from openstack in some of your packages even if it is **fine** when you see deeply the "Why" and the how they did it even they expend a lot of time checking out how to solve things and then they did not make thing too generic in some points [0][0.1].

The Point- May be the **logic behind the why PbR can be used but not the pbr package itself, which is "little configuration more decisions taken" which is good for the case where you dominate all teh set of platform but is not the case of more than 1000 communitary modules in odoo's word, even they do not recommend use PBR if you will do too much customizations.

The documentation **looks good but it is not[1] basically pbr is a for of setuptools monkeypatched, even they are complaining about that, but they dig too deep on this way of do things and they are survivig with such decisions.

Now 2 options:

a.- Continue using pbr, and dig into the documentation to do thing properly (and maybe contribute with the great openstack project) >> I do not like this option, they have a momentum worst than ours.
b.- Dig into other options which they are and python itself is evolving [2] or not there. (odoo evolve to fast then use old tools with old incorrect decisions can bring us more problems). Resuming this [3] which is supported for maybe Odoo 10 all the word is moving to Json (python is doing that) and odoo itself too [4].

Any of them must mention this topic properly, last 3 weeks I dig into huge problems with mixed new packages with pbr installed because setuptools 20.x introduce new behavior that was not properly working with PbR itself.

2.- Plain list of packages vs PyPi:

I am sorry if my scope is too high but IMHO have a list of zips "officials" is not for me a proper solution for anything, just this 3 features of pypi are almost mandatory for anything:



It means it is not correct IMHO push to a solution with a pull strategy this must be a push strategy (the app store strategy is wrong now but at the end we know odoo is a little wrong some times).

I prefer an API where people PUSH their versions and when I say people I mean PSC's and not something automatic that is saying some half trues to developers or sysadmins, it is too dangerous IMHO.

That work is better done by http://www.odoo-code-search.com/ and it is already done BTW.

FYI PyPI is already freely available. https://wiki.python.org/moin/CheeseShop

3.- I feel your first move was GREAT!! seriously but for this approach I think we need to thing big if not nobody will use it properly and we can cause more damage than solution.

4.- Question: Why a package per module and not a package pero project? >>75 percent of modules won't work bythemself standalone, what's the point of separate them?

Experiences like plone and tryton teach us that REALLY plain index of stuff are just for experienced programmers which at the end **block** the community and not push the collaboration.

When you navigate into plone and tryton index it is a real pain, I think we can do better, I am against officialize a place which is incomplete, I vote for a GOOD sprint (even paid or crowdfounded) with a proper milestone for this tasks.

What I am worry about? Nhomar is crazy it is a first step?

I am worry that once this is officialized the it adquires a momentum that is impossible to stop and becomes a blocking point instead a helper, don't take me wrong I by myself dedicated a lot of hours to the same topic and passed myself for your decisions taken already (Pbr for example) and I realize of the huge issues behind which I am mentioning, I think this is an extraordinary **0.0.0.1-alpha** release of whatever we can do but by far can not be officalized yet.

5.- This package scares me a little: https://pypi.python.org/pypi/odoo-autodiscover
The posibilities of brake an actual installation with the autodiscovering feature is too dangerous I prefer the  I prefer the current way of doing it declaratively (it means I must start odoo telling the addons-path explicitally) but I understand it is optional ¿right?.

6.- No unittests? even if you are monkeypatching the start script?. too dangerous IMHO.

7.- The general Idea is GREAT with the packaging, but I think 1 package per repository properly versioned with a global bumpversion[5] config file in enought and simplier to mantain.

8.- The setuptools_odoo is excellent BTW, I would do the same, I am worried a little is in the support of the more pracmatic tests environments:

setup.py module tests?
tox?

It means the are the key for the packaging system, if the package itself can not be tested individually IMHO it is pointless separate it.
oca_dependencies support?

If you put people to google for the dependency structure we will dig in a problem worst than the repositories management itself.

9.- Fork and PR?

Packaging strategy should pull people to contribute as easy as possible, IMHO the module should have all coordinates to achieve such deal:

- repository.
- specific change_id (or commit).
- centralized place where to send a patch.

10.- Less important but it is a matter of say everything "Branding" are we in the OCA now accepting officialize tools which are depending on spscific contributors branding repositories? IMHO it is not correct. Depends all the packaging infrastructure on a package maintained under github.com/other_brand_than_oca/reponame is not correct. But I know acsone and I understand he is open to put this package under OCA umbrella I supose << this point is just for mention this topic and not miss it.

11.- Dig a little deeper in OpenStack site, I love it, I dream the OCA becomes something like that organization, technically we need to move on that direction IMHO. Then I think use their logics and their approaches is excellent, but be care, not all what they develop in preoperly thought to be used generically.

12.- [SEUDO OT] Documentation: I think is time to put all things toghether and threat documentation as code, this kind of features must be extraordinarly well documented before officialize them, look how openstack does:

Global Documentation: 

How document:


All is opensource also:

- And the tools are opensourced properly: http://docs.openstack.org/contributor-guide/doc-tools.html

Can we copy that? I am doing it internally BTW, but we must be open on that, and I think that even major number of topics and policies are REALLY COOL to be adopted by use as they are to not reinvent the wheel.

Conclusions:

THANKS really THANKS you showed that a LOT of things are possible and I love the idea and this first approach will help us a lot to achieve the end objective.

I just hope this point help us to move forward better and helps you to do it also.

[2] From main page:
PBR builds on top of the work that d2to1 started to provide for declarative configuration. d2to1 is itself an implementation of the ideas behinddistutils2. Although distutils2 is now abandoned in favor of work towards PEP 426 and Metadata 2.0, declarative config is still a great idea and specifically important in trying to distribute setup code as a library when that library itself will alter how the setup is processed. As Metadata 2.0 and other modern Python packaging PEPs come out, PBR aims to support them as quickly as possible.

On Wed, Apr 6, 2016 at 3:23 AM, <Bidoul@pad.odoo-community.org> wrote:
For reference, this is implemented with [1]

This script does two things.

1/ add the setup.py to all repos. This is comparable to the update of README with the addons table or what transbot does in the sense it pushes to the git repos. So I don't think that part can/should be done with travis. In any case, that part is will disappear in the future when the setup.py are more and more fine tuned manually.

2/ generate all wheels on a nightly basis and publish them to wheelhouse.odoo-community.org on the OCA server. This could be done with travis but I'm not sure it's wise to have travis upload files to the OCA server.

When the time will come to push to pypi we can reconsider using travis for that. But there are still a few hurdles to overcome, such as:
* the dependency on odoo (which is not on pypi): I've no feedback from Odoo SA yet on that topic
* the fact that we need to maintain several series (8.0, 9.0) where pypi seems to be more geared towards having a single latest version of any given package; for Odoo addons we have several "latest" versions for a single package (one for 8, one for 9, etc)
* the fact that we currently generate a new package for each commit to an addon (using a "dev" version), that's fine for publishing to wheelhouse.odoo-community.org but it's not good practice to push all dev versions to pypi; to fix that we need proper release management of addons.

Best regards,

[1] https://github.com/OCA/maintainer-tools/pull/156/files

PS: as I side note it's interesting that I forwarded the last e-mail to 3 addresses (Alexandre Fayolle <alexandre.fayolle@camptocamp.com>, Stéphane Bidoul <stephane.bidoul@acsone.eu>, oca-beckport@odoo-community.org), the last one not existing. Nevertheless Odoo sent it to contributors@odoo-community.org. So no harm here, the discussion is interesting, but be careful with Odoo mailing lists, kids. :)

On Wed, Apr 6, 2016 at 8:08 AM, Joël Grand-Guillaume <joel.grandguillaume@camptocamp.com> wrote:
If it is possible to do it on Travis, +1 !

On Tue, Apr 5, 2016 at 11:38 PM, Moises Lopez <moylop260@vauxoo.com> wrote:


I don't know if is good for you travis way
Check follow example pylint-odoo-auto-deploy-pypi

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Nhomar Hernandez
CEO Vauxoo.
Twitter: @nhomar
Odoo Gold Partner
Skype: nhomar00 (Envia mail previo no lo superviso siempre).
Móvil Venezuela:
+58 4144110269
Móvil México:
+52 1 4773933942

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Nhomar Hernandez
CEO Vauxoo.
Twitter: @nhomar
Odoo Gold Partner
Skype: nhomar00 (Envia mail previo no lo superviso siempre).
Móvil Venezuela:
+58 4144110269
Móvil México:
+52 1 4773933942

Reference