1. Mailing Lists
  2. Contributors
  3. Re: Migration to version 9

Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Re: Migration to version 9

Re: Migration to version 9

Avatar

Re: Migration to version 9

by
Elico Corp, Eric Caudal
- 31/08/2015 03:46:07
Hi,
I have additional thoughts on the one module = on repo. This is the time now to discuss where we want to go and not focus yet on the difficulties to get there. I mean moving repos etc. might imply extra work but would be beneficial if end structure proves to be more stable.

Mozetic has reminded us a truth that we tend to forget too often: the end user is our sponsor and is the one we need to please.

We are most of the time having a technical/developer point of view in the way we organize the data and community:
- github is a technical tool: I reckon that even for non-technical like me, it is no that difficult to master for basic operations but still.
- Most of the discussions I see today i about how to make the technical backend not too complex. This is sane but not enough.
- We need to care about the workflow and validate it with the different options but again we care more about the technical workflow than the end-user workflow.

What main kind of end users do we have:
1.- Technical staff (at ease with github, CLI, Linux, etc...): developer and sysadmin
2.- Functional staff with technical background (maybe not able to develop a module but can use github, install modules from CLI). Usually business owners with geek background and no IT resource.
3.- End user/Functional staff (at ease with erp, apps.odoo.com, few knowledge of github or technical details etc.)

Current workflow to get the modules is somehow OK for 1 and 2 but not satisfactory for 3.

What a end user wants?
- Access one module and test it easily to check if it fits his needs
- Test a set of modules (eg: connector or MRP)
- download, install and get update for modules or set of modules

For end user to access the modules, there are several options of which none is currently satisfying:
- Github: difficult to find one specific module, all the more in the current structure (modules are not easy to find in repos).
- apps.odoo.com: actually the easiest way to find specific modules for a neophyte.
- Online apps in Odoo: simple and straight forward (even if I know that some of you are against it)
- Runbot: needs improvements in the interface (but the "try me in runbot" button goes into the right direction), better advertising and nice tutorial but could be a perfect tool for test. Maybe not the right tool for complex sets and requires systematic unit tests and demo data.

Somehow we need to rely on github/apps for distribution unless we build our own infrastructure (which is out of scope for the moment): we need to improve the current way our work is hosted and distributed in Github.

Currently the repo scope is not that straight forward and is moving constantly: there are a lot of discussions about what should be where and about cross-dependencies modules or reporting.
I am not assessing here the technical difficulties of the one module = on repo (some of you have already made good points) but I think they can be fixed (with time and effort though).
Still, we need to build today a structure/infrastructure that can stand the growth up to 2000 modules and current repo structure is not flexible enough (I differ with Raphael here on scope limitation).
Here are some points IMO that are overlooked:
- There is already the idea of application in Odoo which reflects in the naming: website_*, sale_*, etc. It is simple and understandable by everybody. It is most of the time enough for classifications. Repo "website" is already redundant with the natural naming.
- Major difficulty today is to download 500 repos: a good script for end users OR a mirror of all modules to a repo container would be easy to arrange. End users would only have to pull that container in their main directory and restart Odoo (with all limitations of such move!). This repo (or docker or recipe) could contain only stable modules (or a mild idea of certified ones).
- We talk about bundler/recipe etc. There is an easy way to bundle: create a meta-module with dependencies. Whoever wants to create one is welcome: it requires few technical skills and it is easy to create a specific repo for it/them. It might need an uninstall function or procedure but nothing very difficult to implement.

Getting "one module=one repo" is leaner, simpler and more flexible on the structural point of view. I understand it is challenging on the technical side but current model does not scale up and we need to think about best evolution or a middle ground to get there.

--
Eric Caudal [Founder and CEO]
Skype: elico.corp. Phone: + 86 186 2136 1670 (Cell), + 86 21 6211 8017/27/37 (Office)
Elico Shanghai (Shenzhen/Singapore) Odoo Gold Partner, best Odoo Partner 2014 for APAC
On 08/30/2015 05:23 AM, Maxime Chambreuil wrote:
<blockquote cite="mid:1783713490.1098660.1440882441057.JavaMail.zimbra@savoirfairelinux.com" type="cite">
Raphaël,

To me, there is 3 types of modules :
  • the official ones, part of the core and maintained by Odoo SA
  • the community ones, maintained by the OCA
  • the private ones specific to a customer and maintained by the integrator/partner or the customer
I understand you have a 4th type : the community ones maintained by their author. They might be maintained by the OCA in a future but you don't know. It would have to go through a selection process. What would be your criteria ? By selecting a set of modules, you end up defining a functional scope. How can the OCA commit to provide a scope with no resources ? Who will be migrating the defined scope when version X is released ? Today modules are migrated based on the customer demand by the integrator. I think it should always stay that way. Natural selection should be the rule.

"Again, many open source eco-system which do have a proper package manager deal just fine with 1 repo = 1 module."

Can you name a few ? I have checked Android and Gerrit. I understood Google made a tool to manage the repositories, but their repo doesn't depend on each other. Each app is independent and can be tested and reviewed separately. Regarding Gerrit, they use Gerrit to centralize reviews of all their repo. Maybe something we could explore.

"And I don't know any good open source ecosystem that scales to the kind of complexity an ERP is by grouping its repos in the same repos."

The Linux Kernel has only one repo. I think it compares to ERP's complexity. Like in the Linux Kernel, the deeper we go in the dependency tree, fewer the number of maintainers and more specific the module is. But at every step of the tree, the quality is maintained by the upper reviewer and it is still part of the Linux Kernel.

To move into your direction, we need to agree on many things and I am afraid it will never happen. It will also put us in a position to say "No" to someone. I prefer to say "Yes, but you have to support it and make it pass Travis tests". No obligations for the OCA, no need for resources, no commitment for long term support or migration. The OCA provides visibility to find users and contributors and infrastructure for quality modules. Natural selection will do the rest at every new version.

The OCA is not here to define and make a product and provide long term support for it. The OCA is here to support the collaborative development of Odoo features and promote its widespread use. At some point, collaboration has to be forced upon people, because we all want to decide alone.

Regarding incubation, it should happen in the author OCA-forked repository and it could happen by increasing the test criteria with Travis on 9.0 branches. Some modules will be dropped by natural selection.

Concerning egg, deb, or rpm packaging, how is it impacted by the number of repo ? Why can't you do it today ? I don't know Bundler. What does it provide that we don't have with anybox's recipe or pip or Odoo connection to Apps ? The OCA could also distribute his modules through apps.odoo-community.org and use this repo on our instances. Here again, I don't think the OCA is here to impose a choice to everyone. Like Anybox recipe, we all adopted it and it would deserve to be part of the OCA and benefit from his infrastructure and visibility. If you come up with a solution to generate packages and the community uses it, then again the OCA should provide the visibility and the infrastructure.

Sorry for the long email.
--
Maxime Chambreuil
+1 (514) 276-5468 #126

----- Le 29 Aoû 15, à 12:07, Raphaël Valyi <rvalyi@akretion.com> a écrit :

Hello Maxime,

You said "To those who want 1 repo = 1 module, show me the contributors to manage them."

To which I answer: yes I think 1 repo = 1 module is the only sustainable way in the long run, because it's a decentralized model that reflects very much de decentralized econonomy or ecology of open source. Now I'm not telling this should happen at v9, may be not due to the transition effort.

Yes, If the OCA has say 2000 modules to manage, if these 2000 modules becomes suddenly 2000 branch this indeed is more work and that would not be sustainable.

This is why I say: this move should be along with a reduction of scope of what is under the OCA double review process. I think the OCA and the ecosystem would be stronger if the OCA only focus on being a community backing of the core made by Odoo SA along with an interface with the community at large instead of trying to be the whole community itself, something it will never fit anyway as the ecosystem will be growing.
So my proposal is not more work for the OCA, it's the same amount of work, balancing with decentralization.

Finally let me remind you that many aspects of having 1 module =1 repo boil down to having a decent package management. And may be we don't have yet a decent package management for doing the transition on v9. Buildout is not too bad, but it's still very far away from tools such as Bundler. Again, many open source eco-system which do have a proper package manager deal just fine with 1 repo = 1 module. And I don't know any good open source ecosystem that scales to the kind of complexity an ERP is by grouping its repos in the same repos. If you know some please give me examples.

Finally, while I think we should tend toward splitting repos I think it's okay to have some grouping at the core, specially when it refelects it's under the authority of a single group of people. Like, it's okay to have 1 repo for the addons publihsed by Odoo SA, or may be it should be 5 or 6 repos like the OCA. It's Okay to have a single OCA repo for the root modules of a localization, it's may be okay to have some root OCA modules grouped for backing or another topic. Of course the limit is a grey area, but waht we could do is core reviewers vote, like for PR's if a given module is really universal enough to them to be part of these root repos. But my point is soon enough, the consensus that some module is universal enough will not exist for some module. And if despite of that, that module is still of great quality, if it does have the backing of some OCA reviewers, then it could still be an OCA module, but in it's own repo. Also, that would make it smoother to have incubating repos for projects that do not yet have the backing of enough OCA reviewers, but want to show their political intention to be part of the OCA and conform to the OCA quality standards (tests, coverage, cenventions  etc...)

Regards.

On Sat, Aug 29, 2015 at 11:53 AM, Maxime Chambreuil <maxime.chambreuil@savoirfairelinux.com> wrote:
Good morning,

My issues with the 1 repo = 1 module :
  • We will lose some history during the migration.
  • The migration will consume a lot of resources to make the changes on Github, Travis, Coveralls, Codacy, Transifex, Runbot, Apps, the OCA website
  • We don't have the tools to efficiently manage 800 repositories nor the resources to build that tools.
  • It will spread out the collaboration. Some people will work on their repo in their corner and will not get visibility, leading to few reviews and poor code quality.
  • At some point in the development/deployment process, you will need to specify the list of repo to build your environment and you will want to freeze it. On average, we install 100 to 150 modules for a customer. Managing 150 repo URL with their commits to freeze them will be a pain.
  • Same issue if you use eggs, rpm or deb packages for each module : you need the list of modules and their versions. Additional problem here is that module version today is not as reliable as the commit.
The only advantage is that it solves our current problem : new Odoo version release means new branch when we want to migrate the module. No need to remove anything. Clean and complete history.

I think the git-filter option as described on the wiki is the best trade off :
  • we keep the history
  • we provide an easy way to see which modules have been migrated
  • it allows us to match the number of repo with the resources to manage them
  • it encourages collaboration
  • it still allows anyone to deploy with packages by generating them based on module version
To those who want 1 repo = 1 module, show me the contributors to manage them.

Cheers,
--
Maxime Chambreuil
+1 (514) 276-5468 #126

----- Le 29 Aoû 15, à 7:08, Daniel Reis <dgreis@sapo.pt> a écrit :

The key to succeed with the approach of 1 repo 1 module is if we have a really extraordinay package manager, and even the official python ones are not good enought and depends too much of the programmers skills.

That's not the bottleneck. There are already two or three implementations with possible solutions (one of them for MQT).

The first problems are collaboration workflows (handling new modules and PRs) and topic governance (module compatibility and overlapping).

These are the main issues that one module repos need to address in the first place.


--DR

No dia 29/08/2015, às 08:08, Nhomar Hernandez <nhomar@vauxoo.com> escreveu:


On Sat, Aug 29, 2015 at 1:38 AM, <Mozetič@pad.odoo-community.org> wrote:
But what maintenance workflow is expected for the 1 module 1 repo approach?

I like less repos with more modules.

But all excess are wrong.

Our repositories has >500 modules and you lose the vision and migrations, quality control, SQA, CI are nearly impossible there.

One repository with more that 20 or 30 modules can be considered "Huge".

We are trying to split them also per "Area" but when you have 1 module per repository (we have few of them) all the points I mentioned above are really easy to mantain, but at the end the SQA is so reducted to a minimalistic testing approach and environment.

I think both are well it depends of the case.

Today Linux itself manage a huge separation of topics (modules) and install something brings you install hundresds of packages, but the package manager is the key (not for nothing gnu gcc is so so so old and almost untouched since time ago).

BUT anybody is totally happy with actual package managers and prefer install manually some little packages...

The key to succeed with the approach of 1 repo 1 module is if we have a really extraordinay package manager, and even the official python ones are not good enought and depends too much of the programmers skills.

THat's my opinion about that!


--
Nhomar Hernandez
CEO Vauxoo.
Twitter: @nhomar
Odoo Gold Partner
Móvil Venezuela:
+58 4144110269
Móvil México:
+52 1 4773933942

_______________________________________________
Mailing-List: http://odoo-community.org/groups/oca-contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


_______________________________________________
Mailing-List: http://odoo-community.org/groups/oca-contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Raphaël Valyi
Founder and consultant
+55 21 3942-2434




Reference