1. Mailing Lists
  2. Contributors
  3. Re: Runbot long term to install packages and other features

Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Re: Runbot long term to install packages and other features

Avatar

Re: Runbot long term to install packages and other features

by
LasLabs, Dave Lasley
- 07/08/2016 23:58:07
We have been running a Dockerized Runbot at LasLabs since around the time we wrote that Bamboo article, and I would have to agree that it has significantly cut down on our administrative overhead. 

Even though I am fully supporting this initiative, I feel the need to play devil’s advocate. Every good thing comes with a tradeoff and that is also the case here. Following are some issues we have run into, most of which are fixable easily:

  1. I’m not totally privy on hard drive usage w/ normal Runbot, so maybe this is an issue there as well, but the Docker images are being stored beyond instance termination. This has been great from a debugging perspective, but does bring up some scaling concerns when deployed at OCA level
  2. Along the same note of hard drive, obviously the builds are larger in general (~2.5GB + db) due to container overhead. It would probably be possible to slim the base system down, but there will still be the overhead
  3. A lot of the existing OCA Travis configs are not compatible, and will skip builds due to lack of TESTS=1 matrix
  4. Build logs are not dropped into Odoo DB & subsequently displayed on the log page
  5. We had an instance with some dumb config settings on the base Odoo instance, which caused it to restart at one point. While the instances are destroyed in Runbot as they normally would be, the restart unhooked them from their Docker instances, which resulted in a bunch of straggling instances using ports that were then used on subsequent builds.
  6. This seems fixed on Vauxoo’s Runbot, but our’s is outputting colorized logs in core. This causes false positives on builds that are catastrophically broken, such as a missing import - note how there is still a coverage report below the Test Summary, which indicates a travis_after_tests_success. While the issue itself looks to be fixed in newer code & we were able to monkey patch a quick solution to make the builds red, the point here is that adding another layer does add new sets of possible failure points.

With the minor gotchas stated, I would say that this solution has saved more man hours than it has caused - including our integration with internal CI like Bamboo. 

On Aug 7, 2016, at 12:49 PM, Moises Lopez <moylop260@vauxoo.com> wrote:

* History

A) Install packages
The current process to install packages for our runbot for OCA is:
1. Request to install pip and apt packages.
2. Wait manual installing by server-admin.
3. Receive the confirmation of package installed.
4. Rebuild manually.

Between 2 and 3 we could wait a pair of days or weeks

If we need of one person this delay is normal (anyone who is).

But we have a requirements.txt file that we can re-use to install dependencies on runbot.


B) Reproduce travis errors
Currently we have errors from travis that we can't reproduce on local and we don't have access to the build of travis to debug that environment.

We can't reproduce this error from other environment the last one have a video of how to detect them with the same environment: https://youtu.be/r6-IE87UZQc

C) Repositories dependencies
Currently we have a runbot where we need configure manually the repositories dependencies, but we have a oca_dependencies.txt.
Runbot have a method to process them and MQT have other one for travis.
We can re-use it too (similar case from requirements.txt) and avoid duplicated code and re-configurate 2 times the same thing.


* Proposal

- Pieces
The tool travis2docker generates a Dockerfile base on .travis.yml (86% of coverage)

(Our use a custom image with custom packages pre-installed and other things to make caching https://hub.docker.com/r/vauxoo/odoo-80-image-shippable-auto/)

All repos of OCA are using a .travis.yml with MQT centralized.

We have a odoo module runbot_travis2docker for version 9.0 and 94% of coverage

- Join them
We have all pieces to join them to fix A) B) and C).

With this pieces if you add a package from .travis.yml, requirements.txt or oca_dependencies we re-use the same code of MQT to generate the instance with all apt and pip packages, and all repositories dependencies recursively.
We don't need extra custom configuration from runbot instance or server commands. 

FYI we have a stable runbot using on http://runbot.vauxoo.com/runbot
(Offtopic with a isolated environment we have a ssh conexion feature to debug direclty from container if a error is hardly to reproduce locally).

It's 100% is compatible with others CI like as @lasley told us here

Conclusion
My proposal is migrate our current runbot to dockerized one.
@Lasley @alan196 and me give you our help to configure and fix new issues.

FYI @Alan196 send us a video of how configure it here

Note: We have other cool tools, but many of them require a extra file or extra configuration to work.

What do you think?


--
Moisés López Calderón
Vauxoo - OpenERP's Gold Partner
Mobile: (+521) 477-752-22-30
Office: (+52) 477-773-33-46
web: http://www.vauxoo.com
twitter: @vauxoo
           @moylop260           


Reference