Availability |
Odoo Online
Odoo.sh
On Premise
|
Odoo Apps Dependencies |
Discuss (mail)
|
Lines of code | 1671 |
Technical Name |
celery |
License | LGPL-3 |
Website | https://www.novacode.nl |
Versions | 10.0 11.0 12.0 14.0 13.0 9.0 |
Celery: Distributed Task Queue
Odoo integration of the Python #1 asynchronous task queue
- Put model-methods on the Celery Task Queue.
- Monitor and manage the Task Queue in Odoo.
- All Exceptions are catched and available as State=Failure with Exception message/trace shown.
- Requeue of Failed and Pending (stale) tasks.
- No complex installation and setup requirements are needed, except the Celery setup.
Check out the links below, for more info about Celery.
Celery website
http://celeryproject.org
Celery installation
http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html
Put a model-method on the Celery Task Queue
Just Python code to call from an Odoo model
Example
Model and method:
- model: "celery.example"
- method: "schedule_import"
Celery Options:
Shall be provided to kwargs
of the call_task
method
celery = { 'queue': 'high.priority', 'countdown': 5, 'retry': True, 'retry_countdown_setting': 'MUL_RETRIES_SECS', 'retry_countdown_multiply_retries_seconds': 60, 'retry_policy': { 'interval_start': 30 } }
Calling the task:
self.env["celery.task"].call_task("celery.example", "schedule_import", celery=celery)
Celery Options
All Celery options are optional (not required).
Option | Description | Celery Documentation |
---|---|---|
queue |
Name of the Celery/worker queue, the task shall be routed to. | Routing Tasks: http://docs.celeryproject.org/en/latest/userguide/routing.html |
countdown |
The countdown is a shortcut to set ETA by seconds into the future. | ETA and Countdown: http://docs.celeryproject.org/en/latest/userguide/calling.html#eta-and-countdown |
retry |
Set to True to enable the retry of sending task messages. |
Message Sending Retry: http://docs.celeryproject.org/en/latest/userguide/calling.html#message-sending-retry |
retry_countdown_setting |
Specify whether and how to increase the Retry Countdown by:
|
This is a custom option which affects the Celery countdown option. |
retry_countdown_add_seconds |
Specify the seconds (integer field) to add to the Retry Countdown. | This is a custom option which affects the Celery countdown option. |
retry_countdown_multiply_retries_seconds |
Specify the seconds (integer field) to multiply with request retries, which becomes the Retry Countdown. | This is a custom option which affects the Celery countdown option. |
retry_policy |
Options when retrying publishing a task message in the case of connection loss or other connection errors. | Message Sending Retry: http://docs.celeryproject.org/en/latest/userguide/calling.html#message-sending-retry |
Extra kwargs
Extra kwargs
are optional (not required).
Kwarg | Description |
---|---|
transaction_strategy |
Specifies when the task shall apply (ORM create and send to Celery MQ):
|
Monitor and control the Celery Task Queue
List of queued tasks
Task Failure info
Tasks waiting for Retry
Task waiting for Retry with Failure info
Requeue a Failed Task
Requeue multiple Failed Tasks
Installation and configuration
Celery and Message broker
Celery Installation
http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html
Odoo configuration
All you need is to determine the Odoo Celery user and setup the credentials for XML-RPC authentication.
This enables the Celery process to authenticate in Odoo, by the XML-RPC server.
To support different kind of deployment and build tools, the credentials can either be setup as:
-
(OS) Environment variables of the user running the Odoo server process:
ODOO_CELERY_USER
andODOO_CELERY_PASSWORD
-
Put in the
odoo.conf
file, e.g:celery_user = Odoo-User celery_password = Odoo-User-Password
See
example.odoo.conf
, visit link:https://github.com/novacode-nl/odoo-celery/blob/14.0/celery/example.odoo.conf
-
Put in the
odoo.conf
file, under section[celery]
e.g:[celery] user = Odoo-User password = Odoo-User-Password
See
example_section_celery.odoo.conf
, visit link:https://github.com/novacode-nl/odoo-celery/blob/14.0/celery/example_section_celery.odoo.conf
Start the Celery (Odoo) worker server
Check the Celery Documentation for more ways to start and manage the server/proces. E.g. by Supervisor
The Celery (Odoo) worker => Python file odoo.py
, which is located directly under the "celery" module directory.
Start the worker (default/celery queue) on command line, whereas "odoo" references to the Python file:
# celery -A odoo worker --loglevel=info
Start a worker as a named queue:
# celery -A odoo worker --loglevel=info -Q high.priority -n high.priority
"Celery Example" module
Demo of 2 implemented example tasks, queued by 2 buttons
Check out the "Celery Example" module
- After installation, go to the menu "Celery / Example Task".
- Click button(s) "Queue create Line" shown in screensshot, which puts a task on the queue.
- Check the queue (menu: Celery / Tasks). Check the form of the Example record - new Lines had been created.
Celery Examples module
https://apps.odoo.com/apps/modules/11.0/celery_example/
Changelog
0.23
- Fix cancellation of stucked tasks(s) wasn't possible.
- Fix requeue of stucked tasks(s) wasn't possible.
0.22
-
Upon processing task
rpc_run_task
(method), only update the related record fields (res_model
,res_ids
) if these have a value set in the method its result.
This enables the API to store these related record fields upfront, when the task is created.
0.21
- Some security hardening: support inactive Celery User, which refuses regular (web) authentication/login.
0.20
-
[FIX] In
call_task()
the search ofcelery.task.setting
results inAccess Denied by ACLs for operation: read, model: celery.task.setting
.
Solved by calling thecelery.task.setting
search()
withsudo()
.
0.19
- [FIX] Related record: res_ids (now ListSerialized field).
- Also change kwargs field to sensible class(name): TaskSerialized to KwargsSerialized.
0.18
- Configurable (database) transaction strategy, when the task shall apply (ORM create and send to Celery MQ).
- From now on - by default a task shall apply after commit of the main/caller transaction, instead of immediately in the main/caller transaction.
0.17
- Task scheduling added - being able to schedule tasks in a specified time interval or certain day(s) of the week.
- A new task state - Scheduled, is handled by an Odoo cron job - "Celery: Handle Scheduled Tasks".
0.16
- Configurable celery queues added to task settings.
0.15
- Scheduled (cron) cleanup of tasks - with optionally specifying/overriding: (1) timedelta (days=90, hours, minutes, seconds), (2) states to keep and (3) batch_size=100.
- Create database index for the State Time (state_date) field.
0.14
- Create database index for the Reference field (ref).
0.13
- Get XML-RPC URL from ir.config.parameter (celery.celery_base_url) by Settings.
0.12
- Also support to get the Celery (Odoo) user and password from the odoo.conf file, under section [options] too.
0.11
- Put task into a new state "Retrying", right before the Retry starts.
0.10
- Renamed task retry settings: 'MUL_RETR' to 'MUL_RETRIES', 'MUL_RETR_SECS' to 'MUL_RETRIES_SECS'.
0.9
-
Hide (Odoo) password in the "retry" payload used by the MQ broker for XML-RPC authentication.
GitHub issue:https://github.com/novacode-nl/odoo-celery/issues/17
0.8
- Fix task retry countdown/interval ignored. For more info see
https://github.com/novacode-nl/odoo-celery/issues/14
- Add task retry countdown settings, to increase countdown during retries ('ADD_SECS', 'MUL_RETRY', 'MUL_RETRY_SECS').
- Search view of stuck tasks: (1) Add/show field
reference
(2) search-view with filters and grouping. - Track task changes of fields
model, method, handle_stuck
- Disable create/copy in the task form-view.
0.7
- Task Reference, which serves:
- Easier searching for tasks by Reference.
- Check (ORM query) before
call_task()
call, to prevent redundant creation of recurring Pending tasks.
0.6
Introduction of "Stuck" tasks. A stuck task could be caused, for example by: a stopped/restarted Celery worker, message broker, or server.- Settings: to specify when a task (model, method) is Stuck after seconds from Started or Retry (states).
- Stuck Tasks Report: which shows Stuck (not completed) tasks.
- Cron or manually put tasks in "Stuck" state.
- Ability to cancel (Pending or Stuck) tasks, which never completed.
- Track and messaging about state change. Chatter on form view.
0.5
- Routing tasks to specific (named) queues.
0.4
- FIX: Store task state (Started, Retry) before execution.
0.3
-
Hide (Odoo) password in payload used by the broker for XML-RPC authentication.
GitHub issue:https://github.com/novacode-nl/odoo-celery/issues/4
0.2
- Task state information.
0.1
- Initial version.
Please log in to comment on this module