I use Digital Ocean’s App Platform for a docker run application. I also use
celery for asynchronous processing and cronjobs inside of this application. For months it worked without issue, and then one day my logs started filling up things like
Errno 38 above, all with
I wasted a lot of time troubleshooting so now let me save you time.
Digital Ocean simply doesn’t support multiprocessing on their app platform.
At least as far as I can tell, this thread finally made it click for me: https://github.com/Koed00/django-q/issues/522
The ultimate fix was to run a separate instance of the app in docker on a VM instead of the app platform (which yes, defeats the purpose of having the IaaS app platform in the first place).
Hopefully I got enough keywords in here that you, dear reader, can save yourself a day of development work.
CRITICAL/MainProcess] Unrecoverable error: OSError(38, 'Function not implemented') Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/celery/worker/worker.py", line 202, in start self.blueprint.start(self) File "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/usr/local/lib/python3.10/site-packages/celery/bootsteps.py", line 365, in start return self.obj.start() self.on_start() P = self._pool = Pool(processes=self.limit, File "/usr/local/lib/python3.10/site-packages/celery/concurrency/asynpool.py", line 464, in __init__ super().__init__(processes, *args, **kwargs) File "/usr/local/lib/python3.10/site-packages/billiard/pool.py", line 1045, in __init__ self._create_worker_process(i) File "/usr/local/lib/python3.10/site-packages/celery/concurrency/asynpool.py", line 482, in _create_worker_process return super()._create_worker_process(i) File "/usr/local/lib/python3.10/site-packages/billiard/pool.py", line 1141, in _create_worker_process on_ready_counter = self._ctx.Value('i') File "/usr/local/lib/python3.10/site-packages/billiard/context.py", line 179, in Value return Value(typecode_or_type, *args, lock=lock, File "/usr/local/lib/python3.10/site-packages/billiard/sharedctypes.py", line 81, in Value lock = ctx.RLock() File "/usr/local/lib/python3.10/site-packages/billiard/context.py", line 108, in RLock return RLock(ctx=self.get_context()) SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1, ctx=ctx) File "/usr/local/lib/python3.10/site-packages/billiard/synchronize.py", line 70, in __init__ sl = self._semlock = _billiard.SemLock( OSError: [Errno 38] Function not implemented