Console API to do large operations

Hello!

Current API version executes via web server an have several limitations, such as execution time.

We have about 1,5 million links in database and clear by cron via web API does not working.

Does YOURLS have the way to do long operations?

Hello @vitalyzhakov, thanks for your message and sorry for late reply.

I’m afraid it will require some extra step in the code on your side.
You can reuse the plugin you found and try to do segmentation on the database query.

@LeoColomb, hello!

I have patch, that works for me joshp23/YOURLS-Expiry/pull/18

This plugin adds way to run prune by CLI.

But if I extends official docker container, application is broken.
It because many operations, for example, YOURLS/docker-yourls/blob/master/docker-entrypoint.sh#L14-L17 executing in docker-entrypoint.
Can it be moved in Dockerfile for extending purpose?

Hello @vitalyzhakov.

That’s cool! :slightly_smiling_face:
I suggest either adding the plugin in the volume after YOURLS container creation, either putting the file in the /usr/src/yourls/user/plugins directory when extending via Dockerfile.

Hello, @LeoColomb!

Cron container must start cron daemon.

Your container is not stateless, it have logic in entrypoint
YOURLS/docker-yourls/blob/master/docker-entrypoint.sh#L14-L17

Can you move it to Dockerfile?

I try to start cron from CMD, with no success.

root@de6ff0e0ea04:/var/www/html/user/plugins/expiry/bin# php prune.inc.php 

Fatal error: Uncaught YOURLS\Exceptions\ConfigException: Cannot find config.php. Please read the readme.html to learn how to install YOURLS in /var/www/html/includes/Config/Config.php:92
Stack trace:
#0 /var/www/html/includes/load-yourls.php(18): YOURLS\Config\Config->find_config()
#1 /var/www/html/user/plugins/expiry/bin/prune.inc.php(13): require_once('/var/www/html/i...')
#2 {main}
  thrown in /var/www/html/includes/Config/Config.php on line 92

If you need help with docker logiс, I think, can help.

After about 1 hour script works, killed with error:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in
/var/www/html/includes/vendor/aura/sql/src/Rebuilder.php on line 275

Application have memory leaks
Снимок экрана от 2020-01-14 12-51-34

Increasing memory does not help

root@4b8fd99fbe26:/var/www/html/user/plugins/expiry/bin# time php prune.inc.php --signature=05e2685fc7

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/includes/vendor/aura/sql/src/Rebuilder.php on line 275

real	270m18.364s
user	3m8.880s
sys	1m34.724s