Discussion:
[Maia-users] multi threading problem
Krylionov Igor
2015-11-18 10:30:55 UTC
Permalink
Hi group!

When the script is run: /usr/local/bin/perl
/usr/local/share/maia/scripts/process-quarantine.pl
I get regular error messages:
bayes: cannot open bayes databases /var/maiad/.spamassassin/bayes_* R/W:
lock failed: File exists
After the transfer spamassasin in debug mode, I found the following:
Nov 18 12:00:30.058 [22973] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 0 retries
Nov 18 12:00:30.590 [22965] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 2 retries
Nov 18 12:00:30.592 [22971] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 1 retries
Nov 18 12:00:30.756 [22970] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 2 retries
Nov 18 12:00:30.757 [22969] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 3 retries
Nov 18 12:00:30.759 [22972] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 1 retries
Nov 18 12:00:31.251 [22965] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 3 retries
Nov 18 12:00:31.416 [22970] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 3 retries
Nov 18 12:00:31.501 [22971] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 2 retries
Nov 18 12:00:31.919 [22973] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 1 retries
Nov 18 12:00:32.218 [22969] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 4 retries
Nov 18 12:00:32.230 [22972] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 2 retries
Nov 18 12:00:32.398 [22970] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 4 retries
Nov 18 12:00:32.510 [22965] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 4 retries
Nov 18 12:00:32.654 [22971] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 3 retries

Multiple processes are simultaneously attempting to write into the database!
I checked, runs one instance of the script - /usr/local/bin/perl
/usr/local/share/maia/scripts/process-quarantine.pl.

I tried using MySQL, received a similar error:
Nov 18 14:01:01.956 [25729] dbg: bayes: _put_tokens: SQL error: Deadlock
found when trying to get lock; try restarting transaction
Nov 18 14:01:01.981 [25725] dbg: bayes: _put_tokens: SQL error: Deadlock
found when trying to get lock; try restarting transaction
Nov 18 14:01:01.985 [25726] dbg: bayes: _put_tokens: SQL error: Deadlock
found when trying to get lock; try restarting transaction
Nov 18 14:01:01.999 [25726] dbg: bayes: _put_tokens: SQL error: Deadlock
found when trying to get lock; try restarting transaction
Nov 18 14:01:02.026 [25724] dbg: bayes: _put_tokens: SQL error: Deadlock
found when trying to get lock; try restarting transaction
Nov 18 14:01:02.051 [25725] dbg: bayes: _put_tokens: SQL error: Deadlock
found when trying to get lock; try restarting transaction

Has anyone solved this problem?

ps: info for developers - With version 5.x GifLib (FreeBSD 10.2) not
uses gifinter,
to not receive the error "warn: FuzzyOcr: Can not find executable for
gifinter",
you should use an outdated version 4.x GifLybwarn: FuzzyOcr: Cannot
find executable for gifinter.

Regards, Igor.
Blake Hudson
2015-11-18 16:55:28 UTC
Permalink
Igor, migrating to MySQL is the correct solution. However, I haven't ran
into a deadlock with a SQL based bayes db. I would suggest checking that
your bayes tables are using the INNODB engine type (which provides the
best write concurrency). I would also recommend that your
bayes_store_module and bayes_sql_dsn are set to MySQL and not just
generic sql in your spamassassin config. For example:
bayes_store_module Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn DBI:mysql:maiaDatabaseName

If you still run into deadlocks, you may have a performance problem
within MySQL and I would suggest focusing on that. MySQL provides
several template my.cnf files that provide generic suggestions for
machines that exceed the minimum CPU and RAM requirements of MySQL.
Using, or borrowing values from, these configs can greatly improve
performance.

--Blake
Post by Krylionov Igor
Hi group!
When the script is run: /usr/local/bin/perl
/usr/local/share/maia/scripts/process-quarantine.pl
bayes: cannot open bayes databases /var/maiad/.spamassassin/bayes_*
R/W: lock failed: File exists
Nov 18 12:00:30.058 [22973] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 0 retries
Nov 18 12:00:30.590 [22965] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 2 retries
Nov 18 12:00:30.592 [22971] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 1 retries
Nov 18 12:00:30.756 [22970] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 2 retries
Nov 18 12:00:30.757 [22969] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 3 retries
Nov 18 12:00:30.759 [22972] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 1 retries
Nov 18 12:00:31.251 [22965] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 3 retries
Nov 18 12:00:31.416 [22970] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 3 retries
Nov 18 12:00:31.501 [22971] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 2 retries
Nov 18 12:00:31.919 [22973] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 1 retries
Nov 18 12:00:32.218 [22969] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 4 retries
Nov 18 12:00:32.230 [22972] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 2 retries
Nov 18 12:00:32.398 [22970] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 4 retries
Nov 18 12:00:32.510 [22965] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 4 retries
Nov 18 12:00:32.654 [22971] dbg: locker: safe_lock: trying to get lock
on /var/maiad/.spamassassin/bayes with 3 retries
Multiple processes are simultaneously attempting to write into the database!
I checked, runs one instance of the script - /usr/local/bin/perl
/usr/local/share/maia/scripts/process-quarantine.pl.
Deadlock found when trying to get lock; try restarting transaction
Deadlock found when trying to get lock; try restarting transaction
Deadlock found when trying to get lock; try restarting transaction
Deadlock found when trying to get lock; try restarting transaction
Deadlock found when trying to get lock; try restarting transaction
Deadlock found when trying to get lock; try restarting transaction
Has anyone solved this problem?
ps: info for developers - With version 5.x GifLib (FreeBSD 10.2) not
uses gifinter,
to not receive the error "warn: FuzzyOcr: Can not find executable for
gifinter",
you should use an outdated version 4.x GifLybwarn: FuzzyOcr: Cannot
find executable for gifinter.
Regards, Igor.
_______________________________________________
Maia-users mailing list
http://www.renaissoft.com/cgi-bin/mailman/listinfo/maia-users
Loading...