Здравствуйте.
Вопрос в следующем - могут ли одновременно работать несколько Access/Accounting серверов с одинаковым rootDeviceId?
Суть в следующем: одна группа Access/Accounting принимает запросы авторизации/аккаунтинга, предположим, одной группы клиентов,
вторая, параллельно, обслуживает другую группу.
Находил темы про резервирование и проблемы с ним в части аккаунтинга, но это не совсем то.
Пока, не требуется именно резервирование, нужно разделить обработку RADIUS-запросов от групп клиентов.
Запускаю вторую группу Access/Accounting серверов, естественно app.id у всех приложений разный.
Получаю, на боевом сервере в accounting/error.log:
Код:
accounting 08-23/13:04:35 ERROR [accwrkr-2-p-12-t-1] SessionFlushingManager - Cannot find session for update with id=2610999, connectionId=2730751, sessionStart=2017-08-23 00:00:00.0 in DB!
accounting 08-23/13:04:35 ERROR [accwrkr-2-p-12-t-1] SessionFlushingManager - Cannot find connection with id=2730751 in DB while flush!
accounting 08-23/13:04:35 ERROR [accwrkr-2-p-12-t-1] SessionFlushingManager - Cannot find session for update with id=2610997, connectionId=2730750, sessionStart=2017-08-23 00:00:00.0 in DB!
accounting 08-23/13:04:35 ERROR [accwrkr-2-p-12-t-1] SessionFlushingManager - Cannot find connection with id=2730750 in DB while flush!
accounting 08-23/13:05:01 ERROR [accwrkr-3-p-13-t-1] SessionFlushingManager - Cannot find session for finish with id=2610999, connectionId=2730751, sessionStart=2017-08-23 00:00:00.0 in DB!
accounting 08-23/13:05:01 ERROR [accwrkr-3-p-13-t-1] SessionFlushingManager - Cannot find connection with id=2730751 in DB while flush/finish!
accounting 08-23/13:05:01 ERROR [accwrkr-3-p-13-t-1] SessionFlushingManager - Cannot find session for finish with id=2610997, connectionId=2730750, sessionStart=2017-08-23 00:00:00.0 in DB!
accounting 08-23/13:05:01 ERROR [accwrkr-3-p-13-t-1] SessionFlushingManager - Cannot find connection with id=2730750 in DB while flush/finish!
mq 08-23/13:09:48 ERROR [event-proc-p-2-t-1] Consumer - com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
ru.bitel.bgbilling.common.BGException: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at ru.bitel.bgbilling.modules.inet.api.server.bean.InetConnectionDao.closeConnection(InetConnectionDao.java:484)
at ru.bitel.bgbilling.modules.inet.accounting.Accounting.connectionStop(Accounting.java:1653)
at ru.bitel.bgbilling.modules.inet.radius.InetNas.stopConnection(InetNas.java:845)
at ru.bitel.bgbilling.modules.inet.radius.InetNas.stopConnection(InetNas.java:793)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.notify(InetRadiusProcessor.java:2599)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.notify(InetRadiusProcessor.java:86)
at ru.bitel.bgbilling.kernel.event.AbstractConsumer.notify(AbstractConsumer.java:353)
at ru.bitel.bgbilling.kernel.event.Consumer.notify(Consumer.java:19)
at ru.bitel.bgbilling.kernel.event.Consumer.onMessage0(Consumer.java:91)
at ru.bitel.bgbilling.kernel.event.Consumer$EventListenerRunnable.runImpl(Consumer.java:51)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:46)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.Util.getInstance(Util.java:387)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073)
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2009)
at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5094)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1994)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
at ru.bitel.bgbilling.modules.inet.api.server.bean.InetConnectionDao.closeConnection(InetConnectionDao.java:477)
... 14 more
на втором сервере access/error.log
Код:
connection 08-23/13:11:42 ERROR [sa-p-12-t-93] AcknowledgeConsumer - java.util.concurrent.TimeoutException: Timeout waiting task done (or answer)
ru.bitel.bgbilling.common.BGException: java.util.concurrent.TimeoutException: Timeout waiting task done (or answer)
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.internalDoTaskImpl(AsyncEventWorker.java:276)
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.notify(AsyncEventWorker.java:201)
at ru.bitel.bgbilling.kernel.event.AcknowledgeConsumer.onMessage0(AcknowledgeConsumer.java:83)
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.internalDoTask(AsyncEventWorker.java:176)
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.doTasks(AsyncEventWorker.java:109)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.runWorker(ServiceActivatorDeviceWorker.java:246)
at ru.bitel.bgbilling.kernel.event.EventWorker.internalRunWorker(EventWorker.java:139)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.runImpl(ServiceActivatorDeviceWorker.java:152)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:46)
Caused by: java.util.concurrent.TimeoutException: Timeout waiting task done (or answer)
... 17 more
...
connection 08-23/13:29:41 ERROR [sa-p-12-t-52] InetSaStateModifyEvent - Connection not found with id=2738404
connection 08-23/13:29:41 ERROR [sa-p-12-t-52] InetSaStateModifyEvent - Connection not found with id=2738429
connection 08-23/13:29:41 ERROR [sa-p-12-t-52] InetSaStateModifyEvent - Connection not found with id=2738431
connection 08-23/13:29:41 ERROR [sa-p-12-t-52] InetSaStateModifyEvent - Connection not found with id=2738380
connection 08-23/13:29:41 ERROR [sa-p-12-t-52] InetSaStateModifyEvent - Connection not found with id=2738394
connection 08-23/13:29:41 ERROR [sa-p-12-t-52] InetSaStateModifyEvent - Connection not found with id=2738387
connection 08-23/13:29:41 ERROR [sa-p-12-t-52] InetSaStateModifyEvent - Connection not found with id=2738473
connection 08-23/13:29:41 ERROR [sa-p-12-t-52] InetSaStateModifyEvent - Connection not found with id=2738466
...
connection 08-23/13:12:42 ERROR [sa-p-12-t-24] ServiceActivatorDeviceWorker - [Ljava.lang.Object;@ad59779
accounting/error.log
Код:
accounting 08-23/13:19:59 ERROR [accwrkr-3-p-13-t-1] SessionFlushingManager - Cannot find connection with id=2739853 in DB while flush/finish!
accounting 08-23/13:20:12 ERROR [accwrkr-2-p-12-t-1] SessionFlushingManager - Cannot find session for update with id=0, connectionId=2739853, sessionStart=2017-08-23 12:54:52.0 in DB!
accounting 08-23/13:20:12 ERROR [accwrkr-2-p-12-t-1] SessionFlushingManager - Cannot find connection with id=2739853 in DB while flush!
accounting 08-23/13:20:19 ERROR [accwrkr-3-p-13-t-1] WorkerTask - com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
ru.bitel.bgbilling.common.BGException: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at ru.bitel.bgbilling.modules.inet.accounting.SessionFlushingManager.flush(SessionFlushingManager.java:460)
at ru.bitel.bgbilling.modules.inet.accounting.Accounting.sessionFinish(Accounting.java:1776)
at ru.bitel.bgbilling.modules.inet.accounting.InetConnectionCallRuntime.finish(InetConnectionCallRuntime.java:426)
at ru.bitel.bgbilling.modules.inet.accounting.InetConnectionCallRuntime.tryFinish(InetConnectionCallRuntime.java:369)
at ru.bitel.bgbilling.modules.inet.accounting.worker.SessionFinishWorker.runImpl(SessionFinishWorker.java:112)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:46)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.Util.getInstance(Util.java:387)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073)
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2009)
at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5094)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1994)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
at ru.bitel.bgbilling.modules.inet.accounting.SessionFlushingManager.merge(SessionFlushingManager.java:909)
at ru.bitel.bgbilling.modules.inet.accounting.SessionFlushingManager.flush(SessionFlushingManager.java:385)
... 13 more
accounting 08-23/13:20:32 ERROR [accwrkr-2-p-12-t-1] SessionFlushingManager - Cannot find session for update with id=0, connectionId=2739853, sessionStart=2017-08-23 12:54:52.0 in DB!
Извините за простыни, не нашёл как под кат убрать.
Может я что не донастроил? При чём, на второй access/accounting RADIUS запросы пока совсем не попадают.
Я так понимаю они общаются через ActiveMQ, но deadlock-и то откуда? Не могут разделить - кто будет выполнять?
Или, всё же, ошибка в настройке?