Добрый день.
Сегодня по какой-то причине начали снова сыпаться ошибками сервисы Access и Accounting. В логах Accounting пусто, а вот в Access вот такие ошибки:
Цитата:
radius 12-06/12:20:58 ERROR [rdsLstnr-p-8-t-19] RadiusListenerWorker - Lock wait timeout for ContractRuntime[2287]@831287252
java.lang.RuntimeException: Lock wait timeout for ContractRuntime[2287]@831287252
at ru.bitel.common.util.CheckReentrantLock.tryLockEx(CheckReentrantLock.java:88)
at ru.bitel.bgbilling.kernel.contract.runtime.ContractRuntime.tryLockEx(ContractRuntime.java:173)
at ru.bitel.bgbilling.modules.inet.runtime.InetServRuntime.tryLockEx(InetServRuntime.java:526)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authenticationImpl(InetRadiusProcessor.java:608)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authentication(InetRadiusProcessor.java:469)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authentication(InetRadiusProcessor.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusSession.authentication(RadiusSession.java:115)
at ru.bitel.bgbilling.kernel.network.radius.RadiusSession.accessRequest(RadiusSession.java:92)
at ru.bitel.bgbilling.kernel.network.radius.RadiusProcessor.accessRequestImpl(RadiusProcessor.java:438)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.accessRequestImpl(InetRadiusProcessor.java:385)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.accessRequestImpl(InetRadiusProcessor.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusProcessor.accessRequest(RadiusProcessor.java:423)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusListenerWorker.accessRequest(InetRadiusListenerWorker.java:59)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusListenerWorker.accessRequest(InetRadiusListenerWorker.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusListenerWorker.processPacket(RadiusListenerWorker.java:216)
at ru.bitel.bgbilling.kernel.network.radius.RadiusListenerWorker.runImpl(RadiusListenerWorker.java:135)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:40)
Caused by: java.lang.RuntimeException: OwnerThread: Thread[rdsLstnr-p-8-t-14,5,main]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227)
at java.util.concurrent.FutureTask.get(FutureTask.java:91)
at ru.bitel.common.concurrent.ConcurrentUtils.awaitFutures(ConcurrentUtils.java:38)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.forceConnectionStop(InetConnectionManager.java:1056)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.closeDuplicateSession(InetConnectionManager.java:1006)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.checkDuplicateSession(InetConnectionManager.java:869)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.checkSessionCountLimit(InetConnectionManager.java:770)
at ru.bitel.bgbilling.modules.inet.access.Access.checkSessionCountLimitImpl(Access.java:406)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.checkSessionCountLimit(InetApplication.java:1094)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.authorizationOld(InetApplication.java:719)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.authorization(InetApplication.java:739)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authorization(InetRadiusProcessor.java:829)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authenticationImpl(InetRadiusProcessor.java:611)
... 17 more
radius 12-06/12:21:00 ERROR [rdsLstnr-p-8-t-3] RadiusListenerWorker - Lock wait timeout for ContractRuntime[2287]@831287252
java.lang.RuntimeException: Lock wait timeout for ContractRuntime[2287]@831287252
at ru.bitel.common.util.CheckReentrantLock.tryLockEx(CheckReentrantLock.java:88)
at ru.bitel.bgbilling.kernel.contract.runtime.ContractRuntime.tryLockEx(ContractRuntime.java:173)
at ru.bitel.bgbilling.modules.inet.runtime.InetServRuntime.tryLockEx(InetServRuntime.java:526)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authenticationImpl(InetRadiusProcessor.java:608)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authentication(InetRadiusProcessor.java:469)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authentication(InetRadiusProcessor.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusSession.authentication(RadiusSession.java:115)
at ru.bitel.bgbilling.kernel.network.radius.RadiusSession.accessRequest(RadiusSession.java:92)
at ru.bitel.bgbilling.kernel.network.radius.RadiusProcessor.accessRequestImpl(RadiusProcessor.java:438)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.accessRequestImpl(InetRadiusProcessor.java:385)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.accessRequestImpl(InetRadiusProcessor.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusProcessor.accessRequest(RadiusProcessor.java:423)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusListenerWorker.accessRequest(InetRadiusListenerWorker.java:59)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusListenerWorker.accessRequest(InetRadiusListenerWorker.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusListenerWorker.processPacket(RadiusListenerWorker.java:216)
at ru.bitel.bgbilling.kernel.network.radius.RadiusListenerWorker.runImpl(RadiusListenerWorker.java:135)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:40)
Caused by: java.lang.RuntimeException: OwnerThread: Thread[rdsLstnr-p-8-t-14,5,main]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227)
at java.util.concurrent.FutureTask.get(FutureTask.java:91)
at ru.bitel.common.concurrent.ConcurrentUtils.awaitFutures(ConcurrentUtils.java:38)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.forceConnectionStop(InetConnectionManager.java:1056)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.closeDuplicateSession(InetConnectionManager.java:1006)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.checkDuplicateSession(InetConnectionManager.java:869)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.checkSessionCountLimit(InetConnectionManager.java:770)
at ru.bitel.bgbilling.modules.inet.access.Access.checkSessionCountLimitImpl(Access.java:406)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.checkSessionCountLimit(InetApplication.java:1094)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.authorizationOld(InetApplication.java:719)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.authorization(InetApplication.java:739)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authorization(InetRadiusProcessor.java:829)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authenticationImpl(InetRadiusProcessor.java:611)
... 17 more
radius 12-06/12:21:06 ERROR [rdsLstnr-p-8-t-13] RadiusListenerWorker - Lock wait timeout for ContractRuntime[2287]@831287252
java.lang.RuntimeException: Lock wait timeout for ContractRuntime[2287]@831287252
at ru.bitel.common.util.CheckReentrantLock.tryLockEx(CheckReentrantLock.java:88)
at ru.bitel.bgbilling.kernel.contract.runtime.ContractRuntime.tryLockEx(ContractRuntime.java:173)
at ru.bitel.bgbilling.modules.inet.runtime.InetServRuntime.tryLockEx(InetServRuntime.java:526)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authenticationImpl(InetRadiusProcessor.java:608)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authentication(InetRadiusProcessor.java:469)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authentication(InetRadiusProcessor.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusSession.authentication(RadiusSession.java:115)
at ru.bitel.bgbilling.kernel.network.radius.RadiusSession.accessRequest(RadiusSession.java:92)
at ru.bitel.bgbilling.kernel.network.radius.RadiusProcessor.accessRequestImpl(RadiusProcessor.java:438)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.accessRequestImpl(InetRadiusProcessor.java:385)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.accessRequestImpl(InetRadiusProcessor.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusProcessor.accessRequest(RadiusProcessor.java:423)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusListenerWorker.accessRequest(InetRadiusListenerWorker.java:59)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusListenerWorker.accessRequest(InetRadiusListenerWorker.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusListenerWorker.processPacket(RadiusListenerWorker.java:216)
at ru.bitel.bgbilling.kernel.network.radius.RadiusListenerWorker.runImpl(RadiusListenerWorker.java:135)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:40)
Caused by: java.lang.RuntimeException: OwnerThread: Thread[rdsLstnr-p-8-t-14,5,main]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227)
at java.util.concurrent.FutureTask.get(FutureTask.java:91)
at ru.bitel.common.concurrent.ConcurrentUtils.awaitFutures(ConcurrentUtils.java:38)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.forceConnectionStop(InetConnectionManager.java:1056)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.closeDuplicateSession(InetConnectionManager.java:1006)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.checkDuplicateSession(InetConnectionManager.java:869)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.checkSessionCountLimit(InetConnectionManager.java:770)
at ru.bitel.bgbilling.modules.inet.access.Access.checkSessionCountLimitImpl(Access.java:406)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.checkSessionCountLimit(InetApplication.java:1094)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.authorizationOld(InetApplication.java:719)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.authorization(InetApplication.java:739)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authorization(InetRadiusProcessor.java:829)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authenticationImpl(InetRadiusProcessor.java:611)
... 17 more
radius 12-06/12:21:08 ERROR [rdsLstnr-p-8-t-14] InetRadiusProcessor - Timeout exceed!
ru.bitel.bgbilling.common.BGException: Timeout exceed!
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.forceConnectionStop(InetConnectionManager.java:1068)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.closeDuplicateSession(InetConnectionManager.java:1006)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.checkDuplicateSession(InetConnectionManager.java:869)
at ru.bitel.bgbilling.modules.inet.access.InetConnectionManager.checkSessionCountLimit(InetConnectionManager.java:770)
at ru.bitel.bgbilling.modules.inet.access.Access.checkSessionCountLimitImpl(Access.java:406)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.checkSessionCountLimit(InetApplication.java:1094)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.authorizationOld(InetApplication.java:719)
at ru.bitel.bgbilling.modules.inet.runtime.InetApplication.authorization(InetApplication.java:739)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authorization(InetRadiusProcessor.java:829)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authenticationImpl(InetRadiusProcessor.java:611)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authentication(InetRadiusProcessor.java:469)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.authentication(InetRadiusProcessor.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusSession.authentication(RadiusSession.java:115)
at ru.bitel.bgbilling.kernel.network.radius.RadiusSession.accessRequest(RadiusSession.java:92)
at ru.bitel.bgbilling.kernel.network.radius.RadiusProcessor.accessRequestImpl(RadiusProcessor.java:438)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.accessRequestImpl(InetRadiusProcessor.java:385)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusProcessor.accessRequestImpl(InetRadiusProcessor.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusProcessor.accessRequest(RadiusProcessor.java:423)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusListenerWorker.accessRequest(InetRadiusListenerWorker.java:59)
at ru.bitel.bgbilling.modules.inet.radius.InetRadiusListenerWorker.accessRequest(InetRadiusListenerWorker.java:1)
at ru.bitel.bgbilling.kernel.network.radius.RadiusListenerWorker.processPacket(RadiusListenerWorker.java:216)
at ru.bitel.bgbilling.kernel.network.radius.RadiusListenerWorker.runImpl(RadiusListenerWorker.java:135)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:40)
connection 12-06/12:21:30 ERROR [sa-p-13-t-5] EventWorker - java.util.concurrent.TimeoutException: Timeout waiting for future results will done. Possible answer not recieved.
ru.bitel.bgbilling.common.BGException: java.util.concurrent.TimeoutException: Timeout waiting for future results will done. Possible answer not recieved.
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.doTasks(AsyncEventWorker.java:129)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.runWorker(ServiceActivatorDeviceWorker.java:203)
at ru.bitel.bgbilling.kernel.event.EventWorker.internalRunWorker(EventWorker.java:139)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.runImpl(ServiceActivatorDeviceWorker.java:133)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:40)
Caused by: java.util.concurrent.TimeoutException: Timeout waiting for future results will done. Possible answer not recieved.
... 15 more
Память для сервисов поставил по 512 МБ.
Как прошлый раз просили сделал
Цитата:
Пробуйте вызвать
access.sh gc
несколько раз подряд
а потом сразу access_status.sh и покажите вывод.
Вот вывод раз:
Цитата:
Java Runtime: Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM [1.6.0_45] /usr/lib/jvm/java-6-oracle/jre
Runtime name:
22949@billing.ci.ukrpack.net Java endorsed dirs: /lib/endorsed:/usr/lib/jvm/java-6-oracle/lib/endorsed
OS: Linux 3.2.0-4-amd64 [amd64], file.encoding: UTF-8, user.name: root
Heap sizes: current=246080k free=244796k max=3651840k
Kernel version 5.2 build 1608 from 08.09.2014 17:34:58
Inet version 5.2 build 1339 from 08.09.2014 17:35:15
Radius:
accounting-requests per minute start: 0; stop: 0; update: 0
access-requests per minute accept: 2; reject: 75
ignore per minute access-request: 0; accounting-update: 0
Antispam ban count: 0; used per minute: 0
Started: 06.12.2014 12:20:07 Uptime: 0 d 00:11:21
Memory total: 331 153 408; max: 477 233 152; free: 311 216 920
Memory pools:
Non-heap memory[Code Cache]: max: 50 331 648; used: 3 329 856; peek: 3 340 352
Heap memory[PS Eden Space]: max: 153 026 560; used: 6 155 400; peek: 153 354 240
Heap memory[PS Survivor Space]: max: 12 910 592; used: 0; peek: 10 729 536
Heap memory[PS Old Gen]: max: 357 957 632; used: 13 781 088; peek: 14 968 816
Non-heap memory[PS Perm Gen]: max: 85 983 232; used: 33 119 856; peek: 33 381 728
Thread count: 76
Trees in cache: 0
Connections pool to Master status Idle: 7; Active: 0; maxActive: 300; maxIdle: 20
Вот вывод два:
Цитата:
Java Runtime: Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM [1.6.0_45] /usr/lib/jvm/java-6-oracle/jre
Runtime name:
23215@billing.ci.ukrpack.net Java endorsed dirs: /lib/endorsed:/usr/lib/jvm/java-6-oracle/lib/endorsed
OS: Linux 3.2.0-4-amd64 [amd64], file.encoding: UTF-8, user.name: root
Heap sizes: current=246080k free=244796k max=3651840k
Kernel version 5.2 build 1608 from 08.09.2014 17:34:58
Inet version 5.2 build 1339 from 08.09.2014 17:35:15
Radius:
accounting-requests per minute start: 1; stop: 2; update: 54
access-requests per minute accept: 0; reject: 0
ignore per minute access-request: 0; accounting-update: 0
Flow listener [null:2001]
flow packets in current minute: 0
Started: 06.12.2014 12:20:51 Uptime: 0 d 00:12:31
Memory total: 285 933 568; max: 477 233 152; free: 261 912 384
Memory pools:
Non-heap memory[Code Cache]: max: 50 331 648; used: 4 117 696; peek: 4 128 448
Heap memory[PS Eden Space]: max: 154 206 208; used: 3 568 176; peek: 131 465 216
Heap memory[PS Survivor Space]: max: 12 124 160; used: 0; peek: 10 387 872
Heap memory[PS Old Gen]: max: 357 957 632; used: 20 453 008; peek: 22 029 832
Non-heap memory[PS Perm Gen]: max: 85 983 232; used: 33 515 632; peek: 33 809 944
Thread count: 55
Trees in cache: 0
Connections pool to Master status Idle: 5; Active: 0; maxActive: 300; maxIdle: 20
Ну и по биллингу:
Цитата:
Информация о версии:
Клиент: вер. 5.2 сборка 1224 от 30.10.2014 20:39:46
os: Linux; java: Java HotSpot(TM) 64-Bit Server VM, v.1.6.0_45
Сервер: вер. 5.2 сборка 1615 от 30.10.2014 20:39:49
os: Linux; java: Java HotSpot(TM) 64-Bit Server VM, v.1.6.0_45
bill вер. 5.2 сборка 317 от 05.02.2014 19:22:08
card вер. 5.2 сборка 207 от 08.09.2014 17:34:48
dba вер. 5.2 сборка 151 от 01.10.2013 17:52:42
dialup вер. 5.2 сборка 385 от 01.10.2014 15:17:16
inet вер. 5.2 сборка 1350 от 06.11.2014 18:05:22
ipn вер. 5.2 сборка 252 от 15.04.2014 11:37:48
mps вер. 5.2 сборка 185 от 11.11.2014 17:34:42
npay вер. 5.2 сборка 209 от 06.11.2014 18:05:23
qiwi вер. 5.2 сборка 26 от 15.08.2013 15:15:39
reports вер. 5.2 сборка 199 от 23.07.2014 19:16:47
rscm вер. 5.2 сборка 173 от 15.04.2014 11:37:53
ru.bitel.bgbilling.plugins.documents вер. 5.2 сборка 151 от 13.10.2014 18:23:16
ru.bitel.bgbilling.plugins.helpdesk вер. 5.2 сборка 182 от 14.03.2014 19:49:38
В чем может быть причина такого поведения ?