После замены маршрутизатора, на котором висел биллинг, добавлении этого машрутизатора в устройства и значительных изменений в иерархии устройств, перестал работать обработчик активации сервисов. Что именно повлияло на это - мне не понятно. Пример ошибки:
Код:
mq 08-25/17:53:11 INFO [event-proc-p-2-t-1] InetServRuntimeMap - Caught event: Event[ru.bitel.bgbilling.modules.inet.api.server.event.InetServModifiedEvent] moduleId: 1; pluginId: no; cid: 14; scid: -1; userId: 1; timestamp: 1440514391661
mq 08-25/17:53:11 INFO [event-proc-p-2-t-1] InetServRuntimeMap - Reload InetServRuntime: 74
mq 08-25/17:53:11 INFO [event-proc-p-2-t-1] InetServRuntimeMap - Removing InetServRuntime: 74
mq 08-25/17:53:11 INFO [event-proc-p-2-t-1] InetServRuntimeMap - Adding InetServRuntime: 74
mq 08-25/17:53:11 INFO [event-proc-p-2-t-1] InetServRuntimeMap - ContractId: 14; status: 0; servId: 74
10.10.10.10
Options [] TariffModuleTreeSet []
Device state: -1; optionSet:
connection 08-25/17:53:11 INFO [sa-p-12-t-63] ServiceActivatorSet - Connecting to device
connection 08-25/17:53:11 INFO [sa-p-12-t-63] ServiceActivatorSet - Skip device:10 with disabled ServiceActivator
connection 08-25/17:53:11 INFO [sa-p-12-t-63] ServiceActivatorDeviceWorker - Do task deviceId: 62; Event[ru.bitel.bgbilling.modules.inet.access.sa.event.InetSaServModifyEvent] moduleId: 1; pluginId: no; cid: 14; scid: -1; userId: -1; oldInetServ: 10.10.10.10; newInetServ: null; newInetServState: -1; newInetServOptionList: ; timestamp: 1440514391676
connection 08-25/17:53:11 INFO [sa-p-12-t-63] InetSaServModifyEvent - InetServ already in deleted deviceState
connection 08-25/17:53:16 INFO [sa-p-12-t-63] ServiceActivatorSet - Disconnecting from device
mq 08-25/17:53:33 INFO [event-proc-p-2-t-1] InetServRuntimeMap - Caught event: Event[ru.bitel.bgbilling.modules.inet.api.server.event.InetServModifiedEvent] moduleId: 1; pluginId: no; cid: 14; scid: -1; userId: 1; timestamp: 1440514412957
mq 08-25/17:53:33 INFO [event-proc-p-2-t-1] InetServRuntimeMap - Reload InetServRuntime: 74
mq 08-25/17:53:33 INFO [event-proc-p-2-t-1] InetServRuntimeMap - Removing InetServRuntime: 74
mq 08-25/17:53:33 INFO [event-proc-p-2-t-1] InetServRuntimeMap - Adding InetServRuntime: 74
mq 08-25/17:53:33 INFO [event-proc-p-2-t-1] InetServRuntimeMap - ContractId: 14; status: 0; servId: 74
10.10.10.10
Options [] TariffModuleTreeSet []
Device state: -1; optionSet:
connection 08-25/17:53:33 INFO [sa-p-12-t-70] ServiceActivatorSet - Connecting to device
connection 08-25/17:53:33 INFO [sa-p-12-t-70] ServiceActivatorSet - Skip device:10 with disabled ServiceActivator
connection 08-25/17:53:33 INFO [sa-p-12-t-70] ServiceActivatorDeviceWorker - Do task deviceId: 62; Event[ru.bitel.bgbilling.modules.inet.access.sa.event.InetSaServModifyEvent] moduleId: 1; pluginId: no; cid: 14; scid: -1; userId: -1; oldInetServ: null; newInetServ: 10.10.10.10; newInetServState: 1; newInetServOptionList: ; timestamp: 1440514412968
connection 08-25/17:53:33 INFO [sa-p-12-t-70] InetApplication - OptionSet: []
connection 08-25/17:53:33 INFO [sa-p-12-t-70] ServiceActivatorDeviceWorker - Command result event: ServiceActivatorEvent type=1; inetServId: 74; call: false; oldState: -1; newState: 1; oldOptionSet: ; newOptionSet:
connection 08-25/17:53:33 INFO [sa-p-12-t-70] ServiceActivatorDeviceWorker - Processing deviceId:62; command ServiceActivatorEvent type=1; inetServId: 74; call: false; oldState: -1; newState: 1; oldOptionSet: ; newOptionSet:
connection 08-25/17:53:33 INFO [sa-p-12-t-70] ServiceActivatorSet - Invoking serviceCreate
connection 08-25/17:53:36 ERROR [sa-p-12-t-70] AcknowledgeConsumer - Error on device:62 - java.net.NoRouteToHostException: No route to host
ru.bitel.bgbilling.common.BGException: Error on device:62 - java.net.NoRouteToHostException: No route to host
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.doTask(ServiceActivatorDeviceWorker.java:495)
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.internalDoTaskImpl(AsyncEventWorker.java:211)
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.notify(AsyncEventWorker.java:201)
at ru.bitel.bgbilling.kernel.event.AcknowledgeConsumer.onMessage0(AcknowledgeConsumer.java:83)
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.internalDoTask(AsyncEventWorker.java:176)
at ru.bitel.bgbilling.kernel.event.AsyncEventWorker.doTasks(AsyncEventWorker.java:109)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.runWorker(ServiceActivatorDeviceWorker.java:245)
at ru.bitel.bgbilling.kernel.event.EventWorker.internalRunWorker(EventWorker.java:139)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.runImpl(ServiceActivatorDeviceWorker.java:151)
at ru.bitel.common.worker.WorkerTask.run(WorkerTask.java:86)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at ru.bitel.common.worker.WorkerThread.run(WorkerThread.java:46)
Caused by: java.net.NoRouteToHostException: No route to host
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at bitel.billing.server.util.telnet.TelnetSession.initConnection(TelnetSession.java:44)
at bitel.billing.server.util.terminal.AbstractTerminalSession.connect(AbstractTerminalSession.java:55)
at ru.bitel.bgbilling.modules.inet.dyn.device.terminal.TelnetServiceActivator.connectImpl(TelnetServiceActivator.java:80)
at ru.bitel.bgbilling.modules.inet.dyn.device.terminal.TelnetServiceActivator.getSession(TelnetServiceActivator.java:110)
at ru.bitel.bgbilling.modules.inet.dyn.device.terminal.TelnetServiceActivator.executeCommand(TelnetServiceActivator.java:145)
at ru.bitel.bgbilling.modules.inet.dyn.device.terminal.AbstractTerminalServiceActivator.executeCommands(AbstractTerminalServiceActivator.java:852)
at ru.bitel.bgbilling.modules.inet.dyn.device.terminal.AbstractTerminalServiceActivator.serviceCreate(AbstractTerminalServiceActivator.java:900)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorSet.serviceCreate(ServiceActivatorSet.java:386)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.doCommand(ServiceActivatorDeviceWorker.java:555)
at ru.bitel.bgbilling.modules.inet.access.sa.ServiceActivatorDeviceWorker.doTask(ServiceActivatorDeviceWorker.java:491)
... 17 more
connection 08-25/17:53:36 INFO [sa-p-12-t-70] ServiceActivatorSet - Disconnecting from device
connection 08-25/17:53:36 INFO [sa-p-12-t-70] ServiceActivatorDeviceWorker - Execution error - waiting 60000ms for next try.
Пример конфига устройства (команды активации сервиса убраны)
Код:
connector.http=*:8080
connector.https.sslEnabledProtocols=SSLv2,SSLv3,TLSv1,TLSv1.1,TLSv1.2,SSLv2Hello
context.path=
db.maxActive=300
db.maxIdle=20
db.validationTimeout=10
deviceId=62
dhcp.connection.closeOnNew=1
dhcp.disable.mode=1
dhcp.option.leaseTime=86400
dhcp.option.serverIdentifier=172.16.16.1
dhcp.option82.agentRemoteId.code=2
dhcp.option82.agentRemoteId.length=6
dhcp.option82.agentRemoteId.position=0
dhcp.option82.agentRemoteId.type=0
dhcp.option82.interfaceId.code=1
dhcp.option82.interfaceId.length=2
dhcp.option82.interfaceId.position=3
dhcp.option82.removeHeader=2
dhcp.relay.deviceTypeIds=3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
dhcp.servSearchMode=1
flow.agent.link=4:-1
flow.agent.type=netflow9
ip.resource.categoryId=2,3
port.admin=2005
sa.command.connect=
sa.command.disconnect=
sa.command.exit=
sa.command.timeout=60000
sa.endSequence=#
sa.lazyConnect=1
vlan.resource.category=4
В данном случае используется telnetserviceactivator. Биллинг пытается зайти на устройство с адресом 172.16.7.221 на 23 порт и сразу же выдает ошибку. Никаких пакетов в tcpdump на этот хост не посылается. С самого сервера устройство пингуется. На него можно зайти по телнету без каких-либо проблем. Дефолтный маршрут прописан и виден в системе. Началось это еще до сегодняшнего обновления (когда поломался option82).