Green icon for Endpoints in console turns grey and stops scanning

[theyzer]

We recently installed MWB Endpoint Protection and have noticed that some endpoints that were online in the console switch to offline in the console. This is despite the endpoints being actually on and used. Initially when this happened, I went to the endpoint and uninstalled MBW Endpoints, completed a restart, reinstalled it, and completed a second restart to fully install the MBW. This did reinstate the online status on the console but this was both an inconvenience to IT and the user as they are unable to use the machine during the process and some of the these machines are in constant use. The other problem is that the scans are not being done while the endpoint is showing as offline and these endpoints do not seem to come back into online status without my intervention. 

Why do some endpoints change to offline despite actually being online and actively used and how can I stop this from happening / fix this issue without having to resort to using the uninstall/restart/reinstall/restart process, which with 80+ computers is unrealistic as a solution.

[exile360]

Greetings,

I'm sorry that you have been impacted by this issue.  Hopefully someone from Malwarebytes Support will be able to determine a fix for the problem, but in the meantime I will see to it that the Product and QA teams are made aware of it so that they may investigate whether this is a one-off issue or a larger bug that requires attention from the Developers to resolve.

[Kalrand]

We've had a similar or possibly the same issue in the past, if you check the MBEndpointAgent service is it not started on the trouble endpoints? We found this mostly happens with Windows 10 clients. It appears to be related to when the Agent tries to start in the Startup process, it tries and gets chocked out, so what we've done is to change the service start mode to Automatic Delayed Start. Your mileage may vary but since doing this it's cut down our dropouts to maybe 1 or 2 a week, I think some endpoint updates reset the service start mode. Below is the procedure we use, script and deployment method. Let me know if this helps!

 

[theyzer]

Thank you @Kalrand for the info. I'm not sure this is the same issue as these computers aren't turned off, so it wouldn't be a matter of the Agent getting choked out in the Startup process. These computers stay on 24/7 generally speaking, so not sure the stop-gap would be of value in our case.

[Kalrand]

@theyzer Our workstations are on 24/7 as well, minus a scheduled reboot in the morning. The workstation is on however the Agent service doesn't start properly thus the Cloud endpoint shows offline.

[exile360]

If the devices are going into sleep or hibernation mode that also could be a factor, so that might be worth checking as well.

[theyzer]

4 minutes ago, exile360 said:

If the devices are going into sleep or hibernation mode that also could be a factor, so that might be worth checking as well.

Possibly @exile360, but wouldn't that affect all the computers in the same area - they all go into sleep over the weekend but most of the endpoints are fine. How would I check if that is an issue? I'm pretty new at this.

[exile360]

Possibly, but differences in hardware, drivers and software configurations might determine why some are and others are not affected, particularly if there are differences in performance since it could be as simple as a timing related issue (which would explain why, as another user mentioned, configuring the service to delayed start fixed it for them).

You could check Event Viewer to see if anything shows up there around the time that the systems were awakened from sleep/hibernation mode.  There also may be logs created by the client monitoring service, though I'm not certain on that point as I really specialize in the consumer product more than the business product, but one of the Support team members should know.

I had another thought as well.  We have seen intermittent startup issues on systems where Fast Startup is enabled, which it is by default on Windows 10 (and I believe Windows 8/8.1 as well).  Instructions on how to check as well as how to disable it can be found in the articles here as well as here.

[theyzer]

new information - So when the endpoint is goes offline in the console, I also cannot run a scan from the endpoint as the MWB icon is gone. I also discovered that the MWB service was not running - could this be why the icon is missing from Taskbar hidden icons area?

Also, our endpoints have gone offline (in the console, not in reality) in the midst of use as well in the middle of the night when they surely aren't in the process of being awakened. When I check the process' properties, all recovery options are set to 'restart the service', so what could be causing it to stop or not restart? I don't see how the Fast Startup or awakening from sleep/hibernation mode could be the cause as neither is occurring when the endpoint has gone offline.

Thanks for any help. At least I now know I can just restart the service and not uninstall/reinstall MWB, but would still like to prevent the changing to offline status when still online. Note, most endpoints register as offline when they are and resume online status when they resume being online without issue, but every so often an endpoint that didn't have this issue suddenly has it.

[exile360]

Yeah, the service is supposed to remain resident in the background.  It's the core of pretty much everything that the endpoint software does, so if it isn't running that would explain why they're showing as offline and why it isn't functioning.  It could be some kind of conflict or issue causing it to crash.  I'd suggest getting in touch with one of the Malwarebytes Support techs to have them take a look.  You also may be able to find some clue as to what's going on by checking the log created by the service which, at least in the consumer version (and I assume the Endpoint Protection version as well, but I'm not positive) is located at %PROGRAMDATA%\Malwarebytes\MBAMService\logs\MBAMSERVICE.LOG.  There should be a log there for every day since the client was installed which was created by the service.  It logs most of the activity for the endpoint and may include some kind of error(s) from around the time that the clients went offline that may help to diagnose whatever the problem might be.  Each entry is date/time coded at the start of each line so tracking down the entries from around the time of the endpoint going offline shouldn't be too difficult.

[exile360]

By the way, one more thing that may help would be to enable Event Log Data which is essentially verbose debug logging in the software.  There's an option to enable it in the Malwarebytes client under Settings>Application under Event Log Data.  Toggle that setting to On and allow it to collect data when the issue occurs then go ahead and shut it off again (you don't want it logging like that for too long as the logs can get very large with all those entries; its mainly for diagnostic purposes).  If you do, be sure to save the log to submit to Support for review as it may aid them in tracking down the source of the problem.

If you haven't reached out to Support directly yet, I'd recommend doing so.  They may be contacted by filling out the form on the bottom of this page to create a support ticket and include a description of the issue along with a link to this thread for their reference and in case they want to gather details from the other users being impacted by this issue (for the Dev and QA guys to try and fix it, assuming it is some kind of bug or conflict).