Enpoint Protection Concerns

[spnkzss]

There are quite a few issues with this software, that obviously, lots of people are upset about.  I'm not going to rant and rave, but I also will not spend my time troubleshooting what I think are basic functions of the software, especially since it has nothing to do with my environment.  Too many other people are having the same problems.  The only reason that I have not bailed on all of this is your reputation and how many times your software has bailed me out and how well the "core" software works when it works. 

I wanted to try and create a master list of sorts.  Certain things that hopefully we can get Malwarebytes to give us some more "official" answers too.  There are lots of hands in the pot, but I'm not getting any warm and fuzzies about what has been done.  I will gladly edit this list if someone feels we need to update it and if multiple people are seeing the same things.  I will also GLADLY update it if things get fixed.

 

1.)  Offline Endpoints.  Associated with one of the services not starting.

2.)  mb-clean not removing everything so a reinstall does not work Setup failed 0x80070643.  Edit1: 10:02EST. Have noticed that even though you run it, and reboot, the malwarebyte icon is still down by the clock.

3.)  Tons of Event Log errors

4.)  Memory Leak

5.)  Computers not running scans.  Even though it's online and I tell it to scan, the task fails.

5b.)  No easy well to tell if scans have ever run on machines unless you go to each one and dig in.  I know this is border line not CORE function, but if I don't have any faith that the software is doing what it's supposed to be doing, then I'm going to uninstall and find something else that helps me sleep better at night.  I have LOTS of machines that this software has been installed on for months that have NEVER run a scan.  Yes, I work hard not to let that keep me from sleeping, but....

 

Rob

Edited October 18, 2017 by spnkzss

[spnkzss]

Add to this list exclusions not working.

[IT_Guy]

If you have slow machines caused my MBAMSERVICE, check your C:\ProgramData\Malwarebytes\MBAMService\logs to see if you have MBAMSERVICE.LOG and a bunch of backup files, mine are all filled with the exact same entries:

10/19/17    " 09:30:32.523"    61740422    08ac    0940    ERROR    CHAMCTRL    CControlWatchdogDriver::GetRefCount    "ControlWatchdogDriver.cpp"    305    "GetRefCount (err = 2) = 4294967295"
10/19/17    " 09:30:32.524"    61740422    08ac    0940    ERROR    CHAMCTRL    CControlWatchdogDriver::DecrementRefCount    "ControlWatchdogDriver.cpp"    272    "Error getting driver RefCount - 2"
10/19/17    " 09:30:32.524"    61740422    08ac    0940    ERROR    CHAMCTRL    CControlWatchdogDriver::Remove    "ControlWatchdogDriver.cpp"    370    "Failed to remove reference"
10/19/17    " 09:30:32.524"    61740422    08ac    0940    ERROR    SPSDK    Uninstall    "SelfProtectionUser.cpp"    182    "SelfProtection driver failed to uninstall. LE=0."

 

It repeats this about 2-3x per second and is using 47% of the CPU and almost 100% of the HD.

[likwyd]

2 hours ago, IT_Guy said:

If you have slow machines caused my MBAMSERVICE, check your C:\ProgramData\Malwarebytes\MBAMService\logs to see if you have MBAMSERVICE.LOG and a bunch of backup files, mine are all filled with the exact same entries:

10/19/17    " 09:30:32.523"    61740422    08ac    0940    ERROR    CHAMCTRL    CControlWatchdogDriver::GetRefCount    "ControlWatchdogDriver.cpp"    305    "GetRefCount (err = 2) = 4294967295"
10/19/17    " 09:30:32.524"    61740422    08ac    0940    ERROR    CHAMCTRL    CControlWatchdogDriver::DecrementRefCount    "ControlWatchdogDriver.cpp"    272    "Error getting driver RefCount - 2"
10/19/17    " 09:30:32.524"    61740422    08ac    0940    ERROR    CHAMCTRL    CControlWatchdogDriver::Remove    "ControlWatchdogDriver.cpp"    370    "Failed to remove reference"
10/19/17    " 09:30:32.524"    61740422    08ac    0940    ERROR    SPSDK    Uninstall    "SelfProtectionUser.cpp"    182    "SelfProtection driver failed to uninstall. LE=0."

 

It repeats this about 2-3x per second and is using 47% of the CPU and almost 100% of the HD.

I looked at the log on one of my computers having this issue.  It looks like it is constantly trying to uninstall the SelfProtectionUser.cpp and remove the ControlWatchdogDriver.cpp which is failing every time.

[spnkzss]

Scans not running.  I am randomly going through my endpoints that ARE online and have up to date check in points, however if I go into each one individually, they have NO scans that have ever run on them and I know they are online during the scheduled daily scan.  I try and manually tell them to scan and under Tasks it stays pending for awhile then just goes failed.  Here I thought at least the ones that are online were doing something.  Apparently that isn't even the case.  Talk about lack of warm and fuzzies.  That's messed up.

To get all the ones that are offline online (all but one that is fighting me) I have had to run the clean up tool manually, reboot, then reinstall.  Looks like I need to do this on EVERY one of these clients for me to make sure it's working.  If I have to manually touch EVERY endpoint again, that may finally be my tipping point on finding another product.

[IT_Guy]

You and me both buddy.

I did this song and dance on an offline endpoint yesterday, got it back online, came back today and it's offline again. Everything seemed to be installed correctly and the Endpoint Service was set to start automatically, it just didn't. When I tried to start it manually after a couple minutes it complained it failed to start in a timely fashion. Rebooted the computer again and it came right up.

The inconsistency with which the endpoints come up and down does not reassure me AT ALL that the software is working correctly or protecting anything. In fact the only evidence I have that it IS doing something is on the machines that it keeps detecting the same PUP on, after quarantining everything it tells the user it is restarting to complete the cleanup, and then force-closes everything the users had open without saving. I've had a few users complain about lost documents due to this.

I think I might wait until one of these "updates" gets pushed out before I go around to every machine AGAIN and reinstall everything again.

 

BTW is there anyway of telling if the endpoints have updated anything? Are there version numbers changing somewhere? A change log? Or are we just supposed to believe them that things are being updated and fixed? Given the huge amount of Ransomware going around lately I really hope this is sorted out and working before one of my endpoints gets infected.

[KDawg]

I had sent instruction on clean installation and redeployment on our service ticket, please let me know if endpoints are stilling going offline after performing.

As well if you can describe the errors you are still experiencing, we are happy to assist to troubleshoot and resolve.

[spnkzss]

An update, moved from another thread.

My environment WAS the onsite server based solution.  I upgraded everyone to the cloud solution by  pushing the solution across the network using the Malwarebytes Discovery and Deployment Tool.  Now this part is speculation, but around the end of September (9/29ish), it seems to me that an update was pushed down to the clients from Malwarebytes.  This update did not update well on some of my clients.  I had a couple dozen offline, even though I knew they were online.  I also had quite a few that never did the scheduled scan even though they were definitely online during this time AND showed online in the portal.  Now, not speculation, what I did to resolve the issue, I went through every one of my clients in the portal and confirmed the last time they scanned (I have a daily scheduled scan setup).  I wrote down the ones that did not scan, especially the ones that still show up as online in the portal.  I also wrote down all the ones that were offline.  I then went to each machine on this new list, ran the mb-clean-3.1.0.1031 /cloud command.  Rebooted.  I then ran the mb-clean-3.1.0.1031 WITHOUT the /cloud.  1 in 5 needed to be rebooted again.  I believe this was finishing the clean up of the previous server based product.  I then reinstalled using the .msi for JUST the Endpoint product without the .NET.  They came back online.  I then told that client to scan and quarantine.  So far, all  have continued to be up and running and scanning appropriately for the last 4 days.

 

While I understand that this is what tech support basically tells you to do for everything, there is some more relevant information in this.  The fact that we upgraded the site.  The fact that I had to run the /cloud AND without the /cloud.  The fact that installing with it with the .msi versus the full exe with .NET.  I could not get the install to work correctly with the .NET exe, it HAD to be the installer WITHOUT the .NET.

[spnkzss]

Another KEY piece that I don't want missed is that even if the client was online, scans weren't happening.  If you look at the scan log it was empty, even though the client was online.  That was another thing that needed to be looked for.  The complete removal that I did above solved it.

I ran through the above sequence at one point with mb-clean-3.1.0.1027 and that DID NOT solve my problem.  Again, that is why I was as specific as I was.

I just believe that a lot of the problems that I talked about at the beginning of this thread are all related.

Edited October 23, 2017 by spnkzss

[KDawg]

@spnkzss thank you for the detailed update I am glad to hear it the endpoints are staying online for you.

We always recommend the latest version of the clean tool which can be found here:

https://downloads.malwarebytes.com/file/mb_clean

A note on the errors in event logs, we have made changes to some "informational" errors that are-non impact and will no longer appear with our next update.

 

Again please let me know of any errors or issues that persist

 

Many Thanks,

 

 

 

[IT_Guy]

1 minute ago, KDawg said:

@spnkzss thank you for the detailed update I am glad to hear it the endpoints are staying online for you.

We always recommend the latest version of the clean tool which can be found here:

https://downloads.malwarebytes.com/file/mb_clean

A note on the errors in event logs, we have made changes to some "informational" errors that are-non impact and will no longer appear with our next update.

 

Again please let me know of any errors or issues that persist

 

Many Thanks,

 

 

 

Case in point, KDawg, I know you're one of the only people from MWB that comments on here, but just re-hashing the same statement in every thread, you might as well put a sticky at the top of the forum "SOMETHING NOT WORKING? TRY UNINSTALLING IT AND REINSTALLING IT."

Can you point me to a change-log for Endpoint Protection? I'm curious as to the changes between v1.1.0.199 and v1.1.0.204, and all the versions in between.

Is there any way to verify the various components and plugins are at the right version level or do we just hope that everything is working?

[spnkzss]

7 minutes ago, KDawg said:

A note on the errors in event logs, we have made changes to some "informational" errors that are-non impact and will no longer appear with our next update.

 

Ok. Thank you for the update.  Do we have an idea of when that might happen?  Next week, next month, next year?

[KDawg]

@IT_Guy We do not currently have a formal change logs for EP however if you search support.malwarebytes.com for "platform update" you can see this upcoming and previous update notes

@spnkzss  Here is the information regarding the latest release scheduled for tonight https://support.malwarebytes.com/docs/DOC-2192

 

 

[CHall]

WOW!  I'm so glad I found this forum.  I thought I was alone and have been dealing with all the same issues that everyone else here seems to be dealing with as well.  Had twice opened a ticket with the basic, low-end Tech Support service which was a complete fail (try this, try that, etc... over days with hours between replies that got me nowhere).  I ended up discovering the "mb-clean" tool and after days of trial and error, fixed the issue myself of all endpoints going offline with no way to get them back.

Anyway, as others have also said, I don't want to rant on and on about that and other issues I'm having, as Malwarebytes has bailed me out of virus and malware catastrophes for years.  For free.  I like what they have here and I want to support them, but man, they really need to focus on getting this handful of issues resolved already.  Biggest ones being endpoints going offline and mbamservice getting stuck in some state using high CPU.  This one is the issue I keep battling and just can't figure out.  Everything will be fine, then one-by-one, I get complaints about slow PCs and upon troubleshooting, see that mbamservice.exe is stuck running and causing high CPU, slowing everything else down.  There's no rhyme or reason, or detectable pattern I can see, as to which PCs this happens to.  It's completely random.  But it continues to happen and I have to continually deal with it.  It's gotten to the point now where I have to just accept it as part of my job responsibility and wait patiently for the day they figure out why this is happening and fix it.

Btw, I think these couple of issues that seem to happening to everyone should be development's top priority, rather than unimportant things like changing the console appearance/theme which, in my opinion, is now unpleasant to look at.  I miss the color and screen action of the old cloud console.  Now it's all white and washed-out looking; and that shade of blue on the left......  sigh.

Lastly, I want to express my appreciation for KDawg who seems to be involved and communicates well here in the community.  It's so important as a user/customer to have staff/representatives listening to feedback and participating in the ongoing conversations.  Thanks.

[IT_Guy]

Your high cpu usage is from the mbamservice writing to your log file 100x per second of it trying to uninstall some stuff.

 

Goto c:\programdata\malwarebytes\mbamservice\logs

 

You probably have multiple copies of mbamservice.txt that are all constantly updating and filling up with the same data.

 

Uninstall and reinstall. Might fix it. Might not.

[CHall]

Thanks IT_Guy.  I had read your posts earlier regarding this and believe me, went right to that.  However, this doesn't seem to be the issue in my case.  My logs were quiet and fine.  I did go ahead and switch the Self-Protection switches OFF in my Test Policy though.  I've been testing scenarios in a Test Policy with a couple of endpoints I've placed in a Test Group.  Trust me, these issues have really bitten off a chunk of my IT support time and because of the unstable and unpredictable nature of the problems, they're very time-consuming to test and troubleshoot.  Sometimes it takes days for a particular policy test to prove no-good and I have to try something else.  This is why I have a carefully established test system in place.  Now that I've found this forum, I can gather info from others, as well as share my own findings, hopefully helping the developers find solutions.

[IT_Guy]

If that's not doing it, open up resource monitor through the task manager and explore each section, CPU/Memory/HDD/Network, under the HDD section I found it writing to that log 10MB/s, maybe you can find something somewhere else that is using all your resources.

 

Since installing Endpoint Protection on our systems I have gone from a couple hours a week of hands-on IT work to almost all-day everyday since about September 20th.