Jump to content
nccomp

MBAM 3.0 and av-comparatives.org

Recommended Posts

Pedro, 

I was given permission by MRG Effitas to post to post this below and they also mentioned to me that it would be nice if Malwarebytes had contacted MRG-Effitas since they are not an AMTSO member https://www.amtso.org 

From: Sveta Miladinov [sveta.miladinov@mrg-effitas.com]
Sent: Thursday, March 1, 2018 11:21 AM
Cc: Chris Pickard
Subject: Malwarebytes comments about MRG

Our testing methodologies evolve all the time and adapt to latest threats as not all threats can be tested using “10 year old methodologies”. We test against multiple highly advanced malware types that are extremely difficult to detect/remediate. 

MRG Effitas possesses capabilities to replicate any attack seen in the wild and we can ,most certainly, test these attacks in our lab. 

Our testing environment is highly sophisticated and adaptive and we  do not grab the last stage the payload, we monitor the entire chain. 

Most important of all, we know how malware works, we know how to install a product and ensure it runs properly, we know how to monitor malware activity, and last but not least, everything is recorded, screenshots are taken so it’s pointless to say “they don’t know how to test” when you have an encrypted system and a great big greeting from the attacker asking for his ransom all over the screen. 

We are always happy to answer any questions but so far we have not been contacted by Malwarebytes regarding their concerns, otherwise , I am positive, that the  thread  your are referring to would go in a completely different direction. 

Every industry has testing. Every product has to be evaluated by independent testing companies otherwise we would have chaos. Imagine if there are no third party testing for , lets say cars, we would have wheels flying off all the time, but intensive independent testing has made that impossible and if a car has a fault, it is recalled and the issue is addressed. I fail to understand why testing Anti-Malware products causes so much controversy when a product doesn’t perform well. 

I would advise Malwarebytes to contact us through the proper channels and I am sure that we can deal with this in a professional manner, there is no need for controversy nor speculation, problems are there to be fixed."

Cheers,

Sveta

Best Regards,

Sveta MILADINOV

CEO – Effitas Group

CEO – Effitas Asia Ltd.

Executive Chairman – Effitas Research Ltd.

Efficacy Assessment & Assurance

One Canada Square, Canary Wharf, London E14 5AB

London +44 (0)20 3239 9289

San Jose +1 408 627 7667

Hong Kong +852 8197 7669

Balkans +381 63 212 113

Mobile +44 748 1273715

Office direct: +44 20 81232465

Share this post


Link to post
Share on other sites

At this point I won't even bother asking why Malwarebytes isn't part of Anti-Malware Testing Standards Organization as all members are listed here https://www.amtso.org/members

We have all heard Malwarebytes reasons in this thread/forum (over and over and over again) regarding their opinion on how they really feel about the current testing laboratories out there, their testing methodology, and their posted results. All we know so far is that independent security tester MRG Effitas has responded to Malwarebytes comments about their "10-year old" testing methodology and contradicted everything Malwarebytes said.

I believe in transparency and I personally believe for almost 2 decades now (since my computer virus experience back in the early 2000s) Malwarebytes has not been transparent so until the day I see your product tested side-by-side with your competition and you meet and exceed their performance, then I will ADMIT Malwarebytes is a product I would recommend but until that point, I am not changing my opinion.

So I guess all those people posting videos on YouTube with real computers in the real world showing many other products superior to Malwarebytes on their side-by-side tests, those too cannot hold any weight whatsoever.  So basically what I have learned is this....all the testing laboratories and their results, are not accurate, all the people posting YouTube videos are also wrong and only Malwarebytes who refuses to participate in independent laboratory testing is correct.

 

Edited by Weston1973

Share this post


Link to post
Share on other sites

What little credibility Malwarebytes had left went straight down the toilet in my view with my findings (test results from both AV Comparatives https://www.av-comparatives.org/wp-content/uploads/2017/03/avc_sp_pcpitstop_201702_en.pdf and MRG Effitas https://www.mrg-effitas.com/wp-content/uploads/2017/12/MRG_Effitas_360_Assessment_2017_Q3-1.pdf in side-by-side tests) and the response from the CEO Mr. Sveta Miladinov of independent lab MRG Effitas regarding testing methodologies.  

***This is my last post on this forum/thread.***

You can make your own conclusions from this point forward.

Edited by Weston1973

Share this post


Link to post
Share on other sites

Hello Weston1973,

Let's say you are the owner of Malwarebytes and you tested MBAM internally following the same procedure as AV Comparatives , and the result is disastrous , what would you do???

Would you agree to publically test MBAM by AV Comparatives???? Most likely not. Just continue advertising ( and , btw here in Canada is more than 40$/year ) and hope that enough people will buy the product.

Share this post


Link to post
Share on other sites

Well I was informed that there are millions of malware variants and crypted malware variants made every year -thus antivirus/anti malware vendors can have a rough time keeping up. I have just recently started to resort mainly to sandboxing and running in a virtual environment (often simultaneously).  But that's just me... I don't claim to be no almighty expert on the matter ;)

 

 

Share this post


Link to post
Share on other sites
14 minutes ago, nov1 said:

Well I was informed that there are millions of malware variants and crypted malware variants...

In one year  there are between18,000 to 84,000 meteorites bigger than 10 grams which will hit the surface of the Earth. Did you buy insurance against it????

Most likely not. That being said , In over 15 years I wan never significantly affected by a PC virus (most of them unwanted software or redirecting) ; In more than 3 years and 3 PC I never had a detection initiated by MBAM (other than Web shield, most of the time FP).

MBAM's impact on PC speed is significant though.

Now I am using a well known antivirus (almost zero impact on PC performance) and a back-up software (pay only once).

I do not think anyone needs more. 

Share this post


Link to post
Share on other sites
37 minutes ago, lock said:

In one year  there are between18,000 to 84,000 meteorites bigger than 10 grams which will hit the surface of the Earth. Did you buy insurance against it????

Most likely not. That being said , In over 15 years I wan never significantly affected by a PC virus (most of them unwanted software or redirecting) ; In more than 3 years and 3 PC I never had a detection initiated by MBAM (other than Web shield, most of the time FP).

I believe your best defense is a good imaging program.  The AV and anti-malware programs are useful to alerting you to an infection.  But once you know you have been compromised, the best course of action is to wipe and re-image the PC, or perform a clean re-install if you don't have a recent system image.

In the last company I worked for the IT team did not mess around with trying to clean infected PCs.  They immediately wiped and re-imaged the PC with the standard company issued image.  All of the user profiles including current docs were kept on servers, so the downtime and interruption is much less with this method than trying to disinfect a PC.  And there is never a way to be sure that you are completely clean afterwards, short of nuking your hard drive.

Share this post


Link to post
Share on other sites

I think Weston1973 might have a point.  I came across this AV-Test.org report https://www.av-test.org/fileadmin/pdf/reports/AV-TEST_Enigma_Comparative_Remediation_Testing_Report_May_2017_EN.pdf from May 2017 where Malwarebytes is compared with 4 others.  According to what I understand in reading that report SpyHunter outperformed Malwarebytes too. 

 

Edited by Jamber

Share this post


Link to post
Share on other sites

Weston1973,

Just checked the topics that you started, it looks like you're a spammer trying to make everyone think that Malwarebytes is not a legal company.

Everyone has own opinion regarding which AV is the best. I run Windows Defender and MB 3.0 Pro for years without any issues.

Edited by iamauser

Share this post


Link to post
Share on other sites
20 minutes ago, iamauser said:

Windows Defender and MB 3.0 Pro

It seems like this is not a good combination; somehow MBAM is not 100% compatible with Windows Defender and will slow down your PC.

This is a known issue.

 

 

 

Share this post


Link to post
Share on other sites
1 hour ago, iamauser said:

Weston1973,

Just checked the topics that you started, it looks like you're a spammer trying to make everyone think that Malwarebytes is not a legal company.

Everyone has own opinion regarding which AV is the best. I run Windows Defender and MB 3.0 Pro for years without any issues.

Yes, there seem to be the usual contributions from those who, for whatever reason, post for no other purpose than to badmouth a product.. I've found Malwarebytes and the support provided directly and through these forums to be first class, certainly better than most of the other antivirus and anti-malware products I've used over the years, free or paid-for (and I've been actively using personal computers for real world tasking since 1981).

Windows 7 Home Premium SP1 64x
MB Premium 3.4.2 Beta
Microsoft Security Essentials
CryptoPrevent Premium 9.0.0.0
Casper 10.1 Backup and Tweaking.com Registry Backup 3.5.3

Edited by TempLost
Grammar

Share this post


Link to post
Share on other sites
48 minutes ago, TempLost said:

I've found Malwarebytes and the support provided directly and through these forums to be first class

How did you find Malwarebytes to be first class??? Based on what?

Share this post


Link to post
Share on other sites
2 minutes ago, lock said:

How did you find Malwarebytes to be first class??? Based on what?

My experience - read the post

Share this post


Link to post
Share on other sites

Re: MBAM and Defender, they are 100% compatible. We are using the interfaces available only to AV to manage the registrations and status updates of MBAM in the Windows Security Center. Only Microsoft approved antivirus providers can do this. The difference is that by default we install side-by-side with Defender (even though this behavior can be changed under Settings) as we've always believed that a layered approach is always preferable to relying on a single product.

Re: testing methodologies, we've also always been up front about our disagreement with third party testing vendors (and AMTSO). We disagree with the fact they don't test vector blocking defenses (i.e. not full product), we disagree with their selection of samples (most of the times older than 1 month, no real focus on 0-day effectiveness), we disagree with the "pay to see misses" business model, and we disagree with the use of simulators which do not behave like real malware does nor does it simulate the infection vector. These are typical practices found in most if not all of the 3rd party testing companies, where each testing company incorporates at least two of the above practices. AMTSO, since its inception, hasn't been able to influence any significant change in AV testing in its entire lifespan. AMTSO doesn't have teeth and has failed on its original mission of improving and evolving AV testing.

None of the above should be news to anybody. We've been pretty open and upfront about our views all along. We don't expect everybody to agree with us, but this has been the position since the beginning of Malwarebytes and, even though we will be participating shortly in 3rd party testing, our views about their business model still remain the same.

In summary, if you're a troll, move on and stop spreading FUD. You're not welcomed here. To everybody else, if you have questions or concerns about how Malwarebytes replaces and improves your AV, or our views on AV testing, we have been and remain open to having an honest and transparent conversation. Feel free to PM me and I will gladly try to answer any and all concerns you might have about our technologies or our views on testing.

 

 

Share this post


Link to post
Share on other sites

Wow!  So let me get this straight.  Some guy or girl (we must be politically correct these days) comes on to this board and provides evidence from two labs AV-Comparatives and MRG-Effitas showing many products outperforming Malwarebytes in their respective lab testing results and also provides an email from the CEO of MRG-Effitas disproving everything Malwarebytes has said about laboratory testing methodology and he is called a troll. 

Once I read that comment from the CEO of MRG-Effitas I decided to do some research and found another lab result and provided this lab result from a 3rd testing laboratory namely AV-Test.org from Germany also showing SpyHunter beating Malwarebytes in their testing too that I included in my prior post.

I was also not impressed whatsoever when Pedro Bustamante one of your own employees (VP of Research & Products) calling your own customers or potential customer who seek unbiased lab results from Malwarebytes "test fanboys".

I am on this board to perform some due diligence before I buy this product and my decision has clearly been made now. 

Edited by Jamber

Share this post


Link to post
Share on other sites
11 hours ago, TempLost said:

My experience - read the post

An item cannot be considered "First Class" based on ONE INDIVIDUAL experience.

Share this post


Link to post
Share on other sites
7 hours ago, pbust said:

Re: MBAM and Defender, they are 100% compatible.  ...

What about this:

 

Share this post


Link to post
Share on other sites

@lock that thread is still being investigated and we have not found the root cause (we're still waiting on you to provide logs of the issue happening so we can examine it). If you are running just MSE and MB3, then the program is compatible. There could be other pieces of software that can cause issues though. 

Share this post


Link to post
Share on other sites
3 minutes ago, dcollins said:

@lock that thread is still being investigated and we have not found the root cause (we're still waiting on you to provide logs of the issue happening so we can examine it). If you are running just MSE and MB3, then the program is compatible. There could be other pieces of software that can cause issues though. 

I do not get it: 10 days ago one staff member said:

"Hello lock -  you are correct that on Win 7 machines - MSE  does interfere with MB 3.x versions."

Now you, another staff member, are saying that there is no conflict.

I had MSE 4.10 and MBAM on 3 different PC' s , all Win7 /64 or 32 , with the same result.

Removed everything and installed ESET ; the PC's are at least 3 time faster.

Share this post


Link to post
Share on other sites

I do see some confusion in that post, which I'll address internally and make sure everyone is on the same page. We run MB3 and MSE/Windows Defender on millions of devices without issue, so in cases where something is conflicting, there's usually some third party interaction happening. If you would like us to try and troubleshoot your issues, we can, but we need the information requested in that thread.

Share this post


Link to post
Share on other sites
59 minutes ago, lock said:

An item cannot be considered "First Class" based on ONE INDIVIDUAL experience.

Nor should it be condemned based on a few ill-informed and poorly researched opinions.

Share this post


Link to post
Share on other sites

An AV that has highest detection rate is NOT always the best one. The best AV is the one that you found to be easy to use, and you likes it.

This is why many people are using Defender even though researches show that it is weaker than leading AV likes Kaspersky, because it is included in the OS, free to use and no annoying ads.

Share this post


Link to post
Share on other sites
2 hours ago, dcollins said:

We run MB3 and MSE/Windows Defender on millions of devices without issue

It is not exactly an "issue".

Only if you uninstall both and run something else, you can see the difference. With both MSE and MBAM you have a delay, let's say of 0.6 sec in opening a page; "millions" of users wouldn't bother for this (like me)

Now, you install something else and the page opens instantly (let's say 0.1sec) .

So, we can say that the PC is 6 times faster.

Share this post


Link to post
Share on other sites

If you run an AV like Avast, Avira, or Bitdefender on a modern PC,  you are not likely to 'feel' any impact of running an AV.

Those tools are real-time file scanners that will scan any file you access, so there is some system overhead, but most are likely to be better performers than Defender.

Real-time anti-exploit measures such as MalwareBytes, or HitmanPro.Alert, use different mitigation and risk reduction methods that don't necessarily eat as much CPU time.  So it is more likely that your choice of primary AV (and the PC hardware) that determines your overall performance.

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...

Important Information

This site uses cookies - We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.