Jump to content
nccomp

MBAM 3.0 and av-comparatives.org

Recommended Posts

Hehe, agreed.

Also please bear in mind that while I'm obviously with Malwarebytes on the issue of comparative testing and its relevance, I'd still really like to see them participate in these tests with their Premium version, especially since the most recent builds of Malwarebytes 3 now include something that, to my knowledge, hasn't been in any of the builds tested to date which is a new heuristics malware detection engine custom built late last year and only recently turned on in the product (the new "signature-less anomaly detection" component listed under Scan Options (it also applies to realtime protection even though it's listed under Scan Options) as I believe it will have quite an impact on such tests (plus, if they're using the Premium version I believe the Anti-Exploit component in Malwarebytes would really kill it if they perform accurate real-world tests that begin the infection routine from the earliest point in the attack chain which is where exploits usually come in being one of the first steps in the vast majority of malware attacks these days, regardless of the eventual payload be it a rootkit, ransomware, Trojan or pretty much anything else).

Edited by exile360

Share this post


Link to post
Share on other sites

Those numbers are fine and dandy but until you own up and start getting tested next to your competition you are losing credibility here. 

I am going to tell you a little story and this might not be relevant today but back in the early 2000s I had a nasty virus on my Windows XP computer and I tried every cleaner out there from Malwarebytes, Spybot's Search and Destroy, Comodo Cleaning Essentials, Trend Micro's HouseCall, Emsisoft's  Emergency Kit, and a couple of others I can't remember off the top of my head right now and only ONE PRODUCT was able to eradicate this nasty virus and that product was HitmanPro. 

Now you talk about wanting "real world data" to be tested on, yet I had a "real" virus on a "real" computer and isn't it amazing how in all those MRG Effitas reports I indicated in my post above, the similar results are still happening today. MRG Effitas still shows HitManPro beats Malwarebytes in EVERY initial detection rates of the on-demand security products test for the past 2 years (8 straight quarterly 360 Assessment tests).  

Edited by Weston1973

Share this post


Link to post
Share on other sites

Sure, you're right, but a big part of the reason is due to the fact that many of the files being scanned in those tests aren't binaries and therefore aren't even scanned by the signature based malware detection engine used in the scanner (the anti-exploit realtime protection component would target such threats, but as I said, they only test the free version to my knowledge).

Still though, you're absolutely correct, testing should be done and as I said, I too would like to see it for myself.  I've been pushing for it ever since they began marketing Malwarebytes 3 as an AV replacement and I still believe it's an important thing to illustrate their effectiveness by an outside, independent testing lab.

Share this post


Link to post
Share on other sites

According to the MRG Effitas reports it clearly says this in all their reports "All security applications were fully-functional unregistered versions or versions registered anonymously, with no connection to MRG Effitas."

Notice the words FULLY-FUNCTIONAL so your argument about "they only test the free version to my knowledge" holds no water.

All I do is give you proof, all Malwarebytes does is hide under this need for "real-world data" which I feel is smoke and mirrors.  Isn't it ironic that the most popular Protection Test out there by the largest independent lab in terms of testing numbers AV-Comparatives begins the words "Real-World" in front to their "Protection Test" as you can clearly see below:

http://chart.av-comparatives.org/chart1.php?chart=chart2&year=2017&month=Jul_Nov&sort=1&zoom=2

Doesn't this feel like two children fighting and saying my "real-world test" is better than your "real-world test"!!!!

I am glad you agree with me, it's time the whole entire world sees how Malwarebytes fares against the competition in a side-by-side test once and for all and then the unbiased truth will come out.

 

Edited by Weston1973

Share this post


Link to post
Share on other sites

They actually call out the fact that it's an on-demand scan by placing it in a separate chart from the other active products and refer to it as such.  They didn't activate a paid license which is also why they always only ever refer to it as "Malwarebytes" or "Malwarebytes Anti-Malware" and never ever refer to it as "Malwarebytes Premium" or "Malwarebytes Anti-Malware Pro".  That's also the reason Malwarebytes does so poorly so consistently on the tests where they do use any testing against actual exploits and ransomware because those components aren't active in the free version.

I do wish they'd test using the free trial of Premium, but to my knowledge they haven't done a test that included Malwarebytes since the developers removed the option to opt-out of the trial during installation (though they could disable protection or deactivate the trial post-install, but hopefully they wouldn't do that).

Share this post


Link to post
Share on other sites

They still call HitmanPro SurfRight in their reports yet HitmanPro was sold by SurfRight to Sophos back in December 2015 and no offense but you are grasping at straws now and are using verbage or grey areas in they way things are worded in these MRG-Effitas reports to make it seem like your product wasn't properly and unfairly tested.   The report clearly says all products that were tested were FULLY-FUNCTIONAL.  Do I need to define what the definition of the word fully is?  Your free version without any real-time protection is NOT fully functional since it is missing a component. Clear as mud?

Here's an idea.....why doesn't Malwarebytes participate in the AV-Comparatives Real World Protection Test just once, I am only asking you get tested once in 1 month alone, this way the whole world can see how well or poorly you do.  I am not asking for much, 1 test!     

Why all these smoke and mirrors,  and excuses?   You sound like you may have a virus...maybe it's time you get tested!  LOL

Edited by Weston1973

Share this post


Link to post
Share on other sites

Come on, really?  Just ask MRG to clarify it.  Ask them point blank if they are testing the Premium version of Malwarebytes or the free version.  The grey area in the wording isn't mine, it's theirs.  They refer to it as an on-demand product.  I don't know what that means to you, but to me it means that it is only checking for threats on an on-demand basis, meaning when the user scans with it manually.  If I am wrong, that's fine, but I haven't seen any references to any component of Malwarebytes which is included in the paid version.  I have seen where they talk about things like web filters, exploit guards and other realtime protection components in their tests in reference to some of the other products they've tested, but not once have I ever seen them make such a reference in regards to Malwarebytes on any of their tests and that's because I do not believe they've ever purchased a license for testing and have only ever tested it as a second opinion, on-demand scanner running in free mode without any active protection layers.

I agreed with you that I'd like to see the testing also; I'm not making excuses or saying that they shouldn't participate in the tests as I would love to see it myself regardless of the methodology used because I truly believe Malwarebytes 3 Premium would do extremely well given its current lineup of layers.

Share this post


Link to post
Share on other sites

I just emailed MRG-Effitas this question below and once I hear back from them (and feel free to confirm it once I post it) I will post their response but whatever the answer is, it won't change my opinion that it's time Malwarebytes gets tested next to your competition instead of hiring what it seems to be lawyers who do tech support and write on forums who enjoy disproving evidence on technicalities when it relates to the wording on a test result, and using the need to have "real-world tests" as their objection. 

I am sorry I don't believe for one minute that your product is so "different" than the others (same song and dance at Webroot too which doesn't get tested) and does things so differently that none of the testing laboratories results would give Malwarebytes a fair result.  I believe and I am sure many others too really know why Malwarebytes has been putting off getting tested side-by-side against the competition for a very long time.  You don't need to be a rocket scientist or lawyer for that matter to figure this one out.

My question to MRG Effitas:

Your 360 Reports say and I quote “"All security applications were fully-functional unregistered versions or versions registered anonymously, with no connection to MRG Effitas."

In this report below it says you tested Malwarebytes Anti-Malware 3.2.2.2029 but we want to know was it the FREE version without real-time protection or the registered version which is their PREMIUM version and comes with Real-Time protection.

https://www.mrg-effitas.com/wp-content/uploads/2017/12/MRG_Effitas_360_Assessment_2017_Q3-1.pdf

Edited by Weston1973

Share this post


Link to post
Share on other sites

Hey, you're preaching to the choir here.  I'd love to see Malwarebytes tested just as I said.

And as for my opinion of testing methods etc. goes, I already mentioned that I don't believe it's accurately portraying the abilities of other products either (including many that do well on the tests, not just the ones that 'refuse' to be tested).

As for real-world data, I do know that Malwarebytes currently has millions of paid users, the vast majority of which either use no other protection (i.e. no AV at all, just Malwarebytes) or they only use Windows Defender (which you and I agreed isn't necessarily the most stellar AV available), yet reports of infected users who have Malwarebytes 3 Premium are very few.

By the way, I forgot to address something you mentioned earlier.  You spoke of an infected system that you ran several scans on from several products and Malwarebytes was among those that missed it.  This is precisely the kind of thing we're talking about.  You're judging the efficacy of a product based on the results of an on-demand scan after the fact which didn't include giving the advanced, signature-less capabilities in Malwarebytes the chance to stop the attack before it got to the stage of the system being infected.  It is a fact that the vast majority of threats these days rely on exploits to infect systems and that the exploit phase is one of, if not the first in virtually all of these attacks and the exploit protection in Malwarebytes 3 is by far one of its most forward-looking, effective features when it comes to threat prevention.  The day they begin doing tests which replicate real attacks from the beginning (not just downloading the malicious .JS file directly from the server and executing it from the desktop then expecting the products to either detect the raw .JS file or its binary payload that it downloads, or the PowerShell script they download/execute the same way) is the day I will stand up and take notice.  Setting up a proper test-bed isn't an easy thing, I'll grant you that, but in order to properly test all of these products, not just Malwarebytes, that's what they really need to be doing, otherwise I believe they're doing a great disservice to the users reading their reports.  Heck, one of the product you mentioned, HitmanPro, would likely do quite well in such a test (at least if we're talking about HitmanPro.Alert) as it also includes a rather extensive anti-exploit layer that I'm sure would prove quite effective as well.  I'm just tired of all these tests which are largely based on archaic flat file detection capabilities because getting a 100% pass is too easy with a whitelist database engine or anti-executable engine layer, but to really stand up against real world attacks (including file-less malware and the like) more than that is needed, and that's where these more advanced signature-less behavior based technologies shine; again, not just in Malwarebytes, but in most of the products they're testing.  There's a reason so many vendors have begun adding such additional protective layers over the past few years: it's necessary and it works and they know that a standard signature based approach (which is mostly what's being tested) isn't enough to keep their users safe any more.

Share this post


Link to post
Share on other sites

Not so sure about the MBAM heatmap, basically its showing that its catching a lot that Microsoft and Avast is missing, but both of them have the majority of users so of course they would catch the most from these 2 vendors. Also, I'm sure every AV vendor could produce the same type of map showing the exact same thing as all of them miss things that the others catch and of course they would show the most misses from the most used vendors, eg, Microsoft and Avast. Not saying that MBAM isn't a good product or doesn't provide top notch protection. I would also like to see more AV tests with MBAM involved.  

Edited by digmorcrusher

Share this post


Link to post
Share on other sites

Stop talking like a lawyer for one minute and saying things like "You're judging the efficacy of a product based on the results of an on-demand scan after the fact which didn't include giving the advanced, signature-less capabilities in Malwarebytes the chance to stop the attack before it got to the stage of the system being infected".

Let's stop talking like a lawyer for one minute and see scenario like a consumer for once!  Why would a buy a product that doesn't clean a virus despite all this beautiful wording you just said to me?  I don't care about the origin, or the on-demand scan or anything else for that matter.  The result is very clear!!!!  HitmanPro saved my computer and Malwarebytes was totally worthless in this case when it came to cleaning this virus.

A security product should be able to detect any threat before it enters your computer BUT in the event if that virus/threat does ever enter your computer for whatever reason, a security product whether it is an anti-malware or an on-demand scanner or an anti-virus product should be able to clean this threat that Malwarebytes failed miserably to do in my case along with the others except HitmanPro.  Case closed and my next response to you will be the MRG Effitas response to my question about FREE or PREMIUM version used in their testing.

 

Edited by Weston1973

Share this post


Link to post
Share on other sites

It's not lawyer talk, it's reality.  Obviously Malwarebytes should have detected the threat, of course, I give you that, but it isn't lawyer talk to state a fact just as it isn't absolute proof of superiority to quote a single instance where one product detected a single threat missed by another as the reverse is just as common (again, I refer to the heatmap which proves this very fact and is again only based on Malwarebytes' on-demand scan engine, not any of those signature-less protection capabilities I spoke of).

Also, with regards to what digmorcrusher stated above regarding the statistics, this is absolutely correct.  The larger the user base for a given AV, the higher the probability that it will show more threats missed just based on the amount of raw data gathered.  I simply posted it to illustrate my point that I have a live resource based on raw, real-world data that illustrates constantly that Malwarebytes proves itself capable of detecting threats that all of these other AVs miss, and again, it doesn't include PUPs (which Malwarebytes is notoriously aggressive against), nor does it include any threats/attacks prevented by any of the realtime protection components in Malwarebytes.

I look forward to the info from MRG because it's something I too want clarified once and for all, regardless of what the answer is and again, I wholeheartedly agree that I want to see Malwarebytes 3  Premium tested against these other products because it's something I've been wanting for a long time, ever since they first began marketing it as a possible AV replacement product.

Also please do bear in mind that no matter what we say about X vs Y, we don't expect customers to just take our word for it that Malwarebytes is all they need because even though we do believe this, Malwarebytes has always been and still is designed to run safely in realtime alongside other security products, including AVs as well as other anti-malware applications so if a user isn't confident that Malwarebytes alone is sufficient protection, they are free to run other products with it, be it one of the many reputable free AVs out there or a paid solution.

Share this post


Link to post
Share on other sites

I emailed AV Comparatives too about why Webroot and Malwarebytes don't participate and this was their answer:

"Hi, yes they have participated in the past publicly (at least Webroot) but scored sub-par and stopped then to participate. They might participate again publicly in the next years.

You can find some results in this old/easy test: https://www.av-comparatives.org/wp-content/uploads/2017/03/avc_sp_pcpitstop_201702_en.pdf

I cannot share results which are internal / non-public.  http://chart.av-comparatives.org/awards_by_vendor.php?venID=18 "
 

Now I took a look at the ransomware test AV Comparatives performed and both Webroot and Malwarebytes were tested in March 2017 and in case you're wondering it was the PREMIUM version that was tested "MalwareBytes Anti-Malware Premium 3.0" which is CLEARLY written in this report.  Yet, once again Malwarebytes and Webroot don't come close to attaining 100% Ransomware Detection compared to Bitdefender, Kaspersky, Avast/AVG, Avira,  Symantec, Trend Micro, and some others too who also attained 100% detection but wait, this wasn't a "real world sample" with "real world data" so Malwarebytes once again is forgiven for not attaining 100% right?  You can't use the Malwarebytes FREE version as an excuse this time around though as this AV Comparatives testing report clearly says it is your newer Malwarebytes version 3.0 and also clearly says PREMIUM meaning it has real time protection and that anti-exploit protection you keep telling me about which is only available on this premium real-time protection version.

So now I have given evidence on this forum from two INDEPENDENT labs, MRG Effitas and AV-Comparatives and that according to both their independent test results, there are way better performing products than Malwarebytes.

Like I mentioned earlier, once I receive the response from MRG Effitas on what Malwarebytes version they were using Free or Premium I will post it here.

 

 

 

Edited by Weston1973

Share this post


Link to post
Share on other sites

I would like to know what they consider "real world" though.  Are they just downloading/executing live ransomware samples, or are they visiting infected sites where the user would naturally encounter the exploit that attempts to drop and execute the ransomware thus truly replicating the entire attack chain?  As far as the actual numbers go, I don't believe 93%, 95.1%, 99.2%, and 0 false positives is abysmal, plus this was quite early in the development of the anti-ransomware component.  They were still heavily tweaking the detection to ensure there were no FPs which at fist meant it missed more actual ransomware than it should have (this has since been corrected, but the anti-exploit component still remains the strongest of the bunch when it comes to signature-less behavior based protection).

Again, no excuses, the anti-ransomware component in Malwarebytes just wasn't that good back then.  It's a fact and I won't deny it.  It was a work in progress at the time and while obviously lacking, did still add further protection to Malwarebytes 3 and has gotten stronger over time as the Devs have tuned it up.

I still want to know about their methodology though.  I understand if they just want to flat out test raw detection of ransomware files and ransomware behavior, and there's nothing wrong with that at all.  If that's what the test was (and I believe this to be the case, though I do not know for certain due to lack of explanation in the document), then these are the kinds of results I'd expect from Malwarebytes 3 in early 2017 because that module had just come out of beta and was still very young and remained a work in progress.

Share this post


Link to post
Share on other sites

You seem like a smart person, if you want to know their methodology, why don't you email AV Comparatives like I just did when I wanted a question answered and they will tell you what you are looking for. 

It seems like Malwarebytes always has an excuse and now the excuse is your real-time protection wasn't as good as it now and reason why it didn't perform very well on this testing.  If you couldn't keep up back then on making a competitive product why should consumers believe you now?  Did you also notice AV Comparative's comment about Webroot that they made?  They pretty much said, Webroot's test results were so poor they stopped participating...and that is why in my humble opinion I truly believe is the REAL reason why both Webroot and Malwarebytes won't participate in these type of tests because you won't obtain favorable results.   Yet both Webroot and Malwarebytes have both used the "our product works differently than a traditional AV" excuse which wouldn't lead to favorable results hence justifying why you both don't participate in these type of tests.  The truth finally came out for Webroot's performance based on that "sub-par" comment AV Comparatives made about Webroot and eventually the truth about Malwarebytes's performance compared to their competition will come out too.

Edited by Weston1973

Share this post


Link to post
Share on other sites

Nope, just that one module, not all of Malwarebytes realtime protection.  What it lacked was compensated for by the other components (at least that's what our testing showed as well as feedback from customers).

As for Malwarebytes and their lack of participation, I really couldn't tell you because I am not in charge of the Product team therefore I do not get to decide whether Malwarebytes participates in such tests (otherwise I would have done so myself long ago and published the results, regardless of how well or poorly Malwarebytes might have done because I believe in transparency).

I would again like to reiterate that I am not trying to make excuses here, nor am I trying to deflect from the fact that Malwarebytes isn't participating in these tests.  I believe they should regardless of how meaningful or meaningless they might be, and regardless of how accurately or inaccurately they reflect real-world environments and scenarios if only because it's a question that I really am tired of answering.

I will leave you with one final note.  While I am no threat research engineer or expert when it comes to malware, I do have a good basic understanding of how attacks work and how the behavior of users is exploited by the bad guys in order to infect their systems, and it is based on these things that Malwarebytes has designed much of their current protection technology because there is no silver bullet when it comes to security and you must understand how systems get infected in order to adequately protect them from online threats.  So here is a basic layout of a common infection scenario and how the various layers of protection in Malwarebytes 3 work to thwart such attacks every step of the way, and why I believe the vast majority of these tests are lacking:

  1. User visits website which contains malicious advertisement containing browser exploit of some kind (this could be a Flash Player exploit, targeted web browser exploit focused on Chrome, IE, Firefox, Safari or any other browser or all of the above as some exploit kits are) - This is where the Web Protection in Malwarebytes comes into play.  If the ad is hosted on a known malicious ad server or connects to a known malware host to download its malicious script or payload, Malwarebytes will block the connection thus thwarting the attack long before any binary files have reached the system
  2. Malicious ad checks browser/system info via most common means such as user agent string etc. in order to determine if system is a viable target (if it isn't, the exploit won't even try to execute; some security software actually leverage this as a means of protection by "stealthing" the user's system and/or browser by making it appear incompatible with the most common exploit kits)
  3. Assuming target is viable, exploit attempts to launch in order to download and execute a malicious script, usually a .JS or PowerShell script - This is where the Anti-Exploit layer comes into play; the detection rate of exploits is extremely high in Malwarebytes 3 and the closest thing I've seen to a 100% effective solution against a major aspect of the current threat landscape (and not coincidentally, this is one of the most commonly skipped phases of the attack chain by the vast majority of so-called "real world" tests because replicatiing a real system and consistently finding the same exploit is no simple task as they are taken down and change frequently deliberately to attempt to avoid threat researchers and the like who would seek to grab their malicious wares and improve their efficacy against them, not to mention the fact that ad services such as Google and the like frequently take down malicious ads as they find them and the ads shown on most sites are not static so the ad you see during one visit to a page may not be the same as it is the next time you visit the page, even if only within minutes or even seconds)
  4. Once the exploit successfully launches, it generally downloads other malware such as ransomware, Trojans etc. - This is where Web Protection comes into play yet again, as we try to block the host servers of known malware as well as known malicious Commant & Control servers
  5. If the download was able to bypass those layers, the next step is execution of the malicious binary into memory Here is where the more old school Malware Protection component comes into play, attempting to detect any malicious binary that attempts to enter memory once downloaded (assuming it was a binary file and not another script/exploit of some kind; if it was, then we return to one or more of the above layers/steps)
  6. Assuming none of the previous layers has thwarted the attack, if the threat in question is ransomware, the ransomer begins its work attempting to encrypt files and do things like deleting backups/shadow copies of the files and disabling tools such as System Restore and the like - This is where Anti-Ransomware takes over, monitoring in realtime the behavior of the threat, including looking for common malicious behavior as well as seeing if the threat triggers any of the "traps" Malwarebytes plants on the system when it is installed and Ransomware Protection is activated.
  7. Now, finally, if the malware has made it past all of these layers, you have at least a couple more defensive layers to rely on - First of course, is the Threat scan/remediation which utilizes several advanced heuristics techniques such as Linking in addition to the more traditional malware threat signatures and this is also where the Anti-Rootkit component (if active) comes into play to detect and remove any rootkits which might have been part of the attack

Oh, and there's one final layer I neglected to mention above: whenever a file is checked by Malware Protection or the scan engine, it is also analyzed (by default, at least) by the new signature-less Anomaly Detection engine which includes advanced heuristics algorithms as well as cloud analysis to enhance Malwarebytes' ability to positively identify new and unknown threats and suspicious files.  This feature was only recently (within the past ~5 months) activated and since going live, has been incrementally improving itself through data collection and machine learning as well as occasional tweaking by the Dev and Research team to sharpen its capabilities even further.

That's obviously not every possible scenario, but definitely one of the most common and one I would love to see accurately replicated in a laboratory setting for the purpose of testing Malwarebytes and other security products because as I said before, Malwarebytes is not the only vendor beginning to use the more advanced layered techniques and technologies and as long as these phases of the attack chain are being neglected in testing, the true value of these products (again, not just Malwarebytes) aren't adequately being assessed in my opinion.

Edited by exile360

Share this post


Link to post
Share on other sites
10 hours ago, digmorcrusher said:

Not so sure about the MBAM heatmap, basically its showing that its catching a lot that Microsoft and Avast is missing, but both of them have the majority of users so of course they would catch the most from these 2 vendors. Also, I'm sure every AV vendor could produce the same type of map showing the exact same thing as all of them miss things that the others catch and of course they would show the most misses from the most used vendors, eg, Microsoft and Avast. Not saying that MBAM isn't a good product or doesn't provide top notch protection but that map is FUD and should not be used to promote the product. I would also like to see more AV tests with MBAM involved.  You want to see how a effective a product is, just go to their forum or any forum that does malware removal and see how many users of said products are active in the " I Am Infected" part of the forum. Thats how I judge a products capability.For instance, I use Emsisoft, many people go there for malware removal, few are actually using the product, so imo, EAM is effective. Just a side note, many of these forums use MBAM for malware removal.

I also raised concerns about the methodology used for the Malwarebytes Remediation Map in post # 39 .  For example, does the "missed" count include false positive detections by MB v3.x? Would the other "traditional" AV software registered in the Windows Action Center (e.g., Avast, Kaspersky, Symantec/Norton, Windows Defender, etc.) have detected any of these "missed" threats if MB v3 was not installed on the same machine?

The MB v3.2.2.2029 used for the MRG Effitas 360 Degree Assessment & Certification Q3 2017  testing is referred to as a complementary "on-demand" security product on page 9 of that report so I'm assuming they tested MB Free rather than MB Premium, but the table on that same page also shows a high failure rate for MB v3.x versus three other similar products (SurfRight HitmanPro, Watchdog Anti-Malware, Zemana Anti-Malware) that are not traditional antivirus programs.

Like many other users posting in this forum, I'm looking forward to the day that Malwarebytes finally submits MB Premium v3.x to a few reputable, independent antivirus testing firms like AV-TEST (https://www.av-test.org/en/antivirus/home-windows/) and AV-Comparatives (https://www.av-comparatives.org/comparatives-reviews/) so we can see how well MB Premium v3.x's real-time protection stacks up against traditional antivirus programs in side-by-side "real world" and malware detection tests.  The June 2017 Checkmark Certification report that staffer bdubrow referenced in post # 28 does not include any side-by-side testing with other products so I'll need to see further evidence that MB Premium v3.x is actually a reliable "antivirus replacement" as Malwarebytes claims <here> in their FAQ before I'm ready to ditch my third-party AV.
----------
32-bit Vista Home Premium SP2 * Firefox ESR v52.6.0 * NS Premium v22.12.0.104 * MB Premium v3.2.2.2183-1.0.262

Edited by lmacri

Share this post


Link to post
Share on other sites

MRG Effitas wrote me back and like I said before, your excuse about you assuming their test was performed with your free version holds no water as I proved to you MRG Effitas said "Fully-Functional" in their report.  So now I have provided 2 independent lab companies MRG Effitas and AV Comparatives who both tested your Premium FULL version and yet Malwarebytes didn't perform very well in both their test results.  I think I made my point here!

Here is MRG Effitas' response from Sveta Miladinov from this page https://www.mrg-effitas.com/about-us/our-team below and feel free to email them and confirm this.

It was the full version with real-time protection.

We only test fully functional products, no free versions with limited capabilities.

Cheers,  

Sveta

Edited by Weston1973

Share this post


Link to post
Share on other sites

Hi Sveta,

as we've said many times, we don't agree with most testing methodologies out there as they consistently don't test any of the vector blocking capabilities of security products. MB3 relies heavily on vector blocking for early detecting and blocking of modern threats. Yet most labs, including MRG, grab the last stage payload and funk around with it and just test it against the on-access and post-execution protection layers, completely bypassing the vector blocking capabilities which could have stopped the threat in the real world. Malware spreads using certain techniques which are blocked by MB3, and neither MRG nor any of the other labs replicate those real-world environments, and ergo do not test the full product, only portions thereof.

We will be participating in more lab tests shortly and biting the bullet of adding unnecessary bloat to the product to entertain these unrealistic tests and test fanboys, but that doesn't mean that we agree with them. Their testing methodologies are all ~10 years old.

 

Share this post


Link to post
Share on other sites

Ahh, sorry, I thought you were also called Sveta (I've spoken to Sveta @ MRG few times before).

 

Share this post


Link to post
Share on other sites
28 minutes ago, pbust said:

We will be participating in more lab tests shortly and biting the bullet of adding unnecessary bloat to the product to entertain these unrealistic tests and test fanboys, but that doesn't mean that we agree with them. Their testing methodologies are all ~10 years old.

For myself, I'd be disappointed if Malwarebytes felt they had to go down that route - it would be a retrograde step.

Share this post


Link to post
Share on other sites
18 minutes ago, TempLost said:

For myself, I'd be disappointed if Malwarebytes felt they had to go down that route - it would be a retrograde step.

Thank you, I feel the same way. We're being extra careful of not just adding blot to meet the test demands. In most cases, the samples tested by 3rd party labs are samples which have long been dead. We'll play the game, but without adding unnecessary bloat.

Share this post


Link to post
Share on other sites
2 hours ago, pbust said:

...We will be participating in more lab tests shortly and biting the bullet of adding unnecessary bloat to the product to entertain these unrealistic tests and test fanboys, but that doesn't mean that we agree with them. Their testing methodologies are all ~10 years old.

It's disappointing to hear that long time MB customers who have asked about side-by-side comparative testing and unbiased evidence that MB v3.x is an "antivirus replacement" are being referred to as "test fanboys" by Malwarebytes staff.  I'll be unsubscribing from this thread now.
----------
32-bit Vista Home Premium SP2 * Firefox ESR v52.6.0 * NS Premium v22.12.0.104 * MB Premium v3.2.2.2183-1.0.262

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...

Important Information

This site uses cookies - We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.