After well over 20 years involved in some aspect or another of the security industry, several of them supplying services to the anti-malware industry, I can say with some confidence that AV product testing has given me more white hairs than any other security topic. Only a true obsessive, a hallucinating optimist or hopeless masochist would admit to being a Director of AMTSO, the Anti-Malware Testing Standards Organization, so I won't mention that...
But I will mention that the next AMTSO members meeting is at San Mateo, California, on the 10th and 11th February, conveniently and non-coincidentally arranged just before RSA. There is more information (including the preliminary agenda) on the AMTSO meetings page, but if you're already a member of AMTSO, you probably already know that (in which case, please register!) So why am I mentioning it here?
AMTSO has attracted a lot of negative publicity and comment in the last year or so. Some of it as a result of various people and organizations involved in some form of testing who regard it as a threat (maybe I'll discuss that here some other time); some of it as a result of the common view of AMTSO as an anti-virus enclave rather than as a coalition between vendors and testers, which is the way we see it, though a high proportion of members are vendors; some of it because AMTSO's high membership fee (inevitable given our overheads, I'm afraid) and emphasis on technical issues and expertise has tended to exclude the man in the street (or at any rate the person on his or her home laptop) and is seen as unacceptably elitist.
I don't think elitism is altogether a bad thing in a context where the difficulties and complexities are so often underrated, even by professionals in other areas of security. At any rate, when it comes to the generation of guidelines to more accurate testing, it seems reasonable to expect that documentation to be backed by significant knowledge and expertise. At the same time, the customer is intended to be a major beneficiary of improvements in testing, and deserves more of a voice in the process.
I expect some pretty significant steps in this direction by the end of the workshop: in any case, I'll report back here in due course. I don't think we're going to see the end of bad testing in the near future, nor of AMTSO being used as a stick with which to beat the anti-virus industry. But I do hope to see a more realistic engagement on AMTSO's part with the internet community at large.
David Harley CITP FBCS CISSP
ESET Senior Research Fellow
