Frontier AI is a popular topic of late. Indeed, the impact of Frontier AI to the way we identify, mitigate, and patch vulnerabilities is far reaching. It is nearly certain that the security profession will need to evolve as a result. That evolution already appears to be underway in many enterprises.
In my experience, enterprises have two high-level concerns around Frontier AI:
- Regarding their own applications, they are concerned that they will not be able to keep up with the increased pace of identifying, mitigating, and patching vulnerabilities
- Regarding their vendors, they want to understand how Frontier AI affects product security and how their vendors will leverage Frontier AI
The first concern is certainly an interesting topic, and perhaps I’ll discuss in a future piece. In this piece, I’d like to focus on the second concern. Namely, with so much hype around Frontier AI and so many vendors speaking to it, how can enterprises make sense of the noise? In my experience, this involves probing vendors, questioning their claims, and digging a bit deeper to understand where they really are (versus where they say they are). There are probably many angles enterprises can take to accomplish this, but here are a few that I’ve found helpful:
- Model Providers: It seems that working with Frontier AI model providers has become a bit of a status symbol in the security industry. Sadly, some in the industry have taken to stating that they are working with one or more model providers that they may not actually be working with. In general, misrepresentation is a bad idea, but it is an especially bad idea when it comes to a sensitive topic like product security. Enterprises need to push their vendors to understand with whom, exactly, they are working and what, specifically, they are doing with them. If the answers seem vague, non-commital, or circular, then that is something the enterprise should question.
- Models: While there are a small number of Frontier AI model providers, there are a large number of different models. It is worth an enterprise probing their vendors a bit to understand what specific models they are working with. Different models vary widely in capabilities, limitations, effectiveness, true positives, and false positives. As such, it is too easy for a vendor to make it sound as if they are far more empowered than they actually are.
- Automation: We all love automation, and rightfully so. As the pace of vulnerability identification speeds up, automation becomes an essential component to the process. As such, vendors will be rushing to say that they have automated the process around Frontier AI. But can they possibly be as automated as they say they are? Frontier AI is a relatively new area that is rapidly developing and maturing. Because of this, there are growing pains, false positives, and gotchas that the security community is new to. It is certainly possible to automate important pieces of the vulnerability identification, mitigation, and patching process, but be on high alert when a vendor tells you that they’ve automated the entire process or the vast majority of it.
- Context: Context is king for so many things, Frontier AI included. As amazing a technology as it is, an organization cannot simply throw a bunch of code at a Frontier AI model and expect results. Code needs to be harnessed before it can be input into the model. The better the harnessing, the better the results. So while it is possible for a vendor to claim that they are working with Frontier AI, it is worth understanding more about how they are doing so to ensure good results.
- Results: On the topic of results, how can an enterprise know if a given vendor is getting real results from using Frontier AI? While this is a complex and difficult question, some of the points above are a good place to start. Beyond that, the vendor should be able to articulate metrics around true positives and false positives, quantities of real vulnerabilities, time to mitigate and/or patch, and other important metrics. There should be some substance behind the claim of good results – not just lip service.
- Vetting/Validation/Verification: Like any technology, Frontier AI can be prone to false positives. This is not a bad thing necessarily – sometimes finding novel vulnerabilities and exploits requires taking chances that might also lead to some false positives. The question is how the vendor handles those false positives. Are they taking the time to vet, validate, and verify that a vulnerability is real? Are they also taking time to vet, validate, and verify that the fix is effective and does not introduce operational issues or even additional vulnerabilities? The vendor should be able to address these and other questions around vetting, validation, and verification. If they can’t, that should give the enterprise some pause.
One of my favorite Nietzsche quotes aptly states that “The truth doesn’t mind being questioned. A lie doesn’t like being challenged.” It amazes me how often I encounter statements that fall apart with just one follow-up question, one fact, and/or one grain of truth. The vendor-customer relationship is fundamentally built upon trust. It is paramount that vendors be open, transparent, and honest with their customers around all topics, but particularly around the important matter of product security. Not doing so should be unpalatable to any serious security professional.
Learn More at the AI Risk Summit | Ritz-Carlton, Half Moon Bay
