AI-Based Attacks , Artificial Intelligence & Machine Learning , Fraud Management & Cybercrime

Proof of Concept: Boosting Security and Taming AI 'Lies'

Troy Leach and Avani Desai on Risks of AI Hallucination and Misleading Outputs
Clockwise, from top left: Anna Delaney, Tom Field, Troy Leach and Avani Desai

In the latest "Proof of Concept," Troy Leach of the Cloud Security Alliance and Avani Desai of Schellman discuss the risks of AI hallucinations, As AI models advance, hallucinations pose serious threats to security, especially when quick and accurate decision-making is essential.

See Also: 2024 CISO Insights: Navigating the Cybersecurity Maelstrom

Risks are growing as AI models develop more humanlike thinking and reasoning capabilities, Leach said. While AI may be able to understand what the prompter or software developer wants, it could provide the wrong answer "because it thinks it meets your objective better."

"Trust in AI systems … has to be built and maintained through a really rigorous process and continuous oversight," said Desai, pointing to the need for proactive strategies such as AI red teaming, to understand the vulnerabilities.

In this panel discussion, Anna Delaney, director, productions; Tom Field, vice president, editorial; Troy Leach, chief strategy officer, Cloud Security Alliance; and Avani Desai, CEO, Schellman - discussed:

  • How AI hallucination could disrupt decision-making and cybersecurity operations;
  • Strategies to detect and mitigate risks of "lying" AI models, including AI red teaming;
  • The balance between leveraging "good AI" for threat detection and maintaining human oversight.

Leach has spent more than 25 years educating about and advocating for the advancement of responsible technology to improve the quality of living and parity for all. He sits on several advisory boards as an expert in information security and financial payments. Leach also founded a consulting practice that advises on the opportunities to leverage blockchain technology, zero trust methodology and various cloud services to create safe and trusted environments. Previously, he helped establish and lead the PCI Security Standards Council.

Desai has domestic and international experience in information security, operations, profit and loss, oversight, and marketing involving both startup and growth organizations. She has been featured in Forbes, CIO.com and The Wall Street Journal and is a sought-after speaker on a variety of emerging topics, including security, privacy, information security, technology trends and the rising number of young women involved in technology.

Don't miss our previous installments of "Proof of Concept", including the May 22 edition on ensuring AI compliance and security controls and the July 25 edition on how to outpace deepfake threats.


About the Author

Anna Delaney

Anna Delaney

Director, Productions, ISMG

An experienced broadcast journalist, Delaney conducts interviews with senior cybersecurity leaders around the world. Previously, she was editor-in-chief of the website for The European Information Security Summit, or TEISS. Earlier, she worked at Levant TV and Resonance FM and served as a researcher at the BBC and ITV in their documentary and factual TV departments.




Around the Network

Our website uses cookies. Cookies enable us to provide the best experience possible and help us understand how visitors use our website. By browsing govinfosecurity.com, you agree to our use of cookies.