Zenity Launches GenAI Attacks Matrix to Guide Security Efforts for GenAI Systems, Copilots and Agents
Zenity Launches GenAI Attacks Matrix to Guide Security Efforts for GenAI Systems, Copilots and Agents
TEL AVIV-YAFO, Israel, Oct. 3, 2024 /PRNewswire/ -- Zenity, the leader in securing enterprise copilots and low-code development, has announced a new security framework, the GenAI Attacks Matrix. The open-source project, inspired by MITRE ATLAS and spearheaded by Zenity with help from many of the world's leading security researchers, is focused on attacks that target the users of various GenAI systems, examining how AI systems interact with and on behalf of their users, and vice versa.
以色列特拉維夫-雅法,2024 年 10 月 3 日 /PRNewswire/ — 保護企業副駕駛和低代碼開發領域的領導者 Zenity 宣佈了一個新的安全框架,即 GenAI 攻擊矩陣。這個開源項目受MITRE ATLAS的啓發,由Zenity在許多世界領先的安全研究人員的幫助下牽頭,專注於針對各種GenAI系統用戶的攻擊,研究人工智能系統如何與用戶進行交互或代表其用戶進行交互,反之亦然。
While many well-known security frameworks have historically taken an endpoint-driven approach, with the introduction of enterprise copilots and GenAI systems, security teams need a purpose-built framework to help them defend against the ensuing new wave of risks. This project's scope includes any system that uses GenAI, allows for GenAI to make decisions, and interfaces with or is operated by users (or on their behalf, in the case of agentic AI) and is built towards helping security practitioners understand and contextualize their risk. This explicitly includes licensable AI systems such as ChatGPT Enterprise, GitHub Copilot or Microsoft 365 Copilot, extensions and agents anyone can build with low-code/no-code tools, and custom AI applications built for specific use cases.
儘管許多知名的安全框架歷來都採用端點驅動的方法,但隨着企業副駕駛和GenAI系統的引入,安全團隊需要一個專門構建的框架來幫助他們抵禦隨之而來的新一輪風險。該項目的範圍包括任何使用GenAI的系統,允許GenAI做出決策,與用戶(如果是代理人工智能,則代表他們)進行交互或操作,旨在幫助安全從業人員了解風險並對其進行情境化。這明確包括ChatGPT Enterprise、GitHub Copilot或微軟365 Copilot等可許可的人工智能系統、任何人都可以使用低代碼/無代碼工具構建的擴展和代理,以及爲特定用例構建的自定義人工智能應用程序。
Zenity co-founder and CTO Michael Bargury, said, "What we're hoping to do here is bring the leading AI security researchers together in order to take a focused approach to GenAI systems. Our aim is to collectively document discovered attack techniques in order to clarify the threats to help enterprises devise corresponding mitigation and risk management strategies. AI changes every day, and it is critical that we share information about potential attacks as soon as they are discovered, before they are observed in the wild. I am proud to announce this project and look forward to collaborating with the security community."
Zenity聯合創始人兼首席技術官邁克爾·巴古裏表示:「我們希望在這裏做的是將領先的人工智能安全研究人員聚集在一起,對GenAI系統採取有針對性的方法。我們的目標是共同記錄已發現的攻擊技術,以澄清威脅,幫助企業制定相應的緩解和風險管理策略。人工智能每天都在變化,因此至關重要的是,一旦發現潛在攻擊,在野外觀察到潛在攻擊之前,我們就將其共享信息。我很自豪地宣佈這個項目,並期待與安全界合作。」
Bargury, who also founded the OWASP Low-Code/No-Code Top 10, realized that as the gold rush to place AI in the hands of all business users surges on, it is clear that security for AI is still a great unknown. By letting GenAI act on behalf of business users, enterprises have unwillingly opened up new attack pathways for adversaries to target powerful systems that inherently contain access to loads of corporate and sensitive data and are curious by nature. Attackers are exploiting these systems with promptware, which is content with hidden malicious instructions that gets picked up and acted on by AI apps.
巴古裏,他還創立了 OWASP 低代碼/無代碼前 10 名,意識到,隨着將人工智能交到所有商業用戶手中的淘金熱激增,很明顯,人工智能的安全性仍然是一個很大的未知數。通過讓GenAI代表商業用戶採取行動,企業不情願地爲對手開闢了新的攻擊途徑,讓他們瞄準強大的系統,這些系統本質上包含對大量企業和敏感數據的訪問權限,並且天生具有好奇心。攻擊者正在利用這些系統 提示軟件,其中包含隱藏的惡意指令,這些指令會被人工智能應用程序竊取並採取行動。
This project aspires to lay the foundation for security teams that need to adopt a defense-in-depth approach focused on malicious behavior rather than malicious static content. The primary goal of this project is to document and share knowledge of those behaviors and to look beyond prompt injection at the entire lifecycle of a promptware attack. For more information about joining and contributing to this project, check out the GitHub repository or learn more on our website.
該項目旨在爲需要採用側重於惡意行爲而不是惡意靜態內容的深度防禦方法的安全團隊奠定基礎。該項目的主要目標是記錄和共享有關這些行爲的知識,並在即時軟件攻擊的整個生命週期中超越即時注入。有關加入該項目和爲該項目做出貢獻的更多信息, 查看 GitHub 存儲庫 或在我們的網站上了解更多信息。
About Zenity
關於 Zenity
Zenity, the world's first application security platform for Enterprise Copilots and Low-Code development, protects organizations from security threats, helps meet compliance, and enables business continuity. Established in 2021, many of the world's leading organizations trust Zenity to help configure security guardrails, generate prioritized lists of vulnerabilities, and accurately pinpoint and remediate vulnerabilities by continuously scanning business-led development platforms and providing centralized visibility, risk assessment, and governance. Visit us at for more.
Zenity是世界上第一個用於企業Copilots和低代碼開發的應用程序安全平台,可保護組織免受安全威脅,幫助滿足合規性並實現業務連續性。Zenity成立於2021年,通過持續掃描業務主導的開發平台並提供集中的可見性、風險評估和治理,幫助配置安全護欄,生成漏洞優先級清單,並準確查明和修復漏洞。請訪問我們 欲了解更多。
SOURCE Zenity
來源 Zenity
WANT YOUR COMPANY'S NEWS FEATURED ON PRNEWSWIRE.COM?
想在 PRNEWSWIRE.COM 上刊登貴公司的新聞嗎?
譯文內容由第三人軟體翻譯。