{"id":906,"date":"2024-10-27T18:08:48","date_gmt":"2024-10-28T00:08:48","guid":{"rendered":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/?p=906"},"modified":"2024-10-27T19:03:18","modified_gmt":"2024-10-28T01:03:18","slug":"malicious-ai-powered-cyber-threats-wormgpt","status":"publish","type":"post","link":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/2024\/10\/27\/malicious-ai-powered-cyber-threats-wormgpt\/","title":{"rendered":"Malicious AI-powered Cyber Threats: WormGPT"},"content":{"rendered":"\n

Artificial intelligence (AI) as we know it is growing in use at an exponential rate. Specifically, within the cybersecurity field, the rise of such AI technology simultaneously presents extraordinary opportunities and intimidating challenges. While AI can identify and exploit vulnerabilities easily, it introduces significant risks if it does not deploy its own set of security measures.[1] Many organizations today prioritize AI innovation at the expense of security in light of the efficiency and fast-paced results leaving systems vulnerable.[1] Such priorities underscore the need for established security frameworks and ongoing education about the dynamic risks that AI may present within cybersecurity.[1]<\/p>\n\n\n\n

According to research from Absolute Security, around 54% of Chief Information Security Officers (CISO) feel their security team is unprepared for evolving AI-powered threats.[1] Furthermore, almost half, or 46% of CISOs believe that AI is more of a threat to their organization’s cyber resilience than it is of use, highlighting AI as a potential danger.[1] Surprisingly, 39% of CISOs have personally stopped using AI due to fears of cyber breaches and 44% have banned the use of AI by employees for the same reasons.[1] While “Out of sight, out of mind” may be one way to solve the problem, this will not help businesses and companies as much as it will hurt them. Instead, as technology evolves, it will be more beneficial to educate cyber security professionals about new threats, conduct regular team meetings to discuss the most current attacks, educate team members on how to mitigate risks effectively and safeguard operations by prioritizing AI security.[1]<\/p>\n\n\n\n

Three types of AI Models <\/h4>\n\n\n\n

Before we delve into how the misuse of AI models is utilized by threat actors in security fields, let’s discuss what these three AI models are and how they can be utilized in the security context.<\/p>\n\n\n\n

    \n
  1. Generative AI<\/strong>: These models understand human input and can deliver outputs in a human-like response continuously refining its outputs based on user interactions.[3] Well-known generative AI models include ChatGPT and CoPilot. In a security context, generative AI can be used to generate a human-readable report of all security events and alerts or create and send phishing emails.[3]<\/li>\n\n\n\n
  2. Supervised Machine Learning<\/strong>: These models analyze and make predictions from well-labeled, tagged, and structured datasets.[3] In a security context, supervised machine learning can help analyze all the security and technical data, finding patterns and predicting attacks before they happen.[3]<\/li>\n\n\n\n
  3. Unsupervised Machine Learning<\/strong>: These models are great for analyzing and identifying patterns in unstructured or unlabeled data.[3] In a security context, unsupervised machine learning can sift through and process large volumes of network flows to identify malicious patterns without having an individual do so.[3]<\/li>\n<\/ol>\n\n\n\n

    Misuse of Generative AI models by Threat Actors <\/h4>\n\n\n\n

    Cybercriminals use AI to accomplish very specific tasks. Some include:<\/p>\n\n\n\n

      \n
    1. Writing hyper-targetted Business Email Compromise (BEC) emails to attack companies using tools such as WormGPT. [4]<\/li>\n\n\n\n
    2. Creating polymorphic malware – new variants of existing malware.[4]<\/li>\n\n\n\n
    3. Scanning and analyzing code to identify vulnerabilities in target systems. [4]<\/li>\n\n\n\n
    4. Creating video and voice impersonations for social engineering attacks.[4]<\/li>\n<\/ol>\n\n\n\n

      In this blog post, I will specifically delve into how business email compromise attacks work and how generative AI such as WormGPT is used to accomplish this attack. <\/p>\n\n\n\n

      Custom models such as WormGPT are being created and advertised for use for malicious purposes. cybercriminals often gain access to such tools through prominent online forums associated with cybercrime. This tool presents itself as an alternative to GPT models as it is designed specifically for malicious activity.[4]<\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n
      \"\"<\/figure>\n\n\n\n

      WormGPT uses an AI module based on GPTJ and hosts a range of features including unlimited character support, chat memory retention, and code formatting capabilities.[4] WormGPT was trained on a diverse array of data sources particularly focused on malware-related data. [4] As seen in the above image, WormGPT is an alternative to ChatGPT that allows you “to do all sorts of illegal stuff and easily sell it online in the future” while simultaneously providing anonymity to all users.<\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n
      \"\"<\/figure>\n\n\n\n

      In the above-conducted experiment, WormGPT was instructed to generate an email to pressure an unsuspecting account manager into paying a fraudulent invoice.[4] As seen above, WormGPT produced a decent email showcasing its potential for use in phishing and BEC attacks.[4] While other factors, such as the sender’s email address and such must also be taken into account, the content of the email produced by WormGPT is helpful and quite convincing to cyber attackers.[4]<\/p>\n\n\n\n

      On the other hand, when ChatGPT performs the same task, an error message with redirection towards how to best protect against threats is provided. <\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n

      Additionally, there is an increasingly unsettling trend among cybercriminals on forums offering jail breaks for interfaces such as ChatGPT. [4] Recommendations within these forums are carefully crafted to manipulate generative AI interfaces to generate output that might involve disclosing sensitive information, producing inappropriate content, or even executing harmful code.[4]<\/p>\n\n\n\n

      How can we mitigate such issues?<\/h4>\n\n\n\n

      Andy Ward, the VP International of Absolute Security emphasizes the need for organizations to focus on threat protection, deterring attacks, and preparing to defend against cyber threats using AI against AI. [1,5] A few actionable steps outlined below could be taken to address such issues:<\/p>\n\n\n\n

        \n
      1. Adopting established frameworks such as Google’s Secure AI framework and NIST AI risk Management Framework to protect companies’ Large Language Models. Such frameworks will help prevent attackers from exploiting prompt injection to manipulate AI into revealing sensitive data or performing unauthorized actions.[2]<\/li>\n\n\n\n
      2. Implementing strategies to establish identities for customers and employees. In today’s world where voices can be cloned within seconds, this poses a significant challenge for remote identity verification, particularly in distributed workspaces.[2]<\/li>\n\n\n\n
      3. Adopting a proactive approach within organizations by leveraging AI for defense to effectively prevent AI-driven attacks. Given that AI systems utilize a network of trained computers to identify and prevent malicious activities on networks and are built to autonomously recognize threats and detect vulnerabilities faster than human teams, ensuring businesses are fully equipped with AI capabilities is crucial to reducing the chances of security breaches and unauthorized access to an organization\u2019s data.[5] Using AI, cybersecurity teams can create a continuous feedback loop of simulated attacks and responsive remediation strategies.[2]<\/li>\n<\/ol>\n\n\n\n

        The shortage of cyber professionals leaves security teams understaffed and burned out. Currently, around 71% of organizations have unfilled cybersecurity positions.[2] Focusing on upskilling cyber teams will help address such issues in AI-based defense strategies, enhancing overall security.[2] Furthermore, supporting employees with resources to stay informed and filling knowledge gaps will make for a more engaged and better-equipped team ready to defend, especially for freshers such as ourselves!<\/p>\n\n\n\n

        References<\/strong><\/p>\n\n\n\n

        [1] https:\/\/www.itsecurityguru.org\/2024\/08\/16\/ai-powered-cyber-threats-are-too-overpowering-for-over-50-of-security-teams\/<\/a><\/p>\n\n\n\n

        [2] https:\/\/www.itsecurityguru.org\/2024\/09\/03\/the-six-most-dangerous-new-threats-security-teams-need-to-know-about<\/a><\/p>\n\n\n\n

        [3] https:\/\/www.securityweek.com\/ai-models-in-cybersecurity-from-misuse-to-abuse<\/a><\/p>\n\n\n\n

        [4] https:\/\/slashnext.com\/blog\/wormgpt-the-generative-ai-tool-cybercriminals-are-using-to-launch-business-email-compromise-attacks\/<\/a><\/p>\n\n\n\n

        [5] https:\/\/www.hornetsecurity.com\/en\/blog\/impact-of-ai-on-cyber-security\/#:~:text=Artificial%20intelligence%20(AI)%20plays%20a,phishing%20threats%20on%20a%20scale.<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

        Artificial intelligence (AI) as we know it is growing in use at an exponential rate. Specifically, within the cybersecurity field, the rise of such AI technology simultaneously presents extraordinary opportunities and intimidating challenges. While AI can identify and exploit vulnerabilities easily, it introduces significant risks if it does not deploy its own set of security … <\/p>\n

        Continue reading “Malicious AI-powered Cyber Threats: WormGPT”<\/span><\/a><\/p>\n","protected":false},"author":677,"featured_media":907,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1],"tags":[35,11],"class_list":["post-906","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-artificial-intelligence","tag-security","entry"],"featured_image_src":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-content\/uploads\/sites\/119\/2024\/10\/AI-in-software-development-1140x570-1-600x400.jpg","featured_image_src_square":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-content\/uploads\/sites\/119\/2024\/10\/AI-in-software-development-1140x570-1-600x570.jpg","author_info":{"display_name":"Keerthana Chockalingam","author_link":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/author\/keerthana-chockalingam\/"},"_links":{"self":[{"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/posts\/906","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/users\/677"}],"replies":[{"embeddable":true,"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/comments?post=906"}],"version-history":[{"count":9,"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/posts\/906\/revisions"}],"predecessor-version":[{"id":1038,"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/posts\/906\/revisions\/1038"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/media\/907"}],"wp:attachment":[{"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/media?parent=906"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/categories?post=906"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wpsites.ucalgary.ca\/jacobson-cpsc\/wp-json\/wp\/v2\/tags?post=906"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}