Artificial Intelligence (AI) is transforming technology, and mobile apps are no exception. Organizations across verticals and size are racing to embrace AI technologies to drive productivity, but surveys indicate many consumers remain suspicious of the technology. The security, reliability and privacy implications of AI experiences face a laundry list of questions:
- Are you using my data to train a model?
- Is my sensitive or proprietary data sent to a model that may leak it to other people?
- What data is retained, for how long, and can it be used to make a digital clone?
- Where is data stored, is that centralized, and can it be stolen by a bad actor?
- Should I bet on hosted and centralized AI or decentralized on-device solutions?
- What is your digital moat and how are you protecting or enhancing it with AI?
- Are you using AI correctly?
- Are you using AI safely?
- Can I trust AI generated results?
Risks Introduced by Generative AI
The embrace of AI looks different across organizations; from AI-generated code and copilots to chat-first experiences, mobile apps are leading the way for operationalizing Gen AI. But integrating AI experiences into mobile applications opens doors to security, privacy, reputation and regulatory risks.
- Integrations that lead to regulatory and compliance failure (contract disclosure)
The regulations around AI are still a work in progress but it is clear that disclosure for the use of AI in an app is table stakes. Organizations have updated purchase agreements to identify where AI is used in a solution, app stores are beginning to enforce best practices and regulators are working to update compliance requirements. Consequences range from fines and penalties, losing their place in app stores, to the complete closure of business lines that violate these rules. Product and development teams are challenged by the lack of clarity around these requirements.
As mobile app engineers embrace AI functionality through third party SDKs and open-source components, they may be introducing regulatory and compliance issues. This issue is critical to mobile app teams as third-party SDKs and APIs often compose 60% to 70% of a mobile app and as AI becomes embedded at the OS level for both Android and iOS devices through new operating systems like iOS 18.
CISOs and CTOs are posed with a real challenge. AI is transforming technology, but they also must recognize the functionality these third-party components bring, investigate how data is being processed, transmitted, and stored and what data is available for these AI models to train on.
It is critical that organizations dynamically test their mobile apps to understand what functionality in their apps may be leaking sensitive data locally, transmitting it in a way that is vulnerable to interception or connecting to unwanted or dangerous endpoints. Building a Software Bill of Materials (SBOM) and, especially for AI, a Machine Learning Bill of Materials (ML-BOM) is also important in understanding the third-party components included in mobile apps. As sustaining members of the Open Worldwide Application Security Project (OWASP), NowSecure researchers and engineers are already educating developers, architects, and organizations on the unique risks and governance requirements of large language models (LLMs).
- AI-generated code has lots of bugs and introduces common vulnerabilities
Gen AI has helped teams build mobile apps faster, but it has also increased the occurrence of security vulnerabilities. Training for AI models includes insecure code and as such, AI can also generate insecure code. This has led to over 40% more bugs in apps, which happens to correlate with the purported efficiency gains AI code generation enables! Development and security teams are faced with a growing risk as AI models continue to train on insecure code.
Running static analysis is a great way for security teams to quickly and easily identify security issues that may have been introduced by AI generated code. It is incredibly important, however, that teams do not stop there. Issues introduced by AI may only be identifiable at runtime or with manual investigation.
- Improper use of Gen AI platforms that store data in an insecure manner
According to a survey by the U.S. National Cybersecurity Alliance (NCA), 38% of employees share sensitive work information with AI tools without their employer’s permission. Employees looking to drive productivity may leak proprietary data to AI models that, depending on how the model is trained, can even end up in the hands of competitors. This presents two unique challenges for development, security and Global Risk & Compliance (GRC) teams.
The first challenge is understanding the risk presented by these AI tools for the apps that they build. In addition to the two points above, employees may be pulling open source or even unlicensed code (not the library but the code itself) into their application, hardcoding publicly available secrets into their apps or at worst, committing IP theft by unknowingly using code uploaded by another user. Understanding what data is used in training and how the AI model makes inferences is critical, and today those details are behind a controversial shroud.
Secondly, CISOs must protect themselves and their employees from AI-enabled mobile apps that may use data they upload for training. Take, for example, a scanning app used to generate PDFs from a photo. While a local AI model summarizing the sensitive content may be perfectly acceptable, an app that instead sends the images to a back-end service may potentially leak proprietary information. As a result, CISOs and mobile device managers must assess the risk each individual productivity app presents in terms of the access it has to confidential data and create a procurement process that performs AI-specific risk assessments for new tools and software.
Integrating AI experiences into mobile applications opens doors to security, privacy, reputation and regulatory risks.
Adopt a Programmatic Approach to Managing Mobile App Risk
CISOs and CTOs may feel they are faced with a choice: slow down the adoption of AI in favor of security or embrace AI and accept the risk to drive competitive advantage. NowSecure recommends to go beyond these two choices and instead adopt a programmatic approach to mobile app risk management that enables the use of new technologies and builds a system for evaluating the risks it presents.
The volume of code and velocity of development will continue to accelerate, which makes automated mobile application security testing more critical than ever. It’scritical that this testing notifies analysts about how AI is used throughout the mobile app portfolio. Identifying things like AI files included in the app package, AI libraries an app uses, AI services an app interacts with and even AI service-related API keys that may be hardcoded in the app package or retrieved at runtime provides important visibility about the safe, responsible use of AI in mobile apps. NowSecure Platform delivers that insight.
The threats presented by AI may require manual testing so pairing automated assessments with Mobile App Penetration Testing as a Service (PTaaS) is key. Development and security professionals may look to upskill to Navigate the Pitfalls of AI Generated Code or learn Security Best Practices When Using AI To Write Code.
Identifying and remediating the risks presented by third parties through mobile apps and SDKs is critical to a programmatic mobile app risk management approach. Manually investigate the security posture of the SDKs built into your app or present in your SDK provided to customers with NowSecure SDK Pen Testing and identify risks in the mobile apps in your ecosystem with NowSecure Mobile App Risk Intelligence. With NowSecure MARI, leaders and practitioners in procurement, security and enterprise mobile management can get a consolidated and consistent view of the security, safety and privacy risks associated with thousands of enterprise apps.
NowSecure has always partnered with customers, standards authors and innovators in mobile app security, privacy and compliance. Stay tuned to learn more about the use of AI in mobile app development and functionality, how to identify issues with it and how to properly secure it.
