IT and OT Qualification for AI and Traditional Technologies

Security framework

In Life Sciences companies, AI is being used for tasks such as automating manufacturing processes, optimizing clinical trials, analyzing patient data and several other critical applications for the pharmaceutical, biotechnology and medical products industries.

The IT (Information Technology) and OT (Operation Technology) infrastructure is something very broad in terms of the life cycle of medicines or medical devices. Basically, they are components and systems that support applications used in R&D, clinical trials, manufacturing, quality control laboratories, supply chain, and post-market monitoring.

Just like traditional validations, an important component for reliability, security, and adherence to compliance standards is the Qualification of IT and OT Infrastructure in GxP environments (product quality, patient safety, and/or data integrity).

This process involves a series of activities and documentation to validate that the IT and OT infrastructure meets regulatory requirements and defined quality standards to adequately support critical applications in terms of performance and data integrity.

In summary, the scope of infrastructure qualification refers to the hardware, software, and cloud services that make up the company's IT or OT infrastructure.

Security framework

In Life Sciences companies, AI is being used for tasks such as automating manufacturing processes, optimizing clinical trials, analyzing patient data and several other critical applications for the pharmaceutical, biotechnology and medical products industries.

The IT (Information Technology) and OT (Operation Technology) infrastructure is something very broad in terms of the life cycle of medicines or medical devices. Basically, they are components and systems that support applications used in R&D, clinical trials, manufacturing, quality control laboratories, supply chain, and post-market monitoring.

Just like traditional validations, an important component for reliability, security, and adherence to compliance standards is the Qualification of IT and OT Infrastructure in GxP environments (product quality, patient safety, and/or data integrity).

This process involves a series of activities and documentation to validate that the IT and OT infrastructure meets regulatory requirements and defined quality standards to adequately support critical applications in terms of performance and data integrity.

In summary, the scope of infrastructure qualification refers to the hardware, software, and cloud services that make up the company's IT or OT infrastructure.

The Connection between AI and Infrastructure Qualification: Ensuring Security for IT and OT Innovations

Businesses seeking to innovate through the use of artificial intelligence (AI) and machine learning techniques typically begin by organizing their data to lay the groundwork for advancement. Since many AI methods rely on data analysis, it is vital to appropriately structure information to create applications and models that can effectively utilize this data.

In today's digital age, data is often likened to the new "oil" due to its value. As a result, it is crucial to guarantee the security of this data stored within a company's infrastructure, whether physical or in the cloud. This is where Infrastructure Qualification comes into play, ensuring through documentation that the infrastructure is secure and capable of supporting the applications that rely on these services. Prioritizing risk management, this process focuses on what truly matters to safeguard the integrity of the data and the operations it supports.

One qualification, many implementations

A final GxP application depends on a good infrastructure to work properly.

If the infrastructure supporting multiple AI systems and traditional technologies is not qualified in a unified manner, each time an end application needs validation, its related infrastructure must be verified. In other words, when it is carried out in a unified manner, it can be referenced and there will be no need to repeat all the checks for each system.

More and more industries are using their data as a competitive advantage through the use of AI applications. However, there is a risk of security and privacy breaches if controls are not implemented appropriately.

Furthermore, an unqualified infrastructure can generate potential dangers such as data loss, data confidentiality, model theft, in the case of using AI/ML (Artificial Intelligence and Machine Learning), compliance, cybersecurity, among others.

Below we list some examples of critical impacts/scenarios that should be considered.

Model theft, applied to AI/ML:

Imagine that several training courses in machine learning focused on language models, also known as LLMs (1), were carried out, requiring significant investment from the company. One of the main associated risks is the possible theft of knowledge from these systems, through direct observation of their inputs and outputs, similar to the reverse engineering process.

This can happen through unauthorized access, copying, or extraction of proprietary models, which would result in economic losses and the potential exposure of sensitive information. Once replicated, the model can be inverted, allowing attackers to extract information about the features or training data. In short, attacks that expose your Machine Learning (ML) and GenAI (1) data can not only impact your business but also risk disclosing sensitive data, such as patient information. To mitigate these risks, it is essential to implement measures such as auditing changes, encrypting data at rest and in transit, as well as strict access controls, and implementing data masking (replacing sensitive data with fictitious data).

(1) Large Language Models (LLMs) like ChatGPT's GPT-4 are examples of Generative AI or GenAI. Generative AI is a term that describes any type of Artificial Intelligence model that generates content.

Accidental data exposure:

In the case of sensitive data, good practice is to use data masking, the objective of which is to replace sensitive data with fictitious or altered values, so that the data still appears real, but without revealing the original information.

It is important that data in transit and at rest is also encrypted. The purpose of encryption is to encode data into an unreadable format, which can only be encoded by whoever has the decryption key. This process ensures that data is unreadable if intercepted.

Data manipulation:

In almost all cases, raw data requires preprocessing and transformation before it can be used to build a model. Data curation (2) and labeling (3) are essential to maintaining data reliability and usability. Therefore, manipulating the data can introduce a risk of distorting the results.

The attacker could, for example, manipulate the labels of a fraction of training data. This can compromise model integrity and accuracy and is a concern for models used for decision-making in GxP (product quality and patient/consumer safety) environments.

Actions that could mitigate this risk include auditing changes, encrypting data at rest and in transit, and robust access controls.

(2) Curation: collection, cleaning, organization and preservation of data

(3) Labeling: involves tasks such as identifying raw data (images, text files, videos, among others) and associating one or more informative labels that provide context, allowing a machine learning (ML) model to learn from this data .

Data lineage:

Data can come from multiple sources and undergo multiple transformations throughout its lifecycle. Data transparency and usage requirements in AI training are important for risk management. To meet regulatory requirements, highly regulated industries must ensure clear visibility and traceability of data used for AI. Automatic lineage (journey tracking and data traceability) allows you to create automatic data flow trails, for example, for auditing purposes. Imagine a scenario where sensitive data needs to be deleted, and it is not possible to identify which applications consume that data.

Data Security

Data is the most important aspect of AI systems because it provides the foundation on which AI applications are built. Data security is fundamental and equally important to ensuring the security of machine learning algorithms. New technologies including cloud infrastructures and service models are increasingly being used, expanding the portfolio and scope.

You've probably heard terms like Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). These service models are collectively referred to as XaaS, which is an acronym that stands for any (X) solution that will be made available as a service. There are a multitude of other solutions that can be used to support relevant GxP AI applications, such as DBaaS (database as a service), FaaS (function as a service), FDaaS (foundation models as a service), STaaS (storage as a service).

Cloud computing introduces flexibility in resource capacity, but it also introduces new risks for regulated companies. These risks include:

  • Less or no control over the data center
  • Multiple vendors working together to provide the infrastructure
  • Less control over infrastructure
  • Less control over data
  • Less control over the services applied
  • Data and systems are outside the companies’ network

The need to maintain product and patient safety must be central to the supplier selection process. Risks need to be assessed and managed, and this assessment can vary widely between XaaS providers. An XaaS provider's knowledge of GxP regulations should be considered.

IT Infrastructure Control

Several important aspects must be checked for compliance against the IT and OT Infrastructure Qualification:

  • Change Control Management
  • Configuration Management
  • Security Management
  • Server Management
  • Network Management
  • Incident and Problem Management
  • Help Desk (also known as Service Desk in ITIL®)
  • Backup, Restore and Archive
  • Disaster recovery
  • Performance Monitoring
  • Supplier Management
  • Quality Assurance

When an external vendor hosts or manages some or all aspects of a regulated cloud or IT infrastructure, several components of the Quality Management System (QMS) must undergo assessment. These include:

  • Quality Manual
  • Risk Assessment
  • Document that clarifies how the company deals with data privacy
  • Datacenter Management and Monitoring, if applicable
  • Change and Configuration Management
  • Disaster Recovery Plan
  • Document Control
  • Training Procedure
  • Internal Audit Procedure
  • Emergency Response
  • Backup, Restore and Archive

Key topics that should be considered in IT and OT Infrastructure Qualification projects for relevant AI GxP applications

Below, we present an example of the 10 main risk scenarios that must be considered in a Functional Risk Analysis during IT and/or OT Infrastructure Qualification. Naturally, other risks also need to be included to ensure the robustness of the qualification process.

  • Insufficient control access;
  • Classification of missing data;
  • Poor data quality;
  • Ineffective storage;
  • Missing data monitoring;
  • Ineffective encryption;
  • Lack of data versioning;
  • Insufficient data lineage;
  • Lack of data reliability;
  • Legal data – need and ability for ML to “delete” specific data due to GDPR.

This and other scenarios, including the entire life cycle of AI infrastructure and applications, are covered within the GO!FIVE® validation software.

GO!FIVE® is a specialized software platform designed for the validation of systems, infrastructure, utilities, cleaning, equipment and processes, supporting various validation and qualification activities through agile and digital methods.

Offering extensive content and advanced functionalities, it allows the integrated validation of AI systems and traditional technologies, as well as the qualification of infrastructure in a seamless way.

IT and OT Infrastructure Qualification Cycle

The purpose of this article is to consolidate the main deliverables that must be foreseen in an IT and OT Infrastructure Qualification project.

Categorizing the system according to ISPE GAMP®5 can support the development of the qualification strategy appropriately.

Note: ISPE® (International Society of Pharmaceutical Engineering) developed the GAMP Guide (Good Automated Manufacturing Practice) to guide the Life Sciences community in developing robust validations/qualifications.

This content is not intended to define the strategy, however, here is a summary that can help guide your steps towards good infrastructure qualification:

Qualification Plan:
Usually includes responsibilities, life cycle activities, deliverables, necessary reviews and approvals, an overview of the IT Infrastructure and its components, training needs, initial risk assessment, etc.
Specifications and design:
May include requirements and specifications for servers and peripherals (hardware, operating system, availability, storage), clients¹, network, diagnostic and support tools, database, server room and cloud service configurations.

¹ clients provide users with access to shared resources and services (e.g. file servers, printers).
Risk Analysis:
Risk assessment is typically an iterative process, carried out during planning and specification as more information is obtained. This document seeks to survey risk scenarios, worst-case effects, determine the severity, probability, and detectability of the risk in order to measure its priority, and propose respective mitigations/controls.
Protocol, Script and Test Report:
Can verify installation, system integration, supplier evaluation, documentation verification (e.g.: system architecture, operation manuals, network diagram, cable list, etc.), environmental conditions ( e.g.: server room), network, connectivity, security settings, firewall assessment, antivirus, operating system, backup and recovery, cloud service settings, among others.
Traceability Matrix:
Tests must be traceable to requirements and risks where appropriate.
Qualification Report:
Upon successful execution of the installation and operation specifications and closure of any issues, a report must be written that confirms that all specified activities have been completed successfully, as well as confirming that all critical processes are described and implemented. They usually include information on open deviations/incidents, and references on how qualification will be maintained.

Take advantage of our experience

Through the experience of several projects carried out, FIVE has created a knowledge database that is constantly updated.

We provide a complete solution with experts and software to support projects anywhere in the world.

  • SaaS software for all validation and qualification disciplines (digital and Agile), including IT or OT Infrastructure Qualification Project
  • Specialists

Validation of Computerized Systems | Qualification of Production Equipment Qualification of Utilities | Qualification of IT or OT Infrastructure | Supplier Qualification | or |Data Integrity and Governance

If you want to know more about our services: click here

 

About the author:

Lílian Ribeiro is a chemical engineer, biomedical systems technologist, postgraduate in Integrated Management Systems and currently studying for an MBA in Data Science and Business Analytics. Lílian has over a decade of technical and commercial experience in the food, pharmaceutical, and healthcare industries. As an advocate for paperless validation, she is passionate about introducing efficiency and innovation into life sciences companies. Lílian's vast experience is fundamental in validation and qualification projects, encompassing VLMS, ERP, EQMS, automation (PW) and IT infrastructure qualification.

About the reviewer:

Silvia Martins is an electrical engineer with two decades of experience in the pharmaceutical, biotechnology and medical device sectors. She has received training in GAMP5 and FDA 21 CFR Part 11 in England, SAP® validation in Germany and has experience in data integrity and governance gained in Denmark. As CEO and co-founder of FIVE Validation, a company committed to simplifying compliance processes, Silvia is dedicated to streamlining and simplifying client procedures while maintaining high robustness and compliance.

References:

ISPE GAMP® Good Practice Guide: IT Infrastructure Control and Compliance (Second Edition)