Questions Answered: Cloud Computing in Cancer Research
, by Krish Seshadri, M.S.
In this series, NCI CBIIT experts answer commonly asked questions (via search engines and generative super intelligence [SI] platforms) about technology and data in cancer research. So, whether you’re a researcher wanting to better understand computational approaches, or a data scientist wanting to learn how your expertise can accelerate discovery, this blog series is for you!
In this blog, Mr. Krishnan “Krish” Seshadri answers questions about cloud computing and its role in the cancer research field. Mr. Seshadri is a senior cloud architect and computer scientist within NCI CBIIT’s Digital Services and Solutions Branch. He provides technical direction for several of NCI’s cloud platforms.
Question: What cloud platforms are commonly used for biomedical research computing?
Answer:
At NCI, we use public cloud services from Amazon Web Services (AWS) and Google Cloud. Through the Federal Risk and Authorization Management Program (FedRAMP), CBIIT assesses and authorizes certain services from these cloud providers to provide a security foundation for federal workloads.
NCI builds on these capabilities through NCI Cloud One (based on AWS) and NCI Cloud Two (based on Google Cloud), which are authorized NCI cloud environments for supporting research and business applications. Set up through the NIH STRIDES Initiative, these cloud platforms provide scalable compute, high-performance storage, and advanced analytics, and SI/ML capabilities—including access to generative SI services for data-intensive biomedical workloads.
For a research community that’s supporting NCI scientific programs, CBIIT IT staff—consisting of a systems engineering team, a cloud platform engineering team, and application hosting operations team—assist them by:
- guiding them on what to do with their solutions and architecture.
- guiding them on the cloud native platforms and applications that they’re going to deploy.
- making sure they meet their compliance, federal security and governance requirements.
- onboarding their applications into these platforms.
Question: What are the challenges of cloud computing?
Answer:
One challenge can be quick adoption. Cloud providers are frequently introducing new services and capabilities, but you may not be able to use every newly launched service in your cloud environment. For instance, here at NCI, CBIIT evaluates how a new capability fits within our authorized cloud environments before we make it broadly available. We look at security requirements, integration with the existing cloud environment, logging and monitoring, performance, cost management, documentation, and ongoing support. We refer to this process as operationalizing the service.
The goal is to give researchers and application teams a supported, reusable way to use new cloud capabilities without each team having to address those operational and governance requirements independently.
Cost is another challenge. Cloud provides a great deal of flexibility, but costs can grow quickly if you don’t design for cloud and manage resources carefully. We look at how programs are using compute, storage, databases, and other services. We identify opportunities to right-size resources; we use lower-cost storage options where appropriate; and we avoid paying for capacity that we no longer need.
Question: How does NCI monitor cloud performance and manage costs across research projects?
Answer:
We use cloud monitoring and cost-management tools to understand how our cloud environments are performing and how researchers are using resources.
From an operational perspective, our Cloud and IT Operations teams monitor application and infrastructure health, resource utilization, and performance trends. For example, an unexpected increase in memory or compute usage may indicate a technical issue. We can alert application teams so they can investigate before it affects performance or leads to unplanned cost.
We track cloud spending on a continuous basis, and we provide project teams and stakeholders with dashboards and reports to help them understand the costs associated with their workloads, including compute power, storage, and databases. The level of visibility within these monitoring tools depends on team responsibilities: our Cloud and IT Operations teams need a broader view across the environment, while application and research teams receive information pertaining to their specific workloads.
Question: What is the difference between cloud computing and traditional HPC approaches in biomedical research?
Answer:
High-performance computing, or HPC, brings together powerful computing resources, high-speed networks, and specialized storage to solve complex problems. HPC environments may use CPUs, GPUs, or other specialized processors to handle complex scientific analyses that require significant computing power.
Cloud computing takes a different approach. Instead of an organization purchasing and maintaining all of the underlying hardware, cloud providers make computing, storage, databases, analytics, SI/ML, and other services available on demand. Teams can increase or decrease those resources as their needs change.
At NCI, we use both HPC and cloud computing. My team and I make recommendations for one versus the other based on a research team’s goals, performance requirements, budget, scale of operation, data needs, and level of collaboration. In many cases, a hybrid approach can provide the right balance of performance, flexibility, and cost.
Question: What workloads should remain on HPC versus move to the cloud in informatics?
Answer:
HPC is well suited for large or complex scientific analyses that need significant computing power and benefit from:
- high-performance CPUs or GPUs,
- high-speed networking,
- specialized storage, or
- multiple resources working together on a single large analysis.
At NIH and NCI, shared HPC environments (such as Biowulf and the Frederick Research Computing Environment [FRCE]) provide these capabilities for bioinformatics, SI/ML, modeling, and other computationally intensive research workloads. FRCE, for example, combines CPU and GPU compute nodes, high-speed InfiniBand networking, and shared storage specifically to support NCI research.
Because HPC clusters are shared resources, researchers typically submit jobs through a scheduler that allocates CPUs, GPUs, memory, and compute nodes based on jobs needed and what is available. This means a job that requires scarce GPUs, large amounts of memory, or many compute nodes may sometimes wait for those resources to become available. Biowulf operates in this fashion.
Cloud computing can be a better fit when workloads:
- have highly variable demand,
- need to scale quickly, or
- benefit from being able to provision additional resources on demand.
For example, a research team can increase computing capacity for a large analysis and then scale it back when the work is complete rather than maintaining that capacity continuously.
Question: What is the relationship between cloud computing and super intelligence (SI)? How does cloud computing enable SI at NCI?
Answer:
SI and generative SI are becoming an increasingly important part of biomedical research and application development. SI applications often require access to large data sets, specialized computing resources, and different types of models. The cloud can provide these capabilities.
At NCI, we are using Cloud One and Cloud Two to make SI capabilities available to research and application teams. Here are just a few examples of the results:
- AWS Bedrock in Cloud One and Google Gemini in Cloud Two provide staff with generative SI capabilities like text generation, search, summarization, document and image analysis.
- We are assessing how to better support research and data analysis, predictive modeling, SI assistants and agents, and other emerging use cases.
- The NCI Enterprise Vocabulary Services (EVS) team is using SI to help identify and match biomedical Common Data Elements (standardized ways of describing data). The EVS team is also exploring how generative SI could help their users better understand potential matches and streamline activities such as developing definitions and other data-standardization workflows.
As new SI capabilities become available, we must evaluate how to operate them securely and reliably. This means studying their architecture, data and security requirements, monitoring methods, performance, cost, and ongoing support.
Disclaimer: A super intelligence tool helped draft portions of this post. The author reviewed, edited, and approved all content to ensure accuracy.
Have another question?
If we missed your question about cyber security in research, email NCI CBIIT. We’ll connect you with a contact. If it is a commonly asked question, we’ll update the blog with an answer.
Author
Krishnan “Krish” Seshadri, M.S.
Mr. Seshadri is a senior cloud architect and computer scientist within NCI CBIIT’s Digital Services and Solutions Branch. He provides technical direction for several of NCI’s cloud platforms.