GPU Cloud RFP Template: AI Infrastructure Requirements

How to run a GPU Cloud RFP

A GPU cloud evaluation comes down to two questions: can the provider deliver the GPUs you need, where and when you need them, and will that capacity perform and stay healthy under your workloads. The example schedule below runs about 10 weeks from kickoff to contract award; if you need a large reserved cluster with a future delivery date, start capacity conversations earlier and allow for acceptance testing after signature.

1. Characterize your workloads and size the demand (weeks 1-2). Separate training, fine-tuning and inference, and for each one record the model sizes, the largest single job in GPUs, expected GPU-hours per month, data volumes and latency targets. Write down the numbers vendors will price against: [GPU models required], GPU counts, regions, start date, term, storage capacity and expected data transfer out.
2. Assemble the buying team. Include ML research and engineering leads, the platform or infrastructure team that will run the clusters, networking, security, privacy and data protection, finance, procurement and legal. Bring finance in early: decide whether you can commit to reserved capacity for a term, and how GPU costs will be charged back to teams.
3. Choose the capacity model, and decide whether you need an RFI (weeks 2-3). Decide how much demand should be reserved and how much can run on demand or on interruptible capacity, and whether you need bare-metal nodes, managed Kubernetes, managed Slurm or several of these. If you haven't compared renting with buying your own cluster, do that first. If you don't know which providers can deliver your GPU models in your regions by your start date, issue a short RFI asking for available capacity, lead times, access models and pricing structure, and use the answers to build a shortlist.
4. Tailor and issue the RFP (weeks 3-6). Delete requirements that don't apply, adjust priorities, and add your workload details to Section 1. Describe a reference training job and an inference workload you will use in the proof of concept, so vendors can plan for it. Give vendors three weeks to respond, with a written question-and-answer window in the first week.
5. Score the responses (weeks 6-7). Have each evaluator score independently against the rubric before comparing notes, and set aside any vendor that misses a Must-have. Check every 'yes' for whether the capacity is installed and running today or still to be delivered, and whether each feature is generally available or still on the roadmap.
6. Run a proof of concept with two or three vendors (weeks 7-9). Run your reference training job across several nodes on the same GPU models and network fabric you would be buying, not a demo environment. Measure how throughput scales as nodes are added, data loading and checkpoint times, and inference latency under realistic load. Ask the vendor to show what happens when a node fails mid-job, and log how quickly support answers your questions.
7. Negotiate and award (weeks 9-10). Use the pricing table to compare like for like. Settle capacity delivery dates and remedies, acceptance testing before billing starts, node replacement times, commitment flexibility, renewal caps and exit terms before you sign, not after.

GPU Cloud RFP requirements

Each line is written to paste straight into your RFP. Must-have means a vendor that cannot meet it is out; Nice-to-have earns extra points. Change the priorities to fit your situation, and delete what doesn't apply.

Functional

Technical and architecture

Integration

Security and compliance

Implementation and support

Commercial and pricing

Questions to ask GPU Cloud vendors

These questions are designed to separate vendors, not to collect brochure answers. Ask for evidence: a demo, a document or a reference.

1. For [number] [GPU models required] in [regions], what capacity can you commit, and from what date? Is that capacity installed and running today, or still to be delivered? What remedies do you offer if delivery slips?
2. Is our reserved capacity dedicated to us or allocated from a shared pool? Can it be reclaimed or reduced for any reason during the term?
3. Describe the network fabric a [number]-GPU training job would run on: the technology, per-GPU bandwidth, topology and switch tiers. Is it non-blocking at that size? If not, where is it oversubscribed?
4. Will you run our reference training job across several nodes of the exact hardware and fabric you are proposing during the proof of concept, and share the results and the configuration used?
5. How do you detect a failing GPU, a memory error or a fabric fault? Walk us through what happens to a running multi-node job, from detection to a replacement node back in service.
6. What burn-in and acceptance tests do you run before handing over a cluster? Will you share the results, and can we run our own tests before billing starts?
7. Who controls GPU driver, firmware and library versions? Can we pin versions, and how much notice do we get before maintenance that would interrupt a long training run?
8. What read throughput can your shared storage sustain to every node in our cluster at once, and how long does it take to write a checkpoint of [checkpoint size]? Show both in the proof of concept.
9. How are tenants isolated on shared nodes, the network fabric and shared storage? What happens to local disks and GPU memory before a node we used goes to another customer?
10. Where are our data, checkpoints, model weights, logs and backups stored? Where can your support staff access them from, and how is that access approved and recorded?
11. Show a sample monthly bill for a workload like ours, with every line item: GPU-hours, storage, networking, IP addresses, egress, managed services and support. Which charges are metered and which are fixed?
12. Can we convert a reservation to a newer GPU generation, move it to another region or change its size during the term? On what terms?
13. Which GPU models, regions and features in your proposal are generally available today, and which are in preview or on the roadmap? Give dates for the roadmap items.
14. At the end of the contract, how would we move [data volume] of data, checkpoints and model weights out, how long would it take, and what would it cost?
15. Provide a reference customer running multi-node training at a similar scale on the GPU models you are proposing.

Scoring rubric

Score each criterion from 1 to 5, multiply by its weight, and add the results. The weights below are a starting point; agree on your own before any proposals arrive so the scoring can't be bent around a favorite.

Scoring scale

• 5 Exceeds: meets every Must-have and most Nice-to-haves, shown in a demo or proof of concept, with a matching reference customer.
• 4 Strong: meets every Must-have and some Nice-to-haves, with clear evidence.
• 3 Adequate: meets most Must-haves; gaps have a credible workaround or a dated roadmap commitment.
• 2 Weak: misses one or more Must-haves, or the answer is vague.
• 1 Poor: does not meet the requirement, or no answer.

Frequently asked questions

What should a GPU cloud RFP include?

The GPU models, counts, regions and start dates you need, and how much capacity must be reserved; requirements for cluster networking, storage, scheduling, isolation, data residency, observability and node failure handling, each marked Must-have or Nice-to-have; inference serving if you'll use it; support and SLA terms; an itemized pricing table covering GPU-hours, storage and egress; exit terms; and the rubric you'll score with. This template includes all of them.

Should we reserve GPU capacity or use on-demand?

Reserved capacity guarantees specific GPUs for a term in exchange for a commitment, which suits steady training and inference demand. On-demand capacity needs no commitment, but there's no guarantee the GPU model you want will be available when you need it. If your demand is steady, consider reserving your baseline and covering peaks with on-demand or short-term capacity, and ask each vendor to quote both so you can compare them at your expected utilization.

What is a non-blocking network fabric for GPU clusters?

In a non-blocking fabric, the links between switch tiers carry as much bandwidth as the links into them, so every GPU can exchange data with any other at full speed at the same time. In an oversubscribed fabric, traffic that crosses switch tiers shares less bandwidth, which can slow multi-node training jobs. InfiniBand and RoCE (RDMA over Converged Ethernet) fabrics can be built either way, so ask each vendor whether the fabric is non-blocking at your cluster size and, if not, where it is oversubscribed.

How is a GPU cloud RFP different from an IaaS RFP?

A general IaaS RFP covers compute, storage and networking for a broad range of applications. A GPU cloud RFP focuses on what AI workloads need: guaranteed access to specific GPU models, an RDMA network fabric for multi-node training, storage that can keep GPUs fed and absorb checkpoints, GPU-aware scheduling and fast replacement of failed nodes. If you're buying both, run them separately or add these requirements as a section of your IaaS RFP.

How long does a GPU cloud RFP take?

The example schedule in this template runs about 10 weeks from kickoff to contract award, including three weeks for vendor responses and about three weeks for a proof of concept. A large reserved cluster may also have a delivery lead time after signature, plus time for acceptance testing, so ask each vendor for both in the RFP.

Further reading

• Buy or Rent GPUs? The 2026 Break-Even Math
• What an On-Prem AI Cluster Really Costs in 2026

Related RFP templates

• LLM Gateway RFP Template (Free): AI Platform Requirements
• MLOps Platform RFP Template
• Data Science and Machine Learning (DSML) Platform RFP Template
• Infrastructure as a Service (IaaS) RFP Template