How to evaluate renting compute for AI workloads
Decide whether to rent compute for AI only after you write whether the workload is inference or training, how many hours you expect, where the data sits, and how the compute and the data leave when the rental ends. Leave the decision blank until those four are written.
On this page
Evaluate compute apart from general equipment
Whether to lease or buy a piece of equipment starts from how long it will be used, when cash leaves, who keeps the asset at the end, and who maintains it. Those four are covered in How to choose whether to lease or buy equipment. This note covers only the four items to write before renting compute for inference or training: the workload, the hours, where the data sits, and the exit path.
How to keep records of rented assets, contracts, schedules, maintenance, and returns is covered in How to choose an equipment rental management system. Equipment can be bought, leased, or taken forward with development in the same project. That is the block Buy, lease, or plan it with development on the homepage. This note does not rank providers, and it does not fill in a price.
The public page defines cloud computing
NIST SP 800-145, The NIST Definition of Cloud Computing, was published in September 2011. The document history marks 28 September 2011 as the final. The authors are Peter Mell and Tim Grance. The page gives the DOI 10.6028/NIST.SP.800-145. The abstract says cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (for example networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.
The same passage says this cloud model is composed of five essential characteristics, three service models, and four deployment models. The page does not name those characteristics, service models, or deployment models, and it does not mention graphics processors, prices, or service levels. This note does not fill the names in, and it does not add a provisioning speed the page does not state. The four items below are the list a buyer writes before deciding whether to rent compute.
Write inference or training first
Inference names where the chosen model sits, what kind of request comes in, and which result the system keeps. Training names which dataset is used, which run it is, and which artifact is kept at the end. “An AI project”, with neither of those, leaves the workload blank.
The two can happen one after the other. The record still has two rows. This note does not fill in a model name or a requests-per-second figure, and it does not write a speed you have not measured, with a date.
Hours have to point at one record
Hours come from one of two sources: the start and end of a past run, or a planned window that already has a date. With no past run, mark the planned window as an assumption and write the date you wrote it down. An assumption with no date leaves the hours blank.
Also name who stops the rental when the window ends. This note does not turn hours into money. If you record an amount, it comes from that quote, with the quote date. This note does not fill one in.
Write where the data sits today
Name the system where the training or inference data sits today, whether a copy must leave that place, and who may approve the copy. If the data cannot leave, write how the rented compute can use it without a copy. If that sentence cannot be written, the data location stays blank and the decision stays blank.
This note does not state that a region or a provider keeps data in a given place. If you write that sentence, it comes from the page that provider publishes, with the date you read it. Until that date is written, leave the region out of the decision.
Write what you keep and what is deleted
The exit path names four things: the artifact you keep when the rental ends (a model file, a log, or an output), where that artifact is stored, who deletes the copy on the rented side, and the date someone checks that the deletion is done. Leave any of the four blank and the exit path stays blank.
The general equipment note covers who keeps a physical asset at the end. This note covers this compute session and the data copy. A written window, with no exit path, is not yet a decision to rent.
Use the rental fields you already keep
If you already record rentals, put this compute line on the same fields where they match: what was rented, the contract, the window, and who ends it. Do not open a second book for fields that already match. Hours, data location, and the exit path are extra lines on that same record, not a reason to build another system.
Leave a field blank when it does not match, and name which item failed to match. This note does not require a new rental system before the evaluation can be written.
When to leave the rental undecided
When the workload is still “an AI project”, the hours have no date, the data location is blank, or the exit path does not name who deletes and who checks, do not rent yet. Write the blank item first. Stock or a time limit on a provider page stays out of the decision until you record it with the date you read the page.
This note does not state a price, a quantity available, a service level, or a ranking of providers. The NIST page was published in September 2011, and the final is marked 28 September 2011. Until the four items are written, check the evaluation by whether the workload is inference or training, whether the hours point at one dated record, whether the data names where it sits today and whether a copy may leave, and whether the exit names what you keep and who checks the deletion.