Enterprise AI infrastructure vendors are good at their jobs. The best ones have refined their pitches through hundreds of customer conversations, have answers ready for every standard objection, and know exactly how to frame their product to match the priorities you signal in the first five minutes of a meeting.
That's not a criticism. It's an accurate description of how professional sales works. The challenge for IT leaders and solution architects is getting past the prepared answers to understand the reality of what you're actually evaluating. These seven questions are designed to do exactly that.
Can you run your benchmark on my workload?
Standard benchmark suites measure hardware performance on standardised tasks. Your workload is not standardised. Ask vendors to run their performance demonstration on your specific model architecture, with your data characteristics, at your target scale. Vendors with genuinely competitive hardware will say yes. Others will explain at length why the standard benchmark is more representative. That explanation is its own kind of answer.
What's the failure mode at scale?
Ask the vendor to describe what happens when a GPU fails in a cluster running your workload. How is the failure detected? What's the recovery path? How much workload is lost or interrupted? This question separates vendors who have thought seriously about production operations from those who are optimised for the demo environment.
Who are your three most similar customers, and can I speak to them?
The quality of reference customers, specifically how similar they are to your organisation in scale, workload type, and operational maturity, tells you a great deal. Ask for three customers similar to you. If the vendor provides references from organisations that are significantly larger or have different workload profiles, ask why those are the best comparisons they can offer.
What does year three cost look like?
Ask the vendor to build a three-year cost model covering acquisition, maintenance, software licensing, power, and support. Vendors who engage seriously with this question are thinking about long-term relationships. Vendors who redirect to the acquisition cost are thinking about the deal.
What are you not good at?
This produces the most revealing answers in any vendor evaluation. Every product has genuine limitations. Vendors who acknowledge them honestly and explain what workloads or use cases their product is not the right fit for are demonstrating confidence and integrity. Vendors who claim to be the right answer for every workload are telling you something important about how they approach honesty.
What does your roadmap look like for the next eighteen months?
AI infrastructure technology is evolving rapidly. Understanding where a vendor is investing, and where they're not, helps you evaluate whether the product you're buying today will remain competitive for the operational life of your investment.
What happens if this doesn't work as expected?
Ask specifically about contractual remedies. What performance guarantees exist? What happens if the infrastructure doesn't meet the benchmarks in the proposal? What does the exit path look like if the relationship doesn't work out? How a vendor answers this question tells you a great deal about how they'll behave when you need their support most.
Every Tuesday in Hardware Hive: frameworks that put you in control of every vendor conversation. Subscribe free at hardwarehive.tech