Build a Decision Matrix With a Panel of AIs

Picking between options (vendors, job offers, feature bets, tools) usually collapses into a gut call or a messy doc. A decision matrix fixes that: list your options as rows, your criteria as columns, score each cell, weight the columns, and let the math rank them. The catch is that one person scoring one matrix carries that person's blind spots. Running the same matrix past a panel of models cancels some of that out.

What is an AI decision matrix?

An AI decision matrix is a weighted scoring grid where more than one model fills in the scores, then a judge reconciles them. You define the options and the criteria; each model rates every option against each criterion; you weight the criteria by importance; the weighted totals rank the options. Because several models score on their own, you see where they agree (trust the number) and where they split (look closer).

Here is the shape of a finished grid:

The numbers above are an illustration, not a recommendation. Your criteria and weights are yours to set.

How do I build one in aiDex?

Open aiDex and move through four modes in order.

1. Frame it in Solo. Use Solo with one model to turn a vague choice into a clean list: the options, the criteria that actually matter, and a weight (say 1 to 3) for each criterion. Lock this before any scoring so every model rates the same grid.
2. Score it in Compare. Switch to Compare and paste the options and criteria. Each model in the panel (for example Claude Opus 4.8, GPT-5.4, and Gemini 3.1 Pro) scores every option against every criterion at the same time, in its own column. You now have three reads instead of one.
3. Reconcile in Judge. Hand the three score sets to Judge. The judging model merges them into one matrix, averages or flags each cell, and calls out the criteria where the models disagreed most. Disagreement is the signal: it points you at the cells worth a human second look.
4. Write it up in Pipeline. Run the reconciled matrix through Pipeline to draft the recommendation, critique it, and polish it into a short memo you can send. The stages pass the work down on their own.

If the decision is a standing one (a quarterly vendor review, a hiring loop), keep the panel open in Teams so you can re-run the same matrix when inputs change.

Why score with more than one model?

A single model scores with a single set of priors, and it tends to agree with itself. Two or three models scoring the same grid surface the cells where the answer is genuinely contested rather than obvious. When all three rate a criterion the same, that score is solid. When they split three ways, you have found the real decision, the part that needs your judgment, not the model's. Browse the full roster in the Dex and pick models with different training lineages so they do not all share the same blind spot.

What does this cost to run?

You control it. Use your own provider keys or the ones we manage, and pick the models you want. Per-message costs are visible as you go, so a three-model matrix on a short option list stays cheap, and you can drop to two models or a lighter tier for low-stakes calls.

A decision matrix will not make the choice for you, and it should not. It makes your reasoning explicit, scores it from more than one angle, and shows you exactly where the models disagree so you spend your attention there. That is the whole point of a multi-model workflow: not one oracle, but a panel you can audit. For adjacent patterns, see running a strategy table and getting a consensus answer.