The criterion can't be automated — but it can scale

There's a question that comes up more and more frequently in conversations with colleagues, clients, and industry references: what's left for the experience consultant when AI can do what used to take weeks?

My answer, after years of incorporating AI into the core of our practice, is this: exactly the hardest part. The criterion to know what to analyze. The judgment to distinguish a relevant finding from an anecdotal one. And the responsibility to interpret the data and decide what to do with it in the specific context of each client and each business.

What changed is not the nature of that work. What changed is the scale at which we can do it.

Where we come from

At EGO we've been doing digital experience consulting for clients in retail, banking, automotive, industry, and telco for over 15 years. Our practice has always been rigorous and demanding: understanding the client's ecosystem — which is rarely just a website, but a network of intertwined digital and physical touchpoints — mapping the real customer journey, identifying friction points that don't show up in analytics reports, and building a diagnosis that makes strategic sense for the business.

A complete Discovery process could extend over weeks. Not because we were doing it wrong — but because doing it right required time: interviews, behavioral analysis, dimensional review of the ecosystem, synthesis of findings, scorecard construction, prioritization of recommendations. That process was also what limited how many clients could access it.

We incorporated artificial intelligence into the process progressively, accompanying its maturity. Not as a technology bet but as a methodological question: what part of this work has clear rules and recognizable patterns? And what part requires something no model can replace?

The distinction that matters

AI performs well in the structured part of analysis. The part with defined dimensions, explicit criteria, documented benchmarks. What it can't do — and this is structural, not a temporary limitation that the next model will solve — is understand the specific context of a client. Why that problem exists in that organization. What historical decision produced it. How viable it is to solve given the team and the moment of the business.

An AI output is only as good as the conceptual framework from which it's evaluated. And that framework doesn't come from the model — it comes from years of accumulated practice.

We learned this in the most concrete way possible: by building.

How we built the FAE

The Full Assessment Engine wasn't born from a single decision. It was an iterative construction process that followed, almost without planning it, the logic of the double diamond: first expand, explore, build many things. Then contract, evaluate with criterion, keep only what works.

We developed conversational skills for different dimensions of analysis — ecosystem architecture, UX, SEO, CRO, accessibility, technical performance. Each skill was built on the frameworks we've been refining over 15 years of practice: not as generic prompts, but as analysis structures with weighting criteria, vertical reference thresholds, and proprietary prioritization logic.

We tested them against real cases. With the same criterion with which we'd evaluate any deliverable: is this useful? Is it actionable? Is it well-calibrated for the client's context? Or is it producing technically correct but strategically irrelevant findings?

Some engines performed well from the start. Others produced decontextualized results — analysis that didn't distinguish between a critical problem and a cosmetic one, recommendations that ignored the client's operational reality. Those we removed. Not because the technology failed, but because the methodological criterion said they weren't ready.

What survived that filter is what we productized.

What the engine does, and what it doesn't

The result is three engines in public beta, each oriented to a dimension of the digital ecosystem:

— Engine 01 — Site analysis: architecture, E-E-A-T, SEO, CRO, accessibility, analytics. The most complete diagnosis of web presence with vertical benchmark.

— Engine 02 — Digital CX analysis: brand presence in the ecosystem, touchpoints, voice of market, growth opportunities.

— Engine 03 — Technical diagnosis: stack, security, performance, technical debt, scalability, and compliance.

Each engine generates an executive report with scorecards by module, findings prioritized by business impact, effort-impact matrices, and a roadmap across three horizons. It goes one step beyond mechanical findings because the framework operating underneath has built-in criterion, not just rules.

And it has something no traditional audit tool can offer: every finding is traceable to a concrete observation taken at the exact moment of analysis. It's not a model estimate based on its training knowledge. It's what the server returned today, what the data shows this week, what Google measured at this instant.

But there's something the engine doesn't do, and it's worth saying clearly: it doesn't interpret the business. It identifies, measures, prioritizes within its framework. What it can't do is understand why that problem exists in that organization, what it means for that particular client's strategy, or how to communicate it in a way that generates internal action. That's still the consultant's work.

Why we're opening it up

We're in beta. And we say that without euphemisms — it means the engines are still maturing, every analysis that runs teaches us something, and there are dimensions we'll keep refining.

That's why we decided to open the three engines for free to anyone who wants to try them: CMOs, analysts, UX researchers, product managers, independent consultants, agencies. Not as a marketing strategy — as a genuine bet on building something better with the community that knows the subject best.

If you find a finding that doesn't make sense in your context, if the roadmap prioritizes poorly, if there's a dimension the engine isn't seeing — that's exactly what we need to know.

→ Try the engines with your digital ecosystem

Feedback is welcome and enriches us. If something you find doesn't make sense in your context, or if there's a dimension the analysis isn't capturing well, we want to know.

Write to me: bruno.borgoglio@egodesign.io