{"id":20376,"date":"2025-06-17T06:00:28","date_gmt":"2025-06-17T06:00:28","guid":{"rendered":"https:\/\/uxmag.com\/?p=20376"},"modified":"2025-06-18T04:43:22","modified_gmt":"2025-06-18T04:43:22","slug":"friendly-but-flawed-how-ais-agreeableness-risks-undermining-ux-thinking","status":"publish","type":"post","link":"https:\/\/uxmag.com\/articles\/friendly-but-flawed-how-ais-agreeableness-risks-undermining-ux-thinking","title":{"rendered":"Have SpudGun, Will Travel: How AI’s Agreeableness Risks Undermining UX Thinking"},"content":{"rendered":"\n

Where to start?<\/p>\n\n\n\n

I\u2019ve been discussing the failings of LLMs and other AIs in UX Research, my (supposed) field, for quite some time now. People and orgs far above my pay grade have been doing a better job of this (insert a Google search here for every actual usability and UX group has published on it, which all reveals massive issues, I am lazy) but I personally have enough of an ego to think I can contribute something that hasn\u2019t been discussed enough.<\/p>\n\n\n\n

My graduate education consisted of both finding happy hour beer deals and understanding human decision making, and I immediately saw some of the pitfalls of consumer LLM-based as it came on the scene- namely, that it is reliant upon human raters and contributors as a source. The basic issue there is that people communicated with their AI in development and rated its responses based on their desires. Desires, however, are not generally related to personal judgement, and people love AIs that are confirming and certain- frankly, all humans love to be confirmed in their predisposed beliefs. For evidence there, scrape the cesspit of true professional community, LinkedIn, and look at the bulk of \u201cAI Experts\u201d who likely a few years back were \u201cBlockchain Evangelists\u201d and probably had \u201cGuru\u201d or \u201cNinja\u201d as a title in the past decade, and see their justifications on why the LLMs they clearly do not understand are revolutionizing the world, criticism is invalid, and you should pay them to explain it to you.<\/p>\n\n\n\n

Now, to be clear, I am not some kind of hyper Luddite in the world of AI. Reporting, some kinds of analysis, lots of tasks are wildly assisted by AI, and advancements have made me wildly more productive- namely as when my brain goes \u201cuuuuggghh why do I have to write out this crap\u201d, I can ask AI to do it for me. I think AI in UXR should in general be treated as a Research Assistant, the glorified gopher, note taker and copywriter of the research world borrowed from academia, who enables massive productivity but can\u2019t be trusted for a moment to wipe their own butt without oversight, and would in fact be concerned for it swallowing up those roles for rank newcomers, had I seen any of those positions listed over the last decade. But I am decrepit enough to have been taught that research should be conducted as at least pairs of researchers (a hilarious notion when we\u2019re lucky that any organization will have a single researcher instead of just winging it and quoting Steve Jobs or something), so I think it\u2019s basically a net positive there.<\/p>\n\n\n\n

However, instead, I have seen so many references, again in that birthplace of true intellect, LinkedIn, to \u201csynthetic users\u201d or \u201cAI persona\u201d as an amazing resource for piloting or beginning a \u201ctrue\u201d research effort. This, like most \u201cit\u2019s nearly as good as the real thing and it\u2019ll start you off!!\u201d products, actually ends up being considered \u201cgood enough\u201d by nearly anyone who doesn\u2019t like spending money on research, which, from my illustrious career, is basically everyone. And that\u2019s a huge problem. Because most of this shouldn\u2019t even be a starting point.<\/p>\n\n\n\n

The basic issue here comes from a few places. LLMs and similar models are not trained to be accurate- they are trained to give you the answers you seek, no matter how stupid or pointless, or how important. For example, one of the most fun uses of the technology is things like music composers whose primary purpose is clearly to make reasonably composed pieces on incredibly stupid topics, which is funny. Now, I\u2019m all for the delightful misuse of technology, and no one can complain that modernity is free from new opportunities for comedy- NFTs were at one point promoted and justified, for example- but let\u2019s move that a little forward and consider that the same technology that allows for you to ask for a sad baroque\/ jazz piece about how you\u2019re having a difficult time farting, is the one that people are asking to understand users of their product. So, how does that work out?<\/p>\n\n\n\n

As a simple test, I decided to throw together a little experiment, replicating most \u201csynthetic user\u201d scenarios, but with a potential product that is completely moronic. How \u201creal\u201d are the responses? Can this, indeed, be a \u201cstarting point\u201d for the strapped entrepreneur? Can we use this to pilot our protocols? Is the AI about to take er jeerrrbs?<\/p>\n\n\n\n

Let\u2019s see. This was nearly entirely done in ChatGPT 4o- feel free to mirror every LinkedIn \u201cexpert\u201d and flail about with other models and different queries until you get the response that conforms to your objectives, but you\u2019ll see, I think this is pretty fair.<\/p>\n\n\n\n

\"\"
Image by Saul Wyner<\/a><\/em><\/figcaption><\/figure>\n\n\n\n

Here we go.<\/p>\n\n\n\n

\n\n\n\n

Primary personas for your mashed potato startup<\/h2>\n\n\n\n

1. The Comfort Craver (aka \u201cMash Enthusiast\u201d)<\/h3>\n\n\n\n

Motivation<\/strong>: Pure joy. Mashed potatoes are their love language.<\/p>\n\n\n\n

Needs<\/strong>: Easy access to hot, creamy potatoes at any time, with customization (gravy? chives? vegan butter?).<\/p>\n\n\n\n

Behavior<\/strong>:<\/p>\n\n\n\n