Share from 0:00
0:00
/

Are humans more deterministic than a Large Language Model?

๐—œ ๐˜„๐—ผ๐˜‚๐—น๐—ฑ ๐—ฎ๐—ฟ๐—ด๐˜‚๐—ฒ ๐˜๐—ต๐—ฎ๐˜ ๐˜๐—ต๐—ฒ๐˜† ๐—ฎ๐—ฟ๐—ฒ ๐—ป๐—ผ๐˜.

I've used large language models to bring a great deal of precision and consistency to the qualitative phase of #JTBD research. While a slightly different ๐—ท๐—ผ๐—ฏ ๐—บ๐—ฎ๐—ฝ, or ๐—บ๐—ฒ๐˜๐—ฟ๐—ถ๐—ฐ-๐˜€๐—ฒ๐˜ will be produced when running the same inputs multiple times, they are basically consistent in scope. Just slight variations in language.

Humans, on the other hand, introduce a great deal of variability via bias, and are heavily influenced by factors that they don't bother to document for later analysis.

๐—œ๐˜€ ๐˜๐—ต๐—ถ๐˜€ ๐—ฎ ๐—ฝ๐—ฟ๐—ผ๐—ฏ๐—น๐—ฒ๐—บ?

Directionally, probably not. But, if you ๐˜€๐—ฒ๐—ฒ๐—ธ ๐—ฝ๐—ฟ๐—ฒ๐—ฐ๐—ถ๐˜€๐—ถ๐—ผ๐—ป in your research and also demand an audit trail of the thought process, there's nothing better than an LLM at this point in time - as long as you preserve your prompts and inputs.

โœ… It's orders of magnitude faster

โœ… It's orders of magnitude more consistent

โœ… It's orders of magnitude more scalable

โœ… It's orders of magnitude less expensive

โœ… It's orders of magnitude less invasive of your daily routine

As a result, you can pivot your thinking and scoping before talking to anyone in the field. How many times has a stakeholder asked you to pivot after qualitative interviews, or worse, after fielding a survey?

Practical Innovation w/ Jobs-to-be-Done is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

No, LLM outputs aren't deterministic like the outputs from code is expected to be, but humans were far less deterministic when executing tasks before we had software or assembly lines. No two people did things exactly the same way. There was always nuance - we called this craftsmanship ๐Ÿ˜‰.

Nothing's perfect. No one is perfect. Striving for perfection misses the point and potentially wastes resources by overserving an audience that wants to get the job done better, with less friction, cost, and time.

Are you looking for a differentiated approach to market research? One that has more predictable outcomes and can now be executed faster, and cheaper?


If you'd like #AI accelerators that will set you apart - even if you don't use the formal #JTBD approach - you can get them here

How to plan your JTBD research

How to select the right market to study

Educate yourself and your colleagues with the JTBD Strategy Stack - https://community.zeropivot.us/courses/offers/adc27a2e-9e72-4404-97f2-112cc089f437

Fast track your Journey Analysis with this toolkit

Or, feel free to schedule some time to discuss how my partners and I can help you

~MikeB

Discussion about this video

User's avatar

Ready for more?