How AI makes people confidently wrong

Machine Society – The accuracy of people using AI drops by one-third, but their certainty that they’re right more than doubles. New research published last week reveals the mechanism behind the workplace pandemic of people who are overconfident and underqualified. The mechanism is fluency-based substitution: The ready availability of a syntactically coherent, decisive AI answer is accepted for its surface plausibility rather than because they fact-checked or verified the information (a process called “cognitive surrender”). The nature of AI answers lowers the threshold at which people judge they “know enough,” reducing a healthy suspension of judgment and inflating confidence in unverified information. This dynamic partly explains the related pandemic of slop zombies. As I told you in a recent piece, slop zombies are “people who use AI chatbots for everything. They bludgeon you with massively long, detailed, mediocre reports, messages, emails, and slide decks they didn’t even read (but expect you to read). They illustrate everything with AI slop and marvel at their own creativity.” In that piece, I claimed that “slop zombies get dumber and lazier while their confidence rises. They believe the ideas and work of the AI chatbots are their own.” Slop zombies delegate skepticism and judgment to the AI, which is often confidently wrong. Users then believe that the AI’s “knowledge” is their own…”

Posted in: AI, Internet, Knowledge Management