A former director at the Mayo Clinic claims she was fired in retaliation for reporting the network's horribly inaccurate AI tools.
  • CovertOperative@piefed.zipEnglish
    193·
    2 days ago

    I do actually think AI can be very helpful in the medical field, for research in diagnostics or quickly writing reports, if responsibly done by professionals who can verify the output and with omission of patient identity.

    What happened here according to the article is that they had rules on how to use AI responsibly, and they broke them all for quicker profit and tried to hide their failures.

    • Zron@lemmy.worldEnglish
      7·
      1 day ago

      AI is horrible for charting.

      My wife is a Respiratory Therapist. They rolled out AI charting a few months ago and she tried it once. She spent just as much time proof reading the AI notes as it would have taken to write them herself.

      The main problems are that medical notes are critical to patient care, and are legal documents that can and will be used in the event of legal action. If you’re using AI to document patient care, you’re opening yourself up to potentially disastrous miscommunication that can very possibly get someone killed or have their life altered. And then if someone does die, now the court can see that you documented things incorrectly and could hold you responsible.

      • CovertOperative@piefed.zipEnglish
        22·
        2 days ago

        I’m well aware that CEOs are pushing the exact opposite of responsible AI use. But that doesn’t mean it is fully impossible to use the tool responsibly, which is also substantiated in your linked article:

        A small subset of participants though – less than 10% – worked differently and used AI as a tool to gather data that they then analysed themselves. These individuals made more accurate predictions than others participants and showed stronger brain activation too.

        Ming suggests that ultimately, the goal could be a form of “hybrid intelligence” where humans and machines “do the hard stuff” together. By this she means we need to think first and use tools to challenge us later, rather than simply letting them answer questions for us. Kosmyna agrees and suggests learning subjects without AI tools first to build a foundation and then think about using LLMs.

        It’s like dynamite. It can cause a lot of harm, but also make reasonable things like mining and demolition a lot easier. The difference is Alfred Nobel was smart enough not to distribute free samples on every street corner.

        • XLE@piefed.socialEnglish
          4·
          2 days ago

          I’m sticking to the kratom analogy because it, unlike dynamite, is produced by a facility somewhere, nobody can tell what’s inside of it, the composition can be changed by surprise, and there’s no actual rules or way to figure out what responsible usage looks like. Maybe 10% of people will be fine forever with the mystery tool. Maybe their lack of addiction is the glitch.