Yeah, the other trial, the Alliance trial, I will explain that the design is a single-center randomized trial in Korean adults with breast or colon cancer starting at least three months of chemotherapy, were randomized one-to-one. In the intervention arm, the treating physician could consult Chat GPT whenever they felt the need. The control arm was standard care, and the same five physicians treated both arms...
Yeah, the other trial, the Alliance trial, I will explain that the design is a single-center randomized trial in Korean adults with breast or colon cancer starting at least three months of chemotherapy, were randomized one-to-one. In the intervention arm, the treating physician could consult Chat GPT whenever they felt the need. The control arm was standard care, and the same five physicians treated both arms. Leaving use to the physicians was a design choice. We wanted the trial to look like everyday practice, so we did not mandate use and did not script the prompt. The primary endpoint was quality of life, EORTC QLQ-C30, global health status. From baseline to the end of treatment, the pre-specified interim analysis was planned. There is a need to be safety and feasibility was planned at 60 patients, and the trial is what I prepared. This trial resulted in three findings. First, feasibility. For every patient in the LLM arm, the physician filled up a monthly report, and I explained the result of three findings. First, feasibility. For every patient in the LLM arm, the patient filled in a monthly report. And across 99 monthly reports, the tool was used in 47, fewer than half. When it was not used, the reason was recorded, almost always “not needed”. Second, safety. Under physician review, the output was rated appropriate or moderate, never inappropriate. There was no moderate or severe hallucination and no adverse event was linked to the health intervention. Third, the patient. More toxicity was recorded in the LLM, 5.4 events per patient against 2.8, mainly toxicities with no routine management. But those reductions did not differ and no quality of life benefit was shown. We stopped in no month after interim analysis, but feasibility, the implication for practice. Two things. Under physician review, clinician user of an LLM approach appeared safe in this trial, and it matters because clinicians are already doing it. But LLM support in the form the clinician asking when they felt the need did not make the patient better, and I would not expect the patient to have a benefit from simply giving clinician access. When I did it, something was the symptoms with no routine, whether it helps more, where specialists are few, and patients are many, is a hypothesis. Physician review is harder to come by there, so safety would be to be tested again. And lastly, I will explain the safety. How can we set the safety from the LLM using? On data and safety, we have three rules. De-identified information only can be entered, and a temporary chat mode only used with no link to the record and every output rated by the physician before any action. Adverse events were collected in both arms and for each event per physician judged whether it was related to the LLM.
This transcript is AI-generated. While we strive for accuracy, please verify this copy with the video.
