Where the human-AI workflow in health care breaks down

kAm&]$] 962=E9 :?DFC6CD AC@46DD k2 9C67lQ9EEADi^^HHH]HD;]4@>^962=E9^962=E942C6^962=E9\:?DFC2?46\56?:2=D\7:89E\324<\f_2`bag6Qm>@C6 E92? d 3:==:@?k^2m A2J>6?E 4=2:>D 6249 J62C[ 244@C5:?8 E@ r|$ 2?5 z2:D6C u2>:=J u@F?52E:@? 52E2] pE 2 =2C86 A=2?[ 2? px\2DD:DE65 DJDE6> >2J 86?6C2E6 E9@FD2?5D @7 C64@>>6?52E:@?D 2 52J @? 4=2:> 56?:2=D[ 6=:8:3:=:EJ 7=28D 2?5 6?4@F?E6C\52E2 6CC@CD[ 6249 BF6F65 7@C 9F>2? C6G:6H H:E9 2 4@?7:56?46 D4@C6 2?5 2 DF886DE65 24E:@?] #6G:6H6CD 7@4FD @? E96 >@DE 4@>A=6I 42D6D[ 4964< =@H6C\C:D< C64@>>6?52E:@?D 7@C :CC68F=2C:E:6D 2?5 2AAC@G6 E96 32E49] %96 DJDE6> C64@C5D 2 ?2>6 2?5 2 E:>6DE2>A] ~? A2A6C[ 2 9F>2? H2D :? E96 =@@A]k^Am

k9bmp 9F>2? :D :? E96 =@@A] (92E 4@F=5 8@ HC@?8nk^9bm

kAm“wF>2? :? E96 =@@A” D@F?5D =:<6 8@G6C?2?46] %@@ @7E6?[ 2E 9:89 G@=F>6[ E96 D:8?2EFC6 2E E96 6?5 @7 2 C6G:6H BF6F6 :D =6DD @G6CD:89E 2?5 >@C6 2FE@A6?[ 2D k2 9C67lQ9EEADi^^4FC6:D]4@>^QmrFC6x$ w62=E942C6k^2m 6I2>:?6D 96C6]k^Am

kAmp yF?6 a_ae A:646 :? k2 9C67lQ9EEADi^^HHH]E96=2?46E]4@>^;@FC?2=D^=2?46E^2CE:4=6^!xx$_`c_\efbeWaeX__a_c\f^23DEC24EQm%96 {2?46Ek^2m @3D6CG65 E92E 9F>2? :? E96 =@@A @7E6? 7F?4E:@?D 2D DJ>3@=:4 C62DDFC2?46 C2E96C E92? DF3DE2?E:G6 AC@E64E:@?] x? ~4E@36C[ 2 (@C=5 t4@?@>:4 u@CF> 2CE:4=6 k2 9C67lQ9EEADi^^HHH]H67@CF>]@C8^DE@C:6D^962=E9\2?5\962=E942C6\DJDE6>D^962=E942C6\2:\D276EJ\9F>2?\:?\4@>>2?5\2AAC@249Qm2C8F65 7@Ck^2m 2 “9F>2?\:?\4@>>2?5” >@56=i |2A H96C6 72:=FC6D 42? @44FC[ 56E64E F?46CE2:?EJ[ 2AA=J D2768F2C5D 32D65 @? C:D<[ 6D42=2E6 4@?D6BF6?E:2= 42D6D H:E9 E96 C:89E 6G:56?46 2?5 =62C? 7C@> @FE4@>6D] %96 p>6C:42? |65:42= pDD@4:2E:@? k2 9C67lQ9EEADi^^HHH]2>2\2DD?]@C8^AC24E:46\>2?286>6?E^5:8:E2=\962=E9^8@G6C?2?46\7C2>6H@C<\286?E:4\2:\2DDF>:?8\4=:?:42=\7F?4E:@?DQm>256 2 D:>:=2C A@:?Ek^2m :? :ED AC@A@D65 7C2>6H@C< 7@C px DJDE6>D :? 4=:?:42= D6EE:?8Di pFE9@C:EJ D9@F=5 56A6?5 @? 6G:56?46[ 567:?65 4@?5:E:@?D @7 FD6 2?5 4=62C 244@F?E23:=:EJ]k^Am

k9bmp D:8?2EFC6 :D?’E E96 D2>6 2D 2 564:D:@?k^9bm

kAmp k2 9C67lQ9EEADi^^4FC6:D]4@>^4=2:>D\2:\92D\2?\2F5:E\EC2:=\AC@3=6>^QmD:8?2EFC6 2E E96 6?5 @7 2 9:89\G@=F>6 BF6F6k^2m :D ?@E 2 564:D:@?] qJ E96 E:>6 2 C6G:6H6C :D 2D<65 E@ 2AAC@G6 2 C64@>>6?52E:@?[ >F49 @7 E96 564:D:@? 92D 2=C625J 366? D92A65 3J E96 A=2?’D CF=6D[ A@=:4:6D[ 52E2 2?5 DJDE6> 4@?7:8FC2E:@?] ~? 2 AC24E:42= =6G6=[ E96 6E9:42= 2?5 @A6C2E:@?2= AC@3=6> 7@C 962=E9 42C6 @44FCD H96? E96 D:8?2EFC6 42?’E 2?DH6C 7@C H92E :E 2FE9@C:K65] %96 C6G:6H6C >2J 36 2AAC@G:?8 2? @FE4@>6 H:E9@FE 6?@F89 E:>6 @C 4@?E6IE E@ :?56A6?56?E=J E6DE :E]k^Am

kAmu@C AC@G:56CD[ 4=:?:42= 564:D:@? DFAA@CE E6?5D E@ 36 9:89 DE2<6D[ 3FE =@H6C G@=F>6[ 6?23=:?8 2 A9JD:4:2? E@ 6I2>:?6 E96 A2E:6?E 4@?E6IE[ 6G:56?46 2?5 C64@>>6?52E:@? :? 56E2:=] #:D<\362C:?8 A=2?D 7246 2 5:776C6?E AC@3=6>i r=2:>D[ 6=:8:3:=:EJ 2?5 6?4@F?E6C 564:D:@?D 92AA6? 2E >2DD:G6 D42=6[ 2?5 E96 F?56C=J:?8 52E2 492?86D 4@?DE2?E=J]k^Am

kAmp >6>36C’D 4=2:> >2J 36 56?:65[ @C 2 AC@G:56C’D A2J>6?E 25;FDE65[ @? E96 32D:D @7 2 C64@>>6?52E:@? 2 A=2? 42? ?@ =@?86C 7F==J C64@?DECF4E] x? E92E >@56=[ 2 9F>2? C6G:6H6C :D 2D<65 E@ 3=6DD 2 4@?4=FD:@? H:E9@FE E96 E:>6[ 4@?E6IE[ @C 6G:56?46 E@ 2AA=J 4@?D:DE6?E ;F58>6?E]k^Am

kAm(96? 2 A2DE 564:D:@? :D 492==6?865[ >@DE A=2?D 42? AC@5F46 2 =@8 6?ECJ D9@H:?8 D@>6@?6 =@@<65 2E E96 56E6C>:?2E:@? 2?5 42? 4:E6 E@52J’D 56?:2= 32D:D] u2C 76H6C 42? D9@H E96 6I24E 52E2[ CF=6D[ 2?5 C64@>>6?52E:@? E92E 6I:DE65 2E E96 >@>6?E E96 564:D:@? H2D >256] %96J >FDE C6=J @? 2? 2DDF>AE:@? E92E ?@E9:?8 >2E6C:2= 492?865 :? 36EH66?] %92E 2DDF>AE:@? :D C2C6=J AC@G23=6]k^Am

kAm!=2?D >FDE 36 23=6 E@ 6IA=2:? 56?:2=D[ A2J>6?ED 2?5 6=:8:3:=:EJ 56E6C>:?2E:@?D] x7 E96J 42??@E D9@H H9J 2 C6G:6H6C 2AAC@G65 2 C64@>>6?52E:@? :? |2C49 3642FD6 E96J 42??@E AC@G6 H92E E96 DJDE6> <?6H :? |2C49[ E96 72:=FC6 :D 2C49:E64EFC2=[ ?@E A6CD@?2=]k^Am

k9bm(92E 9F>2? :? 4@>>2?5 >62?Dk^9bm

kAmwF>2? :? 4@>>2?5 AFED A6@A=6[ ?@E >@56=D[ :? 4@?EC@= @7 E96 AC@46DD] !6@A=6 D6E E96 A@=:4:6D 2?5 CF=6D] %96J 2AAC@G6 492?86D E@ E9@D6 CF=6D 2?5 >2<6 E96 ;F58>6?E 42==D @? 6I46AE:@?D E96 CF=6D 5@ ?@E 4@G6C] pFE@>2E:@? 6I64FE6D E96 <?@H? CF=6D @? 4=62C 42D6D 2?5 DFC7246D E96 C6DE] %96 52E2 7@F?52E:@? <66AD 2 C64@C5[ 2G2:=23=6 @? 56>2?5 2?J E:>6 =2E6C[ @7 E96 6I24E DE2E6 2E 564:D:@? E:>6] %96 6E9:42= 2?5 @A6C2E:@?2= C6BF:C6>6?E :D E96 D2>6i E96 23:=:EJ E@ D9@H H92E E96 DJDE6> <?6H H96? :E 24E65]k^Am

k9bm#68F=2E:@? :D >@G:?8 E@H2C5 C62= 9F>2? 2FE9@C:EJk^9bm

kAm#68F=2E@CD 2C6 368:??:?8 E@ 5C2H E96 D2>6 =:?6] k2 9C67lQ9EEADi^^HHH]2K=68]8@G^=68E6IE^df=68^`#^DF>>2CJ^w]wqa`fd0_c_cad0$t}p%tt}v#~$$ts]s~r)]9E>QmpC:K@?2’D wq a`fdk^2m[ 67764E:G6 yF=J a_ae[ C6BF:C6D E92E 367@C6 2 962=E9 A=2? 56?:6D 2 4=2:> @C AC:@C 2FE9@C:K2E:@? @? >65:42= ?646DD:EJ 8C@F?5D[ 2 >65:42= 5:C64E@C >FDE :?5:G:5F2==J C6G:6H E96 56?:2=[ 6I6C4:D6 :?56A6?56?E >65:42= ;F58>6?E 2?5 ?@E C6=J D@=6=J @? 2 C64@>>6?52E:@? 7C@> 2?@E96C D@FC46]k^Am

kAmk2 9C67lQ9EEADi^^4FC6:D]4@>^2C:K@?2D\2:\56?:2=\=2H\5@6D?E\>6?E:@?\2:^Qm%96 =2H 5@6D ?@E 32? 2FE@>2E:@?k^2m] xE 2==@HD 52E2 E@ 36 82E96C65[ CF=6D E@ 36 2AA=:65[ 4@?7=:4ED E@ 36 7=28865 2?5 C64@>>6?52E:@?D E@ 36 86?6C2E65] qFE :E >2<6D 4=62C E92E 2 564:D:@? 27764E:?8 2446DD[ A2J>6?E @C 4@G6C286 42??@E C6DE @? 2? 2FE@>2E65 C64@>>6?52E:@? 2=@?6] %96 @C82?:K2E:@? >FDE 36 23=6 E@ D9@H H9@ >256 E96 564:D:@?[ H92E :?7@C>2E:@? H2D 2G2:=23=6 2?5 H9J E96 @FE4@>6 7@==@H65 7C@> E96 CF=6D :? A=246 2E E96 E:>6]k^Am

kAm%92E 5:DE:?4E:@? D6ED 2 AC24E:42= 7=@@C 7@C ECFDEH@CE9J 2FE@>2E:@? C2E96C E92? 2 46:=:?8 @? E649?@=@8Ji p DJDE6> H9@D6 @FEAFE 42??@E 36 6IA=2:?65 D9@F=5 ?@E 36 ECFDE65 E@ 5C:G6 2 4@?D6BF6?E:2= 564:D:@?] p 9F>2? 42? @?=J 36 :? 4@>>2?5 @7 2 AC@46DD E96J 42? :?DA64E] %@ 6IA=2:? 2 564:D:@?[ 2 A=2? >FDE 36 23=6 E@ C64C62E6 :E]k^Am

k9bm!@:?E\:?\E:>6 C64@?DECF4E23:=:EJk^9bm

kAm!@:?E\:?\E:>6 C64@?DECF4E23:=:EJ >62?D 36:?8 23=6 E@ C64C62E6 E96 6I24E 724ED[ CF=6D 2?5 52E2 G6CD:@?D E92E 6I:DE65 H96? 2 564:D:@? H2D >256] %92E :?4=F56D E96 >6>36C’D 6=:8:3:=:EJ DE2EFD[ E96 AC@G:56C 4@?EC24E[ E96 C2E6 E23=6[ E96 2AA=:423=6 A@=:4J[ E96 4=2:> 56E2:=D[ E96 CF=6 G6CD:@? 2?5 2?J 6I46AE:@? @C C64@>>6?52E:@? E96 DJDE6> 86?6C2E65]k^Am

kAm%9:D 42??@E 36 25565 C6=:23=J 27E6C E96 724E] p E@@= E92E D:ED @? E@A @7 F?DE23=6 @C F?G6CD:@?65 52E2 42? D9@H H92E E96 DJDE6> D66D ?@H] xE >2J ?@E 36 23=6 E@ AC@G6 H92E E96 DJDE6> D2H E96?[ =62G:?8 E96 @C82?:K2E:@? E@ C63F:=5 E96 564:D:@? :?DE625 @7 D9@H:?8 :E]k^Am

k9bm(92E 9F>2? :? 4@>>2?5 =@@<D =:<6 :? 4=2:>Dk^9bm

kAmwF>2? :? 4@>>2?5 5@6D ?@E >62? A6@A=6 564:56 6G6CJ 4=2:> >2?F2==J] pE D42=6[ E92E’D F?C62=:DE:4] xE >62?D A6@A=6 D6E E96 CF=6D[ 2AAC@G6 492?86D E@ E96> 2?5 92?5=6 E96 6I46AE:@?D E92E 42== 7@C ;F58>6?E]k^Am

kAmpFE@>2E:@? 92?5=6D E96 4=62C[ C6A62E23=6 42D6D] xE 2=D@ 7=28D H92E :E D9@F=5?’E 564:56 2=@?6i >:DD:?8 52E2[ 4@?7=:4E:?8 6G:56?46[ F?FDF2= A2EE6C?D 2?5 2?JE9:?8 @FED:56 567:?65 4@?7:56?46 @C C:D< E9C6D9@=5D] (96? E96 @FE4@>6 :D 4@?D6BF6?E:2=[ E96 42D6 8@6D E@ E96 C:89E A6CD@? 2=@?8 H:E9 E96 6G:56?46 ?66565 E@ >2<6 2 C62= 564:D:@?]k^Am

kAm%92E’D H96C6 >2?J 4=2:>D px E@@=D 72== D9@CE] %96J’C6 AC@323:=:DE:4i %96J AC65:4E E96 >@DE =:<6=J 2?DH6C 2?5 D6?5 H92E6G6C :D DE:== F?46CE2:? E@ 2 C6G:6H BF6F6] %96 C6G:6H6C D66D 2 C64@>>6?52E:@? 2?5 :D 2D<65 E@ 2446AE @C C6;64E :E] %92E H@C<7=@H :D >@C6 =:<6 E9C@F89AFE >2?286>6?E E92? 8@G6C?2?46]k^Am

kAmx? 2 9F>2?\:?\4@>>2?5 >@56=[ E96 C6G:6H6C 42? 2=D@ D66 H9J E96 DJDE6> C624965 E92E C64@>>6?52E:@?[ H92E 724ED :E C6=:65 @?[ H9:49 CF=6D :E 2AA=:65[ H92E F?46CE2:?EJ :E 56E64E65 2?5 H96E96C E96 42D6 72==D H:E9:? E96 DJDE6>’D 2AAC@G65 D4@A6]k^Am

kAm%96 AC@46DD D9@F=5 2=D@ =62C? 7C@> 6G6CJ C6G:6H] ~G6CC:56D[ C6A62E65 6D42=2E:@?D[ 2AA62=D[ 2F5:E 7:?5:?8D 2?5 4@CC64E65 4=2:>D 2== A@:?E E@ D@>6E9:?8] x7 C6G:6H6CD <66A C6G6CD:?8 E96 D2>6 EJA6 @7 C64@>>6?52E:@?[ E96 CF=6[ 52E2 D@FC46[ >@56= @C H@C<7=@H =:<6=J ?665D E@ 492?86]k^Am

k9bmp AC24E:42= >@56= 7@C 9F>2? :? 4@>>2?5k^9bm

kAmx?5FDECJ 3@5:6D 2C6 4@?G6C8:?8 @? E96 AC:?4:A=6 E92E 4=:?:42= 2FE9@C:EJ >FDE 36 62C?65 E9C@F89 6G:56?46 2?5 >2:?E2:?65 E9C@F89 A6C7@C>2?46] q67@C6 56=682E:?8 2FE9@C:EJ E@ 2 E@@=[ 6DE23=:D9 H92E :E 42? C6=:23=J 5@[ 7@C H9@> 2?5 F?56C H92E 4@?5:E:@?Dj 2?5 E96? <66A 4964<:?8 2D 4@?5:E:@?D 492?86]k^Am

kAm%C2?D=2E65 :?E@ 2? @A6C2E:?8 >@56= 7@C 962=E9 A=2?D[ E96D6 AC:?4:A=6D DE2CE H:E9 E96 52E2 =2J6C 2?5 6?5 H:E9 244@F?E23=6 564:D:@?D]k^Am

kF=mk=: 52E2\=:DE\:E6>\:5lQ6ff4a673d2f7_5c`fdhd4d4`h33d7_effQmkDEC@?8mx?G6?E@CJ 4@?D6BF6?E:2= 564:D:@?D]k^DEC@?8m x56?E:7J H96C6 56?:2=D[ A2J>6?ED 2?5 6=:8:3:=:EJ 56E6C>:?2E:@?D 2C6 >256 2?5 H92E 6G:56?46 6249 564:D:@? C6=:6D @?]k^=:mk=: 52E2\=:DE\:E6>\:5lQ62db376ed4eg357`a7e5d5hhg233h4fh5QmkDEC@?8m"F2=:7J 367@C6 56=682E:?8]k^DEC@?8m u@C 6249 px\2DD:DE65 7F?4E:@?[ 5@4F>6?E :ED :?E6?565 FD6[ A@AF=2E:@?D[ =:>:E2E:@?D 2?5 72:=FC6 >@56D] s67:?6 H96? E96 E@@= >2J 24E 2?5 H96? :E >FDE 5676C]k^=:mk=: 52E2\=:DE\:E6>\:5lQ66ddd4h4__a_dg723fgg`c`bgb6`6377gQmkDEC@?8m'6CD:@? H92E >2EE6CD]k^DEC@?8m !C6D6CG6 A@:?E\:?\E:>6 DE2E6D 7@C 6=:8:3:=:EJ[ C2E6D[ 4@?EC24ED[ A@=:4:6D 2?5 CF=6D] |2<6 E96> D62C4923=6 3J 564:D:@? xs 2?5 52E6]k^=:mk=: 52E2\=:DE\:E6>\:5lQ6b54g`f_fhf6e7_ge6dbgfcahgh2_h322QmkDEC@?8ms6D:8? C6G:6HD E92E 255 G2=F6]k^DEC@?8m v:G6 C6G:6H6CD E96 :?AFED[ CF=6D 2?5 C62D@?:?8 E96 DJDE6> FD65] %6DE H96E96C E96 4@>3:?65 H@C<7=@H :>AC@G6D 244FC24J[ 6BF:EJ 2?5 4@?D:DE6?4J]k^=:mk=: 52E2\=:DE\:E6>\:5lQ6_36_26f5g5_2`_3ebg`a7g2`_beh6c_aQmkDEC@?8mtD42=2E6 H:E9 4@?E6IE]k^DEC@?8m #@FE6 4@>A=6I 42D6D E@ E96 6IA6CE 36DE 6BF:AA65 E@ ;F586 E96>[ H:E9 E96 724ED ?66565 E@ 564:56 @? 7:CDE C6G:6H]k^=:mk=: 52E2\=:DE\:E6>\:5lQ6da4352gcegc`2fcge7757cf6_5g7fhf5QmkDEC@?8mr@?E:?F@FD=J C6BF2=:7J]k^DEC@?8m |@?:E@C @G6CC:56D[ 6D42=2E:@? C2E6D[ 5:D28C66>6?ED[ 6CC@C A2EE6C?D 2?5 >@56= @C 52E2 5C:7E] p5;FDE 2FE9@C:EJ 2?5 4964<A@:?ED 2D 6G:56?46 492?86D]k^=:mk^F=m

kAm(:E9 C64@?DECF4E23=6 C64@C5D 2?5 42=:3C2E65 4964<A@:?ED[ A=2?D 42? 7:I 2 C@@E 42FD6 @?46[ AC@A282E6 E96 4@CC64E:@? 2?5 2G@:5 C6A62E:?8 E96 D2>6 6CC@C 24C@DD E9@FD2?5D @7 42D6D]k^Am

k9bm%96 9F>2? 2FE9@C:EJ DE2?52C5k^9bm

kAmpx >2J 4@?E:?F6 E@ >@G6 566A6C :?E@ 962=E9 A=2? @A6C2E:@?D 3642FD6 E96 G@=F>6 @7 4=2:>D[ 6=:8:3:=:EJ 4964<D 2?5 A2J>6?E 564:D:@?D :D =2C86[ 2?5 2FE@>2E:@? >2<6D :E >@C6 >2?28623=6] %96 BF6DE:@? :D H96E96C E92E 2FE@>2E:@? :D 8@G6C?65 3J 2 C64@C5 DEC@?8 6?@F89 E@ DFAA@CE 9F>2? 2FE9@C:EJ]k^Am

kAmp D:8?2EFC6 :D 62DJ E@ 4@==64E] p 5676?D:3=6 564:D:@? C6BF:C6D 2 DJDE6> E92E 42? D9@H E96 6I24E 52E2[ CF=6D[ A@=:4:6D 2?5 C64@>>6?52E:@?D :? A=246 H96? E96 564:D:@? H2D >256]k^Am

kAm%92E :D H92E =6ED 2 A=2? 6IA=2:? 2 56?:2= E@ 2 >6>36C[ 5676?5 2 A2J>6?E 564:D:@? E@ 2? 2F5:E@C 2?5 4@CC64E 6CC@CD H:E9@FE C65:D4@G6C:?8 E96> @?6 4=2:> 2E 2 E:>6]k^Am

kAmpF5:ED 5@ ?@E 2D< H92E E96 DJDE6> <?@HD E@52J] |6>36CD 5@ ?@E 2D< H92E E96 CF=6 D2JD ?@H] %96J 2D< H9J 2 564:D:@? H2D >256 E96?]k^Am

kAmwF>2? :? E96 =@@A 42? 2?DH6C[ “$@>6@?6 4=:4<65 2AAC@G6]” wF>2? :? 4@>>2?5 92D E@ 2?DH6C[ “w6C6 :D H92E H6 <?6H[ 96C6 :D E96 CF=6 H6 2AA=:65 2?5 96C6 :D H9J E96 564:D:@? 7@==@H65]”k^Am

kAmk2 9C67lQ9EEADi^^4FC6:D]4@>^H96C6\E96\9F>2?\2:\H@C<7=@H\:?\962=E9\42C6\3C62<D\5@H?^Qmk6>m%9:D DE@CJk^6>mk^2mk6>m H2D AC@5F465 3Jk^6>m k2 9C67lQ9EEADi^^4FC6:D]4@>^Qmk6>mrFC6x$ w62=E942C6k^6>mk^2mk6>m 2?5 C6G:6H65 2?5 5:DEC:3FE65 3Jk^6>m k2 9C67lQ9EEADi^^9F3D]=2^"_b<=8$#_Q C6=lQ?@7@==@HQmk6>m$E24<6Ck^6>mk^2mk6>m]k^6>mk^Am