Your Brain on ChatGPT: Accumulationof Cognitive Debt when Using an AlAssistant for Essay Wrting TaskNataliyaKosmynaEugeneHauptmannYeTongYuanJessicaSituMITMediaLabMITWellesleyCollegeMITCambridge,MACambridge, MAWellesley,MACambridge,MAXian-HaoLiaoAshlyVivianBeresnitzkyIris BraunsteinPattieMaesMass.CollegeofArtMITMITMediaLabandDesign(MassArt)Cambridge, MACambridge,MACambridge,MABoston,MAUnitedStatespvaluesEEGAlphaBandFigure1.ThedynamicDirectTransterFunction(dDTF)EEGanalysisofAlphaBandforarounsLLMSearchEnaine.Brain-only,includingp-valuesto show siginificancefrommoderatelysignificant(")tohighlysignificant(**,NataliyaKosmynais thecorresponding author, please contact herat nkosmvna@mit edluDistributed underCCBY-NC-SA
AbstractWith today's wide adotion of LLM products like ChatGPT fromOpenAl, humans andbusinesses engage and use LLMs on a daily basis. Like any other tool, it carries its own set oadvantages andlimitations. Thisstudyfocusesonfindina outthecoanitive cost ofusina an !!Min the educational context of writing an essav.a We assianed participants to thee groups:ILM group,Search Engie group, Brain-only groupwhere each participant used a desianated tool (or no tool in the latter) to write an essay Weconducted 3 sessions with the same aroup assinment for each particiant.In the 4th sessioweaskedLLMgroupparticipants touse no tools (we refer to them as LLM-to-Brain), and theLM). We recruited a totalof 54participantsforSessions1,2,3,and18particiantsamonathemcomoletedsession4Weusedelectroencephalographv(EEG) to record participants'brain activity in order to assessnitive load. and to aain adeeper understandina ofneuraactivations during the essay writina task. We performed NLP analysis. and we interviewed eachand an Aliudae (a specially built Al agent)EntitiesRecoanition(NERs)LLM. SearchEnaineandBrain-onlv aroushadsianificantlvdifferentneuralconnectivityBrainconnectivitv svstematicallyscaled downwiththe amount of externalsupport:the Brain-onlv aroup exhibited the stronaest. widest-ranginenetworks,SearchEnginegroupshowedintermediateenaaaementandLLMassistanceelicitethe weakestoverall couplina. Insession participants showed weaker neuraandunder-engagemenparticipantsdemonstratedhigherwidespreadoccipito-parietal and prefrontal nodethe visual processina. similar to the onefrequentlvperceivedintheSearchThe reoorted ownership of LLM group'sessays in the interviews was ow The Search Engine group had strong ownership, bur lessethan the Brain-only group. The LM group also fel behind in their ability to qucte fom theessavs thev wrote just minutes prionAs the educationalimpact ofL.Museonly beains to setle with the aeneral oooulation. in thisstudv we demonstrate the pressina matter of a likela learning skills based on theresultsofourstudy.Theuseof LLMhad ameasurable impact on oarticipants. and while thebenefitswereinitiallyapparent,aswedemonstratedoverthecourseof4months.theLLMgroup'sparticipantsperformedworsethantheircounterpartsintheBrain-onlyaroupatallevelsneural, linguistic, scoringWe hope this study serves as a preliminary...
