Conr1/0:Condensing Reasonine Patterns viaCode Inout-OutoutPredictionJunlongLil23*Dava CGuo'DeianYamelYu Wu'JunxianHe3Abstractnized as a comerstone of advanced Large LanguageModachievingArtificialReasoning is a fundamental capability of Largeang,2022;QiaoLanguageModels. While priorresean);Xiangetal,2025).Currentinantlv focuses on enhancine narowfundamental paradox: whileg(Shaoetal,2024;Yangnget al.2024;ToshniHuiet al. 2024) benefitmost other reasoningscientific inferencecrucial to iden-OISong patternseflect the inteEA9ISLO'ZOSZ:AIXIEputs, we can verify each prediction and furtenhance theCoTsresulting in CODEI/O++performance.Our data and models are availablat https://github.com/hkust-nlp/Code1O.1.Introductionalting data incorporates a vae prob lem solvine. forming the basis for quickly txploration, recursive decompo and adapting to new tasks (Dehaene etal,2004;KnauLearning from these samples&Wolf, 2010; Wamg & Chiew,2010), Itis alsonelexts provided by the raw code filesain repeated exposure to these reasoningWork done during intershipat DepSeek-A1.-DeepSek,allow ing them to better internalize these skills2Shangha iao Tong University HKUST. Comespondence,ual pre-training on raw code, our code inanh@cse.ust.hk>.put/output prediction leaming is introduced as a distinctnining stage positioned before general instruction uning
Cobrl/O:Condensing Reasoning Phtterns via Code Input-Output PreditionVerification:Re-ExecuterReferenceCodefuncRevision`(optiona)ExecuteWhat istheinput?{"input";<the input>)InputGeneratorSamplexNWhatis the output?RawCode*Query"Youaregiven("output";<the output>}Figure 1:Oveview of our training data construction : Raw code files are gathered from various sources and converted into saunified frma nput-oupur pirs re then genened by execuring the code, whle aural languape of for prcictins ncollected from DeenSeek-V2.5. The verified CoT s can undergo ontional revisions to furtherenhance reasoninechainsvingas an intermediate step tgabilities of the base model. The prstruction pipeline is presented in this sectionecting rawcode files from various sourcesin transtformed into a unified formate sampled from the transformedn overvicw is depicted in Figure 1.es in selecting diverse rawwith differentbure algorithms. Beyond these twotehigh-quality code files from achallenging math problems, andoximately 810.5K code files. Furtherces can be found in Appendix C.1.ad are hard toexecute inasel-ontainedprocessthemusingDeepSeck-V2.5024), which refines them into a unixeuable for usto olletiopu-upunplisfonrediction tasks. This transforation organizes the datag components, and we providea completee in Table 8in Appendix G: 1) Cleaned Referencereasoning abilitiesCode: We preprocess the raw code files by cleaning and
Cobel/O): Condensing Reasoning Pa terns via Code Input-Out put PreditionQueryind a list of coinpossible to makethe amount with the givenins【1:])Given output=4,predict inputthe amount`13.Theut prediction respectivelycution of these codesnits on th...
