Large-scale language models (LLMS) have made significant advances in natural language processing, but inference remains a sustained challenge. Tasks such as mathematical problem solving and code generation benefit from structured training data, but include target deductions from sparse and fragmented data, scientific inference, and symbolic inference. A broader reasoning task (scientific reasoning, symbolic reasoning) benefits. Traditional approaches such as continuous advancement over code often implicitly embedding inference signals, making it difficult for the model to generalize. Even the way text-to-code generation remains constrained by syntax-specific learning, limiting applicability beyond programming-related tasks. Exposing LLMS to basic inference patterns while maintaining logical rigor requires a more structured approach.
Deepseek AI Research presents Codei/O, an approach to converting code-based inference into natural language. By converting RAW code into an I/O prediction format and expressing inference steps through the rationale of the idea (COT), CODEI/O will allow LLM to be a core inference process such as logic flow planning, decision tree traversal, modular decomposition, etc. can be internalized. Unlike traditional methods, Code/o separates inference from code syntax, allowing for wider applicability while maintaining logical structures.
Technical overview and benefits
Codei/o follows a structured data processing pipeline.
Collecting raw code files: Over 450K functions were collected from multiple sources, including algorithmic repository and educational programming datasets. Data Standardization: The collected code was refined using DeepSeek-V2.5 to ensure clarity and execution compatibility. Generating input/output pairs: Functions were executed on different inputs to create structured training examples across different inference tasks. Generate inferences for thought chains: Natural language descriptions were generated to provide structured inference using models such as DeepSeek-V2.5. Validation and improvement: Predictions were validated through execution, and false responses were repeatedly corrected, improving inference accuracy.
Key features of codei/o:
Transformation learning: Transforms a variety of code patterns into natural language cot rationales, making them transferable beyond the context of programming. Syntax decomposition learning: Separate logical inference and code syntax, improving adaptability across inference tasks. Multitasking improvements: Improve performance across symbolic, scientific, logical, mathematical, and common sense inference domains. Verifiability: Predictions can be validated via cached ground truth matching or reexamination. Iterative refinement: A refined version of Codei/O++ employs multi-turn revisions to increase the accuracy of inference.
Empirical results and performance
The effects of CODEI/O were tested on four base models (range 7B to 30B parameters) with 14 inference benchmarks covering logic, symbolic inference, mathematics, scientific deduction, and common sense inference.
Survey results:
Consistent improvement: CODEI/O training provided higher scores across inference benchmarks compared to traditional pretraining methods. Overall Task Generalization: Unlike existing approaches that improve certain tasks but reduce performance elsewhere, Codei/O showed a balanced enhancement. Comparison with baseline: Codei/O outperform datasets such as OpenMathinStruct2, OpenCoder-SFT-Stage1, WebInstruct. Effectiveness of Multi-Turn Sophistication: CODEI/O++ has further improved results by repeatedly improving false responses and leveraging execution feedback for better inference quality.
For example, in logical and symbolic inference benchmarks such as BBH and Cruxeval, CODEI/O led to significant performance improvements. The mathematical inference tasks (GSM8K, Mathematics, and MMLU STEM) showed improvement over existing baselines. Even common-sense inference, where code-based methods usually struggle, Codei/O maintained robust results.
Conclusion
CODEI/o presents a structured method to enhance LLMS inference by leveraging input/output conversions from real code. Instead of focusing on isolated inference tasks, we extract universal inference patterns and translate them into natural language descriptions. This structured learning approach ensures that the model acquires robust inference skills in different domains.
The introduction of multi-turn revision (CODEI/O++) further improves the accuracy of inference, indicating that iterative learning from execution feedback increases model reliability. By making predictions verifiable, CODEI/O offers a scalable and reliable way to improve LLM inference.
By bridging codebases and natural language inference, Codei/o offers a promising orientation to enhance cognitive abilities in LLMS beyond programming-related tasks.
Please see the paper and the github page. All credits for this study will be sent to researchers in this project. Also, feel free to follow us on Twitter. Don’t forget to join 75K+ ML SubredDit.
Commended open source AI platform recommended: “Intelagent is an open source multiagent framework for evaluating complex conversational AI systems” (promotion)
Asif Razzaq is CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, ASIF is committed to leveraging the possibilities of artificial intelligence for social benefits. His latest efforts are the launch of MarkTechPost, an artificial intelligence media platform. This is distinguished by its detailed coverage of machine learning and deep learning news, and is easy to understand by a technically sound and wide audience. The platform has over 2 million views each month, indicating its popularity among viewers.
✅ (Recommended) Join the Telegram Channel
