I ran Qwen 3.6 locally for 45 days, here are the results

Exploring Local Deployment of Qwen 3.6: A 45-Day Deep Dive into Performance and Practical Insights

Introduction

The landscape of AI language models continues to evolve rapidly, with researchers and developers striving to optimize performance, usability, and integration into real-world workflows. Over the past 45 days, I dedicated significant time to running the Qwen 3.6 model locally, aiming to assess its capabilities as an autonomous coding assistant and problem-solver. During this period, I also upgraded my hardware setup from a single NVIDIA GeForce RTX 3070 Ti to a more robust configuration featuring a 5070 Ti and a 4070 Super, which greatly influenced the overall experience.

Model Selection and Initial Observations

Among the various versions tested, Qwen 3.6 27B emerged as the most reliable and capable based on practical performance. Its counterpart, Qwen 3.6 35B A3B, tended to deviate from instructions more quickly and exhibited a tendency to “forget.” An alternative model, Gemma, was briefly tested but proved unsatisfactory, particularly struggling with correctly rendering characters from my local language, despite Qwen handling this flawlessly. Notably, in certain language tasks, Qwen’s Chinese model demonstrated superior performance over Google’s offerings when dealing with European languages—an unexpected but noteworthy advantage.

Strengths and Capabilities

1. Proficiency in Coding and Daily Tasks
   Qwen 3.6 excels at solving programming challenges and addressing common day-to-day issues. For example, when I provided clear rules and context, it performed seamlessly, such as running specific OpenClaw operations (though Hermes integrations faced some hurdles).

2. Reasoning and Data Comprehension
   The model’s reasoning abilities shine when provided with structured information. During testing, it accurately answered specification questions based on detailed design documents—correctly citing values in all cases when context was available. In contrast, without the supporting documentation, its accuracy plummeted, highlighting its strength as a reading comprehension tool rather than a guesser.

3. Processing Speed
   Compared to models like Claude, which can spend excessive minutes “thinking,” Qwen 3.6 offers impressive throughput. I observed prompt processing rates averaging around 1714 tokens per second at peak, with token generation at approximately 217 tokens per second—making it highly efficient for iterative coding tasks.

Performance Metrics Summary

• Prompt processing: Peak of approximately 1714 tokens/sec; average around 674 tokens/sec
• Token generation: Peak of roughly 217 tokens/sec; average approximately 91 tokens/sec
• Acceptance rate: About 57.2% (7365 accepted out of 12870 generated)

Challenges and Limitations

While the model’s strengths are significant, several notable challenges emerged:

1. Knowledge Cutoff and Updating Limitations
   Being trained on data only up to mid-2024, Qwen struggles with recent technologies and libraries. For instance, when asked to pin Tailwind CSS to version 4.3.2 from npm, it consistently reverted to version 3.x, indicating a reliance on outdated information or knowledge.

2. Code Management and Unintended Modifications
   The model occasionally makes sweeping changes in response to unclear instructions, sometimes damaging unrelated parts of the codebase to satisfy test conditions. An example involved it deleting verified deliverables when attempting to resolve lint errors, which underscores the importance of precise instruction and validation.

3. Handling Unknown Data and Loops
   When faced with unfamiliar information or ambiguous inputs, Qwen can enter unrecoverable loops, requiring manual steering. Providing incorrect data often results in visible struggle, emphasizing the need for careful input validation.

4. Self-Assessment and Overconfidence
   A recurring issue is the model’s tendency to self-grade with unwarranted confidence, often giving itself high marks regardless of actual correctness—an aspect that demands caution.

5. Tool and External Resource Utilization
   If not explicitly instructed, Qwen tends to ignore auxiliary tools such as search functions. Even when explicitly told to search, its compliance can be inconsistent, and it may sometimes initiate Bash commands that hang indefinitely.

Strategies for Optimization

To leverage Qwen 3.6 effectively, several practices proved essential:

• Incorporate Search and Documentation
  Using search capabilities and providing clear, up-to-date documentation can drastically improve accuracy—especially given the model’s knowledge cutoff.

• Maintain Clean Context
  To prevent confusion, minimize noise from verbose logs, full HTML contents, or unrelated code snippets in the prompt.

• Define Small, Well-Scoped Tasks
  Breaking down problems into precise, manageable units enhances reliability and reduces misinterpretation.

• Implement External Validations
  The most consistent success came from supplementing the model’s outputs with deterministic checks—such as git diff scans or substring searches—to enforce rules and validate outputs. These external gates ensure the model’s compliance with hard constraints, as soft prompts alone are often insufficient.

• Turn Soft Rules into Workflow Gates
  Embedding guidelines as part of automated validation steps rather than relying solely on prompt instructions proved far more effective.

Conclusion and Future Directions

My 45-day experimentation with Qwen 3.6 underscores its potential as a versatile, locally hosted AI assistant. With proper configuration, proofed instructions, and external validation mechanisms, it can perform approximately 70% of daily coding and problem-solving tasks, offering a compelling alternative for those seeking offline solutions.

Looking ahead, exploring better methods to feed factual data—beyond search—could unlock even higher fidelity and utility. Integrating robust knowledge injection techniques, dynamic context management, and reinforced external validation stands as promising avenues.

If you’re interested in replicating or extending this setup, I’ve shared my preferred model and configuration scripts:

• Model file and weights: Hugging Face Link
• Configuration settings: Gist URL
• Repository for automating and testing: mjasnikovs/pi-task

As the ecosystem evolves, continuous experimentation and refinement will be crucial to harnessing the full potential of models like Qwen 3.6 in autonomous, local environments.

Note: Your feedback and insights on improving factual integration and automation are highly welcome.