Qwen-2.5-Coder-32b was JUST released, an open source LLM that is fine-tuned for coding but also performs VERY well in agentic workflows. This model CRUSHES all benchmarks, especially for coding, and even outperforms Claude 3.5 Sonnet and GPT-4o in many ways as you can see on the Ollama library page for the model!
Let me just say - what a time to be alive! Especially for local and open source AI! I say the local AI revolution has started because Qwen-2.5-Coder-32b has really proved that we're nearing the point of local AI being as good as the big guys like GPT and Claude. It's only a matter of time.
In this video, I dive into my own testing for Qwen-2.5-Coder-32b, including trying it out with oTToDev (our Bolt.new fork) and an AI agent I created specifically to test new LLMs like this. I appreciate benchmarks, don't get me wrong, but I always want to try more practical things like this, and I wanted to share it with you too! And man, this model surpasses all my expectations and the performance of ANY other local LLM, even the larger ones!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
00:00 - Intro
01:42 - Model Overview + Hardware Requirements
04:22 - oTToDev with Qwen-2.5-Coder-32b
06:10 - Codellama's Epic Failure with oTToDev
07:45 - Novita
09:43 - Testing Qwen-2.5-Coder-32b as an AI Agent
15:12 - Outro
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Give Novita a shot today - it's honestly my new favorite platform for GPU instances and LLM inference. Not just because of our collab for this video but because of all the features I talk about in the video that I genuinely appreciate - the insanely competitive pricing, flexibility of your GPU instances, ease of setup, and the uniqueness of their available models.
Using the link below will help support me as a creator as well! :)
https://novita.ai/?ref=nmqzzdk&utm_source=affiliate
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Qwen-2.5-Coder-32b on Ollama (completely free!):
https://ollama.com/library/qwen2.5-coder:32b
Qwen-2.5-Coder-32b on OpenRouter (not free but about as cheap as GPT-4o-mini!):
https://openrouter.ai/qwen/qwen-2.5-coder-32b-instruct
oTToDev (our fork of Bolt.new):
https://github.com/coleam00/bolt.new-any-llm
Join the Discourse community for oTToDev:
https://thinktank.ottomator.ai
My LLM eval agent:
https://github.com/coleam00/ai-agents-masterclass/tree/main/llm-agent-evaluation-framework
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Artificial Intelligence is no doubt the future of not just software development but the whole world. And I'm on a mission to master it - focusing first on mastering AI Agents.
Join me as I push the limits of what is possible with AI. I'll be uploading videos at least two times a week - Sundays and Wednesdays at 7:00 PM CDT! Sundays and Wednesdays are for everything AI, focusing on providing insane and practical educational value. I will also post sometimes on Fridays at 7:00 PM CDT - specifically for platform showcases - sometimes sponsored, always creative in approach!
ai,artificial intelligence,ai agents,software engineering,software development,coding,automation,saas,llm,llms,breaking ai,google drive,asana,rag,rag ai agent,rag agent,streamlit,gpt,claude 3.5 sonnet,claude,groq,groq llama 3,ai tool calling,ai function calling,qwen,qwen ai,qwen llm,qwen 2.5 coder,qwen 2.5 coder 32b,qwen 2.5,qwen 2.5 32b,local ai,local llms,langchain,langgraph,ottodev,bolt.new,bolt,ai coding,ai coding assistant,local ai agents
