San Francisco company Cognition AI has created what it claims is the first AI developer who can perform programming tasks entirely on their own. He writes The world of information.
Cognition founder and CEO Scott Wu wrote in an article Blog post Their AI developer “Devin” can plan and execute complex developer tasks that require making thousands of choices.
– Devin can remember the relevant context at each step, learn over time, and correct mistakes, Wu wrote.
Equipped with all tools
What sets Devin apart from AI programming assistants like Copilot is that he will be able to work more autonomously in executing the task from start to finish. In addition, he must be able to work within a team.
According to Wu, Devin can report progress in real-time, get feedback and work with you to decide on design options.
-We also provide Devin with common developer tools, including the command line, code editor, and browser in a sandbox environment. Everything a person needs to do the job.
In Cognition's demo videos, we can see Devin writing code, reading error messages in the console and using them to fix errors, then running the code again until the code works. It even adds its own code to write debug information to the console. It then uses it as a coding aid, just like a human would.
Here's Cognition's intro video:
He can learn things he can't
Devin should be able to learn technologies he doesn't know, build and deploy applications, automatically find and fix bugs in code, train and tune his AI models, and much more.
By giving him a link to the Gihub issue, Devin should also be able to solve the issue, including setting up code environments, reproducing the bug, fixing the bug in the code, and finally testing the bug fix.
According to Cognition, Devin outperforms other AI assistants by a large margin when it comes to solving real GitHub issues in open source projects. While Devin fixed 13.86 percent of the “issues,” compared to only 4.8 percent for Claude2 or 1.74 percent for GPT-4.
Can Devon do real developer jobs? Cognition claims so, and in this video they ask Devon to solve an assignment posted on the job portal Upwork.
There is of course every reason to be skeptical, after all these videos produced by the manufacturer himself. Currently, Devon is only available to select test customers.
not worried
Although many worry about whether AI will take over developer jobs, not everyone is as afraid of AI assistants as Devin:
– If Devin can replace you, you're not a developer, writes John A De Goes, CEO of Ziverge.
Important note: If Devin can replace you, you are not a software engineer. 🌶️
If you think LLM-based approaches are still months away from replacing engineers, you need to update your AI mental model (because it's broken).
Devine is a nice trick to raise $$, but the real work is still ahead. pic.twitter.com/yfTDH0AbvG
– John De Goes (@jdegoes) March 13, 2024
He thinks Devin is a nice “trick” to raise money from investors, but AI is far from replacing developers.
– You will know when we have real “AI developers” because the companies that build them will immediately start selling software development services. De Goes says this will require major breakthroughs, something that will only take many years or decades.
Jeff Delaney, aka Fireship, made a YouTube video about Devin, saying:
– A tool that fixes Github issues only 13.86 percent of the time is not good enough to replace a developer. But the number will increase in the future.
Watch the video here:
“Web specialist. Lifelong zombie maven. Coffee ninja. Hipster-friendly analyst.”
