AI models still struggle to debug software, shows Microsoft Study – Notiulti [Spanish]

Source: Notiulti | Author: Notiulti | Read the full article in Spanish

In a recent study by Microsoft Research, it was found that even advanced AI models struggle with debugging software effectively. While companies like Google and Meta are increasingly using AI to assist in programming tasks, the study highlights that these AI systems often fail to resolve errors that experienced developers can easily fix. The research tested various AI models on a set of debugging tasks and revealed that their success rates were disappointingly low.

The study involved nine different AI models, which were tasked with solving 300 curated debugging challenges. The results showed that even the most capable model, Claude 3.7, only managed to complete about 48% of the tasks successfully. The researchers noted that many models had difficulty utilizing the debugging tools available to them and lacked sufficient training data to understand complex decision-making processes involved in debugging.

These findings serve as a reminder that while AI technology is advancing, it still cannot replace the expertise of human developers in coding and debugging. The study emphasizes the need for specialized training data to improve AI's performance in these areas, suggesting that the future of coding will still rely heavily on human skills and judgment.

[Read More (translated)]