Summary: GenAI Perception vs Reality, from Jellyfish

I like this GenAI: Perception vs Reality pdf from Jellyfish. Short and sweet. 

Jellyfish provides visibility into software development processes by integrating with Jira, GitHub, and CI/CD pipelines. As a result of their large customer base, Jellyfish are able to analyze more data than most individual companies can. In this case, they looked at 4,200 developers across 100+ companies, to see how using GitHub Copilot impacted productivity.

Some of their results include that engineers who use Copilot will have a 23% increase in coding speed and will ship 21% more. And that junior devs may have as much as a 4x increase in speed.

It is not clear what methodology Jellyfish are using for these stats though, nor how the Copilot usage results might apply to other tools. 

Still, I find this interesting. As we move to adopt GenAI coding assist tools in my workplace, we have been having a debate about how to measure if the tooling actually has any meaningful impact on productivity. Will it impact the “DORA” metrics such as lead time to deploy, deployment frequency or change failure rate, for example? 

So, interesting to see some data backing that enthusiasm for AI coding assist.

Tags: AI, artificialintelligence, copilot, genai, generativeai, github, githubcopilot, jellyfish