So what we did was come up with a score that we could use the compare different code-bases and rank them. I don’t think we had a cool name for it at the time but for this post I’ll call it ‘relative toxicity’. Erik has already described how we calculated the scores using checkstyle so I won’t go into that again, but what we did was sum the scores for all classes in the code-base and get a grand total (this is even shown on the second page of the spreadsheet) and then divide this by the number of classes in the code-base to get an average points per class score. This allowed us to compare different code-bases of different sizes and see where they ranked.
We then ran this against some well known open source projects such as XStream, Spring, JBoss and Hibernate and compared our project code-base against them. This meant that we could tell the managers something a little bit more meaningful to them such as ‘the project code quality is three times as toxic as most open source projects’.
BTW I also retried this on my current project and found our ‘relative toxicity’ level was slightly better than Hibernate.