Methods & Tools Software Development Magazine

Software Development Magazine - Project Management, Programming, Software Testing


Quality of Open Source Software Development Tools

More and more developers are using development tools produced by the open source community like MySQL, Eclipse, PHP or JBoss. Our last poll examined how the quality of open source tools is perceived against their commercial competitors. We run a similar poll 2 years ago and it was interesting to compare the results.

Open source versus commercial tools  2006  2004
Same quality 38%  32%
There is no easy answer to this question 22% 24%
Superior in quality 20% 26%
Inferior in quality  12% 13%
I do not use open source tools 6% 4%
I do not use commercial tools 2% 1%
Participants  524 312

Ending date: August 2006

For many participants, there is no difference in perceived quality between open source and commercial tools for software development. For 22% of the participants, it was difficult to give a precise answer. Diversity exists in both worlds and it is not easy to give a clear indication when you have experiences giving opposite indications. Things have not changed a lot since our 2004 poll, even if the usage of open source tools has surely increased in the mainstream development shop. This may be a reason of the 6% decline in percentage in the "OS software is better than commercial" category, as open source has been more used and could revealed some limits.

The claim that open source software is as good as commercial one seems easy to understand. Besides their open source label, there is little difference in the available support infrastructure between products like JBoss, PHP or MySQL and their commercial competitor. Backed by large companies like IBM, products like Apache or Eclipse will surely receive more testing than a small project in SourceForge. For open source software development tools, a large user base also increases the probability that associated professional services are created to provide commercial support and that the quality of the software is "commercially" managed.

For 20% of the participants, open source development tools are superior in quality to commercial ones. Besides the results of our informal pool, there have been some studies to compare the quality of open source and commercial products. Part of these studies have investigated a claim by many open source software advocates that their code quality was higher. Peer review and the amount of feedback from users are quoted as allowing open source software to achieve high quality results. When it decided to release some software in the open source world the NASA gives "to increase NASA software quality via community peer review " as its first motivation (see references). But if the size of the development team and a smaller user base could be a problem for small vendors, larger commercial organisations could also have implemented internal peer review and they have also a user community with adequate feedback channels. So why could the feedback loops and quality perception be better in the open source community?

Several factors could influence this perception:

- Developers and users (not customers!) have a higher sense of product's ownership. They feel that they both contribute to something special and it is not "just a job" or "just a product"

- The relationship between users and developers are less confrontational because
a) money is not the matter
b) expectations are often different: the product is "younger" and... there is not a marketing organisation sometimes over-selling the benefits ;o)
c) open source organisations seems to have a better responsiveness to customers request/bugs as the process is more collaborative than confrontational

Some references on the quality of open source software:

Display list of archived polls

Software Testing

The Scrum Expert