Quality Criteria for Open Source Products

[Franz Nahrada]

Some of us had some mail discussions recently about the issue of highlighting the quality of open products. I entered in closer discussions with Michel recently and challenged him with the idea, the perspective and the possible project of the foundation to act as catalyzer to bootstrap wholistic and sustainable solutions.

The case for such an endavour was beautifully expressed by the developers of one of the finest open hardware pieces (including the best web documentation for reproduction that I have come accross so far). I am talking about the Twibright Labs people in Prague with their Ronja, an optoelectronic device that allows high speed data links. Ronja is one of my favorite projects - especially when it comes to good web-based documentation. This documentation enables any small enterprise or private person to build the device, although it seems more feasible to leave this to firms that have a lot of proficiency. But what I want to quote here is a much more general remark that contains a lot of truth - from people who walk the talk:

“I can't get no, I can't get no,
I can't get no satisfaction,
no satisfaction, no satisfaction, no satisfaction.”
— Rolling Stones, (I Can't Get No) Satisfaction

"In today's world, there is a large number of free software applications. The free software development model has a potential of producing high quality software with high quality documentation that would make the user able to fully exploit in a straightforward manner it's all promised features. Such a software would be a very powerful tool for the user that would bring him a great level of satisfaction."

"However, sadly, the real situation is far from the theoretical potential."

"Everyone wants to be satisfied by the things he is using. Who wouldn't. But only some projects have a goal of satisfying it's user. Other projects just pretend it. It's natural that the user wants to select those products that will satisfy him, but if he goes only according to what these projects say, it will not work, because all projects say they are the good ones. If he goes by what his friends say, this will not work either, because what works properly in one situation, may completely fail in another."

continue reading here:

http://ronja.twibright.com/satisfying.php

Of course this applies to any type of open hardware or open service, too.

I am sure these thoughts can be complemented with social issues, like which product enables a business cycle that creates the most use value but also answers the question of subsistence. Maybe we need 10 or more criteria to cover all aspects of a free product!

I would love the P2P foundation to take up these issues.

your comments are very welcome!

Franz

[Sepp Hasslberger]

Great discussion of a satisfying product on the Ronja site you linked.

This should be part of every open source project, and perhaps as we get more and more OS in both software and hardware, there should even be a rating system that allows users to rate the satisfaction level (maybe the five stars) they have experienced with the product in question.

I am also especially impressed with Ronja's Optical Data Link which, with a respectable range of just under a mile and the use of either red light or infrared radiation could do much to make our consumer driven mesh networks 'health neutral' in that they won't add to the already considerable overload with microwave radiation that has been found to be full of hazards for human and animal health.

[Franz Nahrada]

Hi Sepp,

thanks for breaking the ice. The health issue is extremely important and it should be crucial for Open Design.

To the more general question:

I agreed with Michel that there SHOULD be a rating system for both software and hardware, but this task would be by far too complex to achieve at the current time.

So I came up with two intermediate proposals:

-> for the P2P foundation to put our brains together and create an award. I have a good offer for sponsoring this award, but without a significant community of excellent jurymembers this is impossible. We invite everybody to support the work on award criteria, details will be published soon.

-> to create a global network of Open Innovation Centers. I want to start a project of that kind here in Austria, but I invite likeminded spirits to go along. This is ideally a physical place with Hotel and Conference Site dedicated to support both meetings and community sprints. It should have all the work facilities to successfully do critical steps of a project in a temporary meshup of users and developers.

[Sepp Hasslberger]

Hi Franz,

this sounds like a good way to go ahead.

In any case, I imagine that a listing of open source projects divided into software and hardware projects with a short description and link to where it's available could be hosted on a Wiki, where users could rate them with a simple mechanism that shows how many users did rate, and what the calculated average rating is in one to five stars.

That would be relatively simple to set up and would be a first step towards bringing this quality issue into a P2P 3.0 context, where the power of user evaluation makes sure the best things "swim to the top".

[Franz Nahrada]

I started such a project within the Oekonux context in 2003/2004. Viennese Students evaluated free projects and created lists within categories. Of course Michel has long surpassed that quantity in the P2P wiki, but it is hard to find anything specifically there, besides the issue of the difficult telling whats good or bad.

Now I think this needs as prerequisite some kind of preformatted project description, so that project issues and goals are easily identifyable. Maybe a questionaire sent to project maintainers, asking them to do the extra work of answering 10 crucial questions about their project - like Marcins outline in Open Source Ecology

* 1 The Theory - Background
* 2 Existing Problems
* 3 Proposed Way to solve them
* 4 Related Existing Work
* 5 Related Existing Data
* 6 Existing constraints
* 7 Related Existing Work that Argues Against
* 8 Facts and Data
* 9 Brief Summary
* 10 Links

Would you volunteer to investigate the best ways to set something up? Which Wiki Engine has a rating module? (or more)Who would technically run the Wiki? Who would do principal maintainance and editorship of such a project? How can the voting mechanism be protected from fraud? I think this community is fine to do such an investigation and we can invite all possible candidates here.

Franz

[Sepp Hasslberger]

My thought was more along the lines of asking only the actual users of such products to give their evaluation of satisfaction with it.

For this, it would be sufficient to have a listing of products, with a short description of - let's say - up to a hundred words or so, a link to where to find the product, and a rating engine introduced with something like:

"If you are a user of this product, please help us evaluate its quality by rating how satisfied you are with the experience" (or some similar wording).

Nothing very complicated.

In a Wiki, it could probably be set up where users or producers of a product could add their product to the record.

I am not sure whether such a rating engine is available in the public domain, or what site could host that. Probably the main thing would be to keep it simple and to make it quite clear that only people who have actually used the product are invited to evaluate. Some safeguarding against multiple votes (but still allowing a user to change their evaluation) would be desirable.

This would certainly be out to a slow start, but once it catches on, it could become quite useful for people to distinguish what kind of product they would like to try.

Probably products should be grouped to put similar ones next to each other.

Sepp

[Sepp Hasslberger]

Here is a comment I just made on another article, on the P2P Foundation's blog:

"Open source hardware could benefit from a culture that promotes the self-assembly, from pre-existing modules, of useful machinery. Those pre-existing modules should be available in great numbers and be interconnectible in various ways. It doesn’t matter whether those modules are manufactured by commercial companies that manufacture parts for today’s industries or by open source hardware developers.

The important thing would seem to be the concept of combining pre-fabricated modules in various novel ways to form new (open source) products.

Could this work to give open source hardware the boost it needs?"

This would imply that the "open source" part of the hardware assembly is not necessarily extended all the way down to the actual manufacture, but could well be in locating and knowing how to combine various existing modules in innovative ways to obtain useful and indeed innovative pieces of hardware. It's kind of updating the age-old practice of DIY to the technological world where fairly complicated and complete components are quite freely available, and where the act of locating a source and of combining those components is the actual added value that results in an open source finished product...

[Franz Nahrada]

Thats one way to go. Marcin has done well in showing a role model