Why it is ok to criticize FLOSS

July 2018

Every time someone criticizes FLOSS (free/libre and open source software), one can bet money that within first several comments someone would say that criticizing software that was done on a voluntary basis is illegitimate.

It's one thing to criticize software being sold, they will say, and another to criticize software that somebody worked on in their spare time. In fact, they'll continue, if you think a FLOSS program should be better, then contribute to it! If nothing else, report bugs.

The legitimacy of criticizing and even mocking commercial software might stem from the fact that you have spent money on it. And, thus, you now feel owed a certain level of promised functionality. Therefore, a failure to deliver gives a customer a right to complain.

With FLOSS nobody is selling this software, you are obtaining it for free. Therefore, the reasoning goes, nobody owes you anything, and your complaining is unwarranted. Proponents of this view will sometimes go even further and say that, in fact, you now owe to the kindness of the developer who made something useful for you. Therefore, you owe it to them to not complain and instead fix it or report bugs.

So, is this really justified? Does releasing software under a FLOSS license automatically shield it from criticism?

While at first glance this logic does seem to make sense, I claim it is actually false due to several reasons:

1. The argument that voluntary work should not be criticized assumes that people can feel cheated only if they paid money
2. FLOSS has a tendency to be oversold, much more so than commercial products, and requires more public criticism, not less
3. Criticism directed at FLOSS is not very consequential, which makes it unlikely to impact developers, or even reach them
4. To the extent that FLOSS criticism is consequential, it's likely to produce positive results for everybody in the long run

I describe my reasoning in this short article. I then provide real world examples of FLOSS that I believe deserves every bit of criticism it gets - and more.

Note: we will be talking about commercial software vs FLOSS. Although FLOSS does not have to be free of charge, it almost universally is. Commercial software, as in software which you have to pay to get a copy of, is almost always proprietary, but not vice versa: proprietary software is not always paid. Which is why we end up with FLOSS vs commercial.

1. Voluntary work can leave people feel cheated

Let's start by taking a separate look at commercial software and FLOSS, and unpack the situation, so that we can identify what exactly gives or does not give a user the right to complain.

A product is being sold. It has value in the eyes of a consumer due to the claim that its developer makes about it, either explicitly or implicitly. A customer then gives the producer money in exchange for this alleged value.

If the product underperforms, then a customer may feel that the money they spent is inadequate to the value they ultimately derived from the product. A demonstration that the claim of the developer does not match reality can even be used in court to receive damages from the developer. At the very least, it gives the right for the consumer to complain about the product, since they feel they were cheated.

But this is not it. There is a community service part to this, and a rather important one: by publicly complaining about a product that overpromises its advantages, one can divert other customers away from the transaction, and force the developer to lower the price or improve the product.

Now let's take a look at FLOSS products and see if we can unpack the situation from a similar perspective. A user installs a product because they believe it to be of value, based on the claim of the developer. They, however, do not pay for it, and instead receive it for free. Thus, when the product underperforms, they don't have a strong case to feel cheated, since no money exchanged hands.

But this is where cracks begin to show. If the product was promoted as reliable working software, but then turned out to be untested, ridden with bugs and having limited functionality, isn't it still misrepresentation on the part of the developer?

Perhaps, one can make an argument that misrepresenting a product is a smaller offense than both misrepresenting and asking for money. But it is still an offense. And a user, therefore, has at least some right to complain. And, just like with commercial software, such criticism serves a communal function of informing the public about the low quality of the product.

The argument that voluntary work should not be criticized assumes that feeling cheated is only legitimate if you paid money. Nothing, however, is farther from the truth. If anything, one can feel even more cheated when someone is presenting their voluntary work as a gift to the world, but is misrepresenting its value.

Also, attempting to buy people's respect merely by providing something for free is thinking very low of them and making it all about money. But the whole value of volunteer work is that this work adds actual value at no charge, not that it is simply at no charge.

2. FLOSS tends to be oversold

That FLOSS tends to be oversold should be obvious to anyone intimately involved with a FLOSS environment like Linux. There are several objective reasons why it happens.

First, most FLOSS projects are a work in progress, and it is typical to present the vision of the project before this vision is actually implemented. This is often required to draw initial interest and resources to the projects, such as more developers, first users, Linux distribution teams. Therefore, the homepage of the project might boast functionality that is a mere intention.

Second, many FLOSS projects have a tendency to outsource testing to users. While this is touted as an advantage of the open source model, in practice it seems to give permission for developers to largely disregard reliability as part of their work. This is often a pragmatic decision, since testing is an incredibly laborious and time consuming part of software development, as well as potentially expensive: if tests have to be conducted on a variety of hardware, the development team must have access to this hardware.

This results in a strong focus on functionality and weak to non-existent focus on reliability. It is not rare to read an ornate description of a program that promises all sorts of advanced functionality, only to discover that the software is basically unusably unstable. Sometimes whole classes of such low quality FLOSS packages are added to repositories, creating a false impression of abundance and wasting everybody's time.

How software ends up in repositories is a separate and very complex discussion. Suffice to say, meritocracy is not the decisive factor there.

Third, quite a few FLOSS projects are ideological in nature, and overselling is an almost conscious strategy for many FLOSS proponents, designed to lure in audiences from proprietary systems, or at least show them that free software is on par. This includes eager users, bloggers and Linux-focused journalists, who write exaggerated positive reviews of Linux software. They also tend to downplay the importance of functioning software and instead focus on the value of its licensing.

Finally, the sentiment that is the subject of this very article is quite pervasive and definitely makes it socially challenging to question the quality of FLOSS, since "people are doing it for free in their spare time".

In fact, both these reasons (ideological interest and "people are doing it in their spare time") create peer pressure and attempt to silence critics. I have seen people being harassed online for complaining about FLOSS. I myself was called a corporate shill for politely implying that a commercial product was much better than a FLOSS one in terms of functionality. Any of my articles being even slightly critical of Linux are often greeted with an accusation of being biased because my salary is probably being paid by a company which produces proprietary software.

The end result of all of this is that information about the actual value of FLOSS is hard to come by. This especially concerns Linux-only software, where the amount of outside reviewers who know what they are talking about is low, and where actual users are unlikely to come forward due to peer pressure, and the operating rule is that you either praise FLOSS, - or say nothing at all.

Note: when I made a critical video about the OpenShot video editor, many commenters assumed it's a video about how great OpenShot is, and without even watching the video left comments along the lines of "OpenShot is great, I also use kdenlive!" This just shows how unaccustomed people are to criticism of FLOSS.

3. Criticism directed at FLOSS is not very consequential

I claim that criticism is more consequential for a commercial product and much less so for a FLOSS one.

In case of a commercial product, a negative review might stifle sales, but at the same time put pressure on the developer to lower their prices or fix the product. In extreme cases it might even make them go out of business.

In case of a FLOSS product the real world consequences of criticism are significantly muted. Key reason for this is that a FLOSS developer is typically not financially dependent on the success of their program. So while criticism might put pressure on the developer to fix the product, essentially criticism is easier to ignore. In fact, for many FLOSS developers more users means more routine boring work, like writing documentation, having to prioritize and fix bugs as opposed to working on exciting new functionality, etc. So quite a number of FLOSS developers don't feel any need for their programs to have bigger userbases.

Another common method to shield from criticism is to just directly say that you don't care about anyone's opinion and will not respond to bug reports and feature requests, which makes you at least somewhat immune to negative feedback. This attitude is at the core of many FLOSS communities, where if you don't like it - "go and fix it yourself".

Apart from personal attitudes of developers, there is a structural reason that makes criticism largely inconsequential. The FLOSS world is very unlike a free market, or rather it is a free market permeated with established monopolies: as soon as a given program reaches a certain level of maturity and active development takes off, forking the project or starting a project from scratch with the goal of implementing the same kind of software is generally frowned upon as a waste of community resources.

For instance, there is but a single serious raster graphics package on Linux - GIMP. If GIMP does not satisfy the user - there are virtually no alternatives that are able to boast comparable stability and initial feature set. If one sees several programs being developed to achieve a similar use case, then it is a good bet that none of them are reliably good.

But once a monopoly is established, developers are obviously under very little pressure to make their product competitive, by virtue of there being no competition. This allows them to shrug off a lot of criticism, work at their own pace, prioritize new features over stability and over polishing existing functionality, spend time on experiments that are incomplete for years, etc.

Although in theory developers can be penalized for their unproductive behavior by the program being forked, this rarely happens. The cost of finding a team to take over the project is very high, and there are many examples when a very sophisticated, useful and powerful program is abandoned because no developer wants to pick it up, for one reason or another. So, dependence on the original development team is high.

This is not to say that all monopolistic programs have problematic and unhelpful developers, not at all. I am simply highlighting the incentive structure of established FLOSS projects, under which criticism is unlikely to be the driving force.

And, finally, in at least some cases there is no one developer and no one development team. Especially true for large projects, development is sometimes done by a constantly changing team of volunteers. And so nobody has a reason to feel personally targeted.

Taken together, this creates an environment where not only does criticism play little role in modifying developer behavior, in most cases criticism does not even reach developers, as they are not actively seeking it, and are sometimes a whole community of different people engaged with the project to different extents. Add to that the environment which is generally hostile to critics and propagating their concerns, - and the effect of criticism in the FLOSS world becomes miniscule to none. Plainly speaking, it produces very little real world consequences for developers of the software being criticized. Or praised, for that matter.

Of course, negative feedback is probably always unpleasant, and in theory a FLOSS developer might become so demotivated by criticism that they would decide to move on. But the claim is not that there is completely no effect, but rather than the effect is weak. I personally know of no case when negative feedback has brought a project to a halt. And while obviously there must be at least one such case, it is definitely not a very typical occurrence.

4. FLOSS criticism is likely to produce positive results for everybody in the long run

There are several reasons why people worry about FLOSS projects being criticized.

One of the main reasons is that negative criticism might scare users away from environments like Linux. While in many cases negative criticism might indeed scare users away, it is not clear what is accomplished by lying to them about the real state of affairs. Perhaps, it is hoped that getting more users on board would somehow help make Linux mainstream. Such an attitude usually comes from dedicated Linux users, who believe that as soon as other people see the benefits of Linux, they will not be able to walk away. But as the benefits that are important to die hard FLOSS fans are rather quirky - privacy to an extreme degree, being able to potentially change the code and distribute programs - this strategy is unlikely to succeed.

Moreover, people who try out Linux, having been lured in with false promises and shiny screenshots of supposedly professional grade software, quickly find out that they cannot get much done or that it is way too inefficient and limited for them, and then become the ambassadors of "stay away from FLOSS" in a way that no critic of FLOSS can ever hope to top. And while FLOSS critics like myself are not specifically addressing the non-FLOSS public and can even be thought of as internal critics of sorts, disgruntled users are going to be talking to people outside the FLOSS community.

Lying to potential users about how good a given FLOSS program is, is a fine way to create more of what was attempted to prevent. It's very much like the Streisand effect.

At the same time even from my experience of producing Bad Geek videos the effect of honest criticism on potential developers might be the opposite: they may feel that there is a niche for them in the FLOSS world, which they might be motivated to fill. If it is commonly known that FLOSS games are weak, quite a few developers might decide to put in work to create a decent FLOSS game and become known in the community.

Another reason for a negative reaction to FLOSS criticism may come from an intuition of "this person is doing it out of the kindness of their heart, in their spare time, and you dare criticize their hard work!" Therefore, they think that criticizing someone's gift is wrong, because perhaps it will discourage people from making gifts. Or there may well be only an intuition, and nothing pragmatic behind it.

We have addressed this in section 1, by saying that if someone does something for free, misrepresenting the usefulness of what they have done is still cheating. It's also very insulting and assumes that people's gratitude and respect can be bought merely by providing something at no cost.

And, finally, a bigger perspective could be at play, although I have not heard this argument made specifically in this context by anyone: you can say that the important bit about FLOSS is that software is released under a license that allows people to fix it. Therefore, by licensing it in a particular manner the developer allows you to just fix it, or complain in a constructive manner by submitting bugs.

First of all, this actually does not address the issue of misrepresenting the state of affairs. If a program's description clearly states that it is not yet ready for prime time and users are expected to help out by filing bugs - that's one case. But if the software is praised as well functioning and presented as "Award-winning!" and then crashes when you start it - this is not a time to remind people to keep their mouths shut and file bugs. This is a time to go and change the website to a more humble description.

Second, forget the "fix the problem yourself" knee-jerk reaction and how unrealistic this is in real life - even filing bugs is a lot of work! Frequently, filling out a ticket takes time and dedication: to make sure this is not a duplicate of an already existing bug, being prepared to register at a particular request tracking service, give a detailed description of the problem, respond to additional requests for additional logs, project dumps, etc.

And all of this comes with no guarantees that the bug will ever be fixed. Many of us have went through this process numerous times and some of those tickets are still unresolved. If there would be a system in place which would allow users to influence the priority of what is being worked on, then this might pay off. But in reality, as you are free to file bugs, developers are as free to ignore them. And so, by this logic you are somehow obliged to help the software, but developers continue to not be obliged to you at all.

If nothing in the description of the software or its website indicates that it has serious unresolved issues, I don't see how a user is expected to not feel cheated and what makes them obliged to expend even more effort to file bugs or fix things themselves. And if developers owe nothing to users, users definitely owe nothing to developers. It's a two-way street.

On the other hand, I am not aware of any area of human expertise where prohibiting criticism was a good way to move forward. If anything, taking a critical look at ourselves helps us more than anybody else. FLOSS developers are used to looking at everything from the point of view of developers. Receiving feedback from people who regard themselves as users is bound to be useful.

Creating a culture of higher standards might force distribution maintainers to remove all the dysfunctional software from their repositories and take an honest look at the amount of good software available to Linux users today. A more accurate representation of the actual state of affairs is the only way to improve. Adding barely playable games and games from 48 hour contests to official Ubuntu repositories is not a way to add value to the world of FLOSS.

5. Conclusion

In conclusion:

• I maintain that voluntary work in general and FLOSS in particular should be criticized freely, with no limitations. If developers want to modify the behavior of critics, all they have to do is be honest about their software. I would find it impossible to make a critical video about a game that honestly says it is in the beginning stages and many things are not done well yet. I would instead be compelled to help people find more developers and players to test if I would find the game interesting.
• FLOSS has a tendency to be oversold, which is why the FLOSS world needs more criticism, not less.
• Fanatics of FLOSS should not worry about critics like myself, because due to the way FLOSS communities are structured, the effect of criticism in real world consequences is extremely low anyway
• And to the extent that it is consequential, it can only bring positive results in the long-term. It is okay to admit certain things suck. Only then we can start working towards making them better.

6. Examples

In this section I present software which I and others have criticized, and why I believe it deserves every bit of criticism.

OpenShot Video editor

OpenShot Video editor has always been presented as a powerful piece of software with a long list of features. It's current website boasts an impressive introduction:

The site also treats us to dramatic statistics:

"It is now available on Linux, Mac, and Windows, has been downloaded millions of times, and continues to grow as a project!"

"Translated into 70 languages!"

There is even a section where the developer tells the story of the project. Its title is "Once upon a time, in a land with no good video editors…" and the first sentence boldly says "Behind every great program, is a great story."

Any person looking at the website would be left with an understanding that they are looking at a mature video editor, used by millions of users around the world, winning awards and being a staple in FLOSS video editing.

What's the reality?

Reality is diabolically different: OpenShot is barely usable. It's very slow, it's prone to constant crashes and its instability is legendary. Reviews at resources like AlternativeTo are revealing, with the majority of people giving OpenShot 1 and 2 star reviews. My own experience with OpenShot has been similar and there is a Bad Geek review of it on my YouTube channel.

So how is this discrepancy possible? How can a program be advertised as award-winning and stunning, and yet fail so miserably once transported to our computers?

As OpenShot developers say, "behind every great program, is a great story".

This story started with OpenShot 1. The program quickly began to draw harsh criticism from its users. In their reaction to complaints developers have demonstrated a lack of responsibility and maturity by choosing to blame it all on GTK and MLT - the libraries they used for the UI and video processing. According to them, OpenShot itself is very stable, the problem lies within those libraries.

This was not a random outburst. In email correspondence with the author of this blog post, which has outlined the numerous failings of OpenShot, the lead developer wrote them and said that he finds their criticism unhelpful, and that the problem is not OpenShot, but the MLT library.

The excuse is fake. Choosing which libraries to use is one of the decisions that the development team has to make, and if they've made it, they should take responsibility. At the very least, they should stop calling their software great and easy to use, because having to fight crashing sprees in the middle of a project is not what most people would understand as ease of use.

Neither the agonizing instability of the MLT library, nor the alleged deficiencies of the GTK library were ever mentioned on the website. And judging by the attitude of the developers, they probably did not see a problem with that.

Having blamed everything on MLT, they promised to produce their own video processing library, and launched a successful Kickstarter campaign. Having almost no options for video editing on Linux, many of us eagerly waited.

Unfortunately, the result was disappointing and continues to be disappointing to this very day. Three years after OpenShot 2 saw the light of day, the program continues to be as unstable as OpenShot 1. I guess MLT was not the only problem…

In fact, my experience with OpenShot 2 is much worse than with OpenShot 1, because it typically crashes or freezes even faster. I rarely get to do anything at all. Which makes me mesmerized by the tutorials that OpenShot developers put out: I am baffled, but not by the "stunning new functionality", but by how they managed to add a video to the timeline and not have the software crash or freeze like it does for me.

The website, on the other hand, continues to portray OpenShot as capable popular software with a reinvigorated rigor.

This story also demonstrates in abundance the low impact that criticism has on FLOSS. In spite of all the complains throughout the 9 year period of existence of OpenShot, the recent update mentions stability only as one of the last points. The update talks about 7 new effects, audio mixing and auto rotate. And only then improved stability is mentioned.

In the commercial world this is unthinkable. If someone wanted to sell a video editor which has such a high level of stability issues, the update would be focused on stability only. This would be the biggest concern and 99% of time and effort would be spend on stabilizing the program.

But this is not how the world of FLOSS operates.

Attal

Attal is available in official Ubuntu repositories and is described as a turn-based strategy game that can be played alone or against others through a network.

It is a FLOSS version of Heroes of Might and Magic. The game is virtually unplayable. Proudly listed in the repositories as version 1.0, it creates an impression of a barely started prototype, with poorly drawn assets and bare bones gameplay. Most of the information is not filled out and information boxes would frequently pop up completely empty. Two first scenarios in the game are titled "Test Fight win" and "Test Quests". In other words, a supposedly complete game (version 1.0, remember?) has test quests that are still present in the game!

And yet this "game" is in official repositories, taking up space and creating an illusion that Linux has lots of FLOSS games.

Nothing in the description hints at the program being a prototype. The website has been unavailable since 2006, although to be fair the site, which I was able to look at through the Wayback Machine, did say that Attal "is a software game still in development." How it ended up in official repositories and why would it continue to be ported year after year is beyond me. It is obvious the project is long dead. And the game is outdated by all standards anyway, chances of it being picked up by a new development team are worse than chances that someone writes a sequel to my Text Game.

Animals

Animals is described as a "traditional AI animal guessing engine using a binary tree DB". The description then goes on to elaborate: "You think of an animal, and this package tries to guess it... when it's wrong, you teach it about your animal."

But the catch is this:

"To be more flexible and help educational aspect this game does not contain an initial database. This also allows it to be used for non animals like guessing of tools or locations."

Get it? To be more flexible. And help the educational aspect of the game.

Yes, maybe there is a very narrow use case for such a program in educational purposes, for children (maybe). But could they have supplied an optional database with a game? Wouldn't this have been much better for the "educational aspect of the game"? Or is the purpose to teach kids how to laboriously tell the AI what to do through a command line? Where's the value in that? (I can already see geeks from all over the world preparing to defend this)

In all honesty, this looks more like a way to justify not having to work on an animal database. And the scope and quality of the application in general creates an impression of it being somebody's programming class assignment.

My question is this: why is this in the official repositories?