The myth of competition and the free market.

Mathematically the free market is not a stable environment.  It quickly becomes an oligopoly and from there it's only a step or two to a monopoly.

The point is that whenever and to the extent that there is a free market, companies must become efficient to compete.

But free markets only exist when companies compete, when they agree to compete, when they cooperate to compete.  The bulk of any company's business is done on a cooperative basis and that cooperation underlies everything we do as a society.

Free markets cannot exist if companies cooperate.  This is why some forms of cooperation have to be made illegal in order to create a "free market" but companies in a competitive environment can still cooperate without an agreement, without even communicating with each other. To state that more plainly: there is no "free market" without government intervention.

As soon as a market has reached the oligopoly stage, there are entrenched players and at this point new players find it very difficult to impossible to enter the market.  An oligopoly company gets far more benefit from cooperating in the market than competing.  Even if there is no overt agreement to cooperate, oligopoly players will benefit more from cooperation than competition.  They will especially cooperate to lock new and potential competitors out of their markets.  They may compete in a desultory way with other oligopoly members but ultimately the oligopoly state benefits them more than competition.

It is the oligopoly companies that are always extolling the virtues of the free market.

The primary enemy, the primary competitor of a company is competition itself.  Without competition companies don't have to be efficient, they can live off the fat of a monopoly or oligarchy.  There is no transparency.  Companies will always fight hardest to not have to compete.   Not having to compete is a massive benefit for them.

Companies in a free market, when it occurs and to the extent it occurs, will compete and will try to end that competition.  Competition will gradually go away for various reasons, because companies go bankrupt, give up in that market,  get bought out or merge and as the major players in a market get bigger, the market becomes an oligopoly and finally a monopoly and competition ceases.  Legislation can lengthen this process, that is can help maintain competition for longer, but cannot really stop it.  As the companies get large enough and rich enough they will lobby governments to change the rules to their benefit or they will go transnational and be out of the reach of governments or apply pressure in other ways.

When companies lobby for changes in law they will always site "the free market" and this is nearly always an indicator of an oligopoly.

ngraph: another random text generator.

In my series of scripts to generate random text for fun and for helping create secure passwords this is my latest.  This new set of scripts generate text based on frequency weighted random choice of ngraphs.  I use it sometimes to generate word-like things  to create passwords.  There may be other uses, for instance creating random text with English characteristics.

ngraph:  Generating text from frequency weighted letter combinations

An ngraph is a group of n consecutive letters occurring in a language.  (This is my definition, there may be another word for it but I couldn't find it.) A set of ngraph frequencies is a set of the number of times each ngraph is used in a group of texts.  So for instance for n=1 we have the frequencies of the letters.  For n=2 we have the set of frequencies of the digraphs, for n=3, the trigraphs etc.  Because it is easy to do on a computer, I include some punctuation (space return - ' , . ; ! ? &).  I use these files among others:
The complete works of Jane Austen and the complete works of William Shakespeare.
Any texts would do.  Gutenburg texts have a few oddities that the script is designed to work with.

There are two main scripts and a subsidiary script.  (ngraph.pl, dbfill.sh and ngraph-db.pl)

The first script is "ngraph.pl": this script has two separate functions.
Firstly it reads a series of files with presumably text in a human language and generates a set of ngraphs for an "n" you specify.
Then ngraph.pl will generate a random set of text based on the ngraphs and their frequency.

Alternatively it can just output the set of ngraph frequencies as text or sql.  The reason for this is that reading the files and creating ngraph frequency tables is a resource intensive process so I decided to create a database of ngraphs and to generate text from that database.  I found a database with n = 1 to 5 to be most useful and above 5 the amount of data gets massive and more actual words are generated. 

The second script "dbfill.sh" is a subsidiary script.  It creates the database and populates it with ngraphs using the first script.

The third script is "ngraph-db.pl"
This uses the database and generates text based on the ngraphs in the database.  Because it has access to a database with ngraphs of say n = 1 to 5 it can generate text from random sized ngraphs as well as a single ngraph.

The generated text can include words but it mostly has word-like things that are a bit memorable but not actual words.  I never use the generated text directly to create passwords but pick and choose bits and let parts of the text inspire a password.

The scripts are available under the GPL here.

Here is a sample output:

$ngraph-db.pl -W -c 1000 -2 -g 1-4