Conception, the twin problems and souls.

People used to believe that women were like a field and men plowed them and planted a seed and those seeds grew into babies.  There are echoes of this belief in the current right to life movement when they say "life begins at the moment of conception."  Apart from the obvious: "so sperms and eggs aren't alive?"  What does this really mean?  That at the "moment" of conception a fertilised egg suddenly gets a soul?  How long does conception actually take?  What part of that process is the "moment"?  Of course it begs the question, does a fertilised egg/fetus get a "soul" suddenly or gradually.  Does that question even have any meaning?  Why should having a soul be an on/off binary thing. 

Then there are the twin problems. 

The first is that identical twins both originate from the same fertilised egg.  Does this mean they share the same soul?  That they only have half a soul each?  That their soul divides into two?  Can souls do that?  That one of them gets another soul when the fetus divides into two?  Which one gets the new one and which the old one? 

The second twin problem is that sometimes with fraternal twins, one of the twins is absorbed by the other.  The "surviving" twin can become a chimera with both sets of genes.  Does this mean they have two souls?

 

Allow only one program at a time to access a resource in linux

I had several programs I needed to run but I wanted to make sure only one program got access to a resource at the same time.  So I wrote a short and simple perl script that would do this.  This script needs a directory in /var/run that it can write to.  Since the script doesn't run as root, I have to create this directory after each reboot.

The script is invoked like this:

1nstance.pl -d /var/run/once -n com1 -- my-program my-args

It will run my-program if there is no other script accessing the named resource com1.  If there is another script currently running and using com1 then it will just quit.

The script can be downloaded from here.

--------------------------1nstance.pl-------------------

#!/usr/bin/perl -w
#
#(c) copyright 2015 Kim Holburn
# Licensed under GPLv3

use strict;
use Getopt::Long;

my $verbose = 0;
my $help = 0;
my $myname = "";
my $mydir = "/var/run/once";
my $PID = 0;
my $time = 0;
my $timeout = 0;

Getopt::Long::Configure('require_order');
GetOptions ('verbose+' => \$verbose,
            'directory=s' => \$mydir,
            'name=s' => \$myname,
            'timeout=i' => \$timeout,
            'help|?' => \$help);

if ($help) {
  print <<EOM;
one instance - make sure a program only runs one instance.
usage $0:
$0 [options] -- <command> [ARGS]
  options:
    -h|-?|--help - print this screen
    -d|--directory <directory> 
       directory to store run files. 
       Default is /var/run/once
    -t|--timeout <n> 
       time in seconds that a program is allowed to run 
       before being considered hung and will be killed
       -t=0 means no timeout. The running process will 
       be left untouched.
    -v|--verbose - print extra messages
    -n|--name
       name of process or resource to be run once
       The default is the name of the program.    

EOM

  exit;
}

if ($timeout < 0) { die "$0 ERROR: specified timeout less than zero ($timeout)"; }
if (10000 < $timeout) { die "$0 ERROR: ($timeout) too large"; }
if ($mydir !~ m{^/}) { die "$0 ERROR: directory ($mydir) is not an absolute path!"; }
if (! -d $mydir) { die "$0 ERROR: directory ($mydir) does not exist!"; }

# first argument is command to run
my $path = shift;
if (!$path) { die "$0 ERROR: No command "; } 
my $program = $path;
# command must be a full absolute path.  
my $dir = ".";
my $args="";
if ($program =~ m#/#) {
  $program =~ s#^.*/##;
  $dir =~ s#/[^/]*$##;
}
if (!$program) { die "$0 ERROR: program name invalid.  Must be a filename"; } 
if (! -e $path) { die "$0 ERROR: program ($path) does not exist"; }
if (-d $path) { die "$0 ERROR: program ($path) is a directory"; }
if (! -x $path) { die "$0 ERROR: program ($path) not executable"; }
if (!$myname) { $myname=$program; }

my $run1 = "$mydir/$myname";

sub deleterun {
  if ( -e $run1 ) {
    unlink $run1;
    if ( -e $run1 ) { die "$0 ERROR: cannot delete file ($run1)"; }
  }
}

if ($verbose) {
  print "debug timeout=($timeout) dir=($mydir) name=($myname) \n";
  print "debug v=($verbose) 1=($run1) prog=($program) args=(";
  print join (")(",@ARGV);
  print ") \n";
}

if (open(my $fh, '<', "$run1")) {
  while (<$fh>) {
    chomp ;
    if (/^\s*$/) { next; }
    if (/^PID (\d+)$/i) { $PID = $1; }
    elsif (/^TIME (\d+)$/i) { $time = $1; }
  }
  close $fh;
  # no PID or time
  # this shouldn't happen and probably means we have the wrong file
  if (!$PID or !$time) { die "$0 ERROR: run file ($run1) has no PID or time"; }
  chomp (my $proc = `ps hp $PID -o %c`);
  # orphaned run file, delete
  if ($proc eq "") { deleterun; }
  else  {
    # another instance running
    my $timediff = time - $time;
    if (0 == $timeout or $timediff <= $timeout) { die "$0 ERROR: another instance of $myname ($proc) running"; }  
    print STDERR "$0 ERROR: another instance of $myname ($proc) has probably hung\n";
    # another process has probably hung
    # grab its children
    my @PIDS = (`ps h --ppid $PID -o %p`, $PID);
    kill ('TERM', @PIDS);
    chomp ($proc = `ps hp $PID -o %c`);
    # couldn't kill it.  Give up.
    if ($proc) { die "$0 ERROR: can't kill instance ($myname) ($proc) PID ($PID)"; }
    # it died OK.  Delete run file if possible.
    deleterun;
  }
}

if (1 < $verbose) { print STDERR "creating run file ($run1) ... \n"; }
open(my $fh, '>', "$run1") or die "$0 ERROR: opening file ($run1) ($!)" ;
print $fh "PID $$\n";
print $fh "time ", time, "\n";
close $fh;

#If something bad happens delete the run file
sub cleanup {
  if (-e $run1) {
    if (1 < $verbose) { print STDERR "Deleting run file...\n"; }
    unlink $run1;
  }
  exit;
};
$SIG{'INT'}=\&cleanup;
$SIG{'TERM'}=\&cleanup;
$SIG{'QUIT'}=\&cleanup;

if ($verbose) { print STDERR "Starting ....\n"; }

system ($path, @ARGV);

cleanup;

Detecting Rogue DHCP servers 2

So my script detecting rogue DHCP servers worked and worked well but having two or three DHCP servers covering the same scope is somewhat problematical.  So we eventually gave in and changed configuration to having a main DHCP server and failover servers.  I had to change the logic slightly to get the script to cover that.  The new version has a mode where it will only alert if it detects a rogue DHCP server it has not been told about, or gets no response from any of the servers in its valid server list.  One or more valid servers and all is right with the world.

As before, this script works in nagios.  I run nagios on ubuntu.  The scripts in the nagios package reside in /usr/lib/nagios/plugins.   Maybe there's a good place to put your own scripts but I put mine in there too (/usr/lib/nagios/plugins/check_rogue_dhcp.pl).

This script uses the nagios builtin DHCP checker: /usr/lib/nagios/plugins/check_dhcp.

Then you need a plugin command config file.  I edited the dhcp.cfg command file (/etc/nagios/plugins/) and added these lines:

# 'check_rogue_dhcp' command definition
define command{
   command_name check_rogue_dhcp
   command_line /usr/lib/nagios/plugins/check_rogue_dhcp.pl -f '$ARG1$' '$ARG2$' '$ARG3$'
}

Then to actually invoke the check you need to define a nagios object where you give the command the IP addresses of the servers (12.34.12.34 etc).  That may need a hostgroup or some other object depending on how you have nagios set up

#check that no rogue dhcp services are running
define service {
   service_description rogue-dhcp
   check_command check_rogue_dhcp!12.34.12.34!12.34.12.45
   use generic-service
   notification_interval 0 ; set > 0 if you want to be renotified
}

There's various ways to do this and I'm not great at strategic configuration of nagios, so I'll leave that to you.

The script can be downloaded from here.

The script:

------------check_rogue_dhcp.pl-------------------

#!/usr/bin/perl -w
# nagios: -epn
# the above line makes nagios run the script as a separately.
# rather than as part of nagios.
use POSIX;
use lib "/usr/lib/nagios/plugins";
use utils qw(%ERRORS);

sub fail_usage {
  if (scalar @_) {
    print "$0: error: \n";
    map { print "   $_\n"; } @_;
  }
  print "$0: Usage: \n";
  print "$0 [<options>] <server> [<server> [<server>]] \n";
  print "$0 [<options>] [-s <server> [-s <server> [-s <server>]]] \n";
  print "    options:  \n";
  print "      [-v [-v [-v]]] (verbose) \n";
  print "      [-t  <secs>] (wait this number of seconds)   \n";
  print "      [-f] (fuzzy - ok if one or more of the designated servers answer)   \n";
  print "      [-F] (force (default) - all the designated servers must answer)   \n";
  print " \n";
  exit 3 ;
}

my $verbose = 0;
my %servers=();
my $opt = "-t 5";
my $time = 5;
my $force=1;

## for some reason I can't test for empty ARGs in the while loop
@ARGV = grep {!/^\s*$/} @ARGV;

# examine commandline args
while ($ARGV=$ARGV[0]) {
  my $myarg = $ARGV;
  if ($ARGV eq '-s') {
    shift @ARGV;
    if (!($ARGV = $ARGV[0])) { fail_usage ("$myarg needs an argument"); }
    if ($ARGV =~ /^-/) { fail_usage ("$myarg must be followed by an argument"); }
    if (!defined($servers{$ARGV})) { $servers{$ARGV}=1; }
  }
  elsif ($ARGV eq '-t') {
    shift @ARGV;
    if (!($ARGV = $ARGV[0])) { fail_usage ("$myarg needs an argument"); }
    if ($ARGV =~ /^-/) { fail_usage ("$myarg must be followed by an argument"); }
    if ($ARGV !~ /^(\d+)$/) { fail_usage ("$myarg must be followed by an number"); }
    $time = $1;
    $opt = "-t $time";
  }
  elsif ($ARGV eq '-f' ) { $force=0; }
  elsif ($ARGV eq '-F' ) { $force=1; }
  elsif ($ARGV eq '-h' or $ARGV eq '--help' ) { fail_usage ; }
  elsif ($ARGV =~ /^-/ ) { fail_usage " invalid option ($ARGV)"; }
  elsif ($ARGV =~ /^\d+\.\d+\.\d+\.\d+$/)
    # servers should be ip addresses.  I'm not doing detailed checks for this.
    { if (!defined($servers{$ARGV})) { $servers{$ARGV}=1; } }
  else { last; }
  shift @ARGV;
}

if (scalar @ARGV) { fail_usage "didn't understand arguments: (".join (" ",@ARGV).")"; }  
my $serversn = scalar keys %servers;

if ($verbose > 2) {
  print "verbosity=($verbose)\n";
  print "servers = ($serversn)\n";
  if ($serversn) { for my $i (keys %servers) { print "server ($i)\n"; } }
}

if (!$serversn) { fail_usage "no servers"; }
my $responses=0;
my $responders="";
my @check_dhcp = qx{/usr/lib/nagios/plugins/check_dhcp -v $opt};
foreach my $value (@check_dhcp) {
  if ($value =~ /Added offer from server \@ /i){
    $value =~ m/(\d+\.\d+\.\d+\.\d+)/i;
    my $host = $1;
    # we find a server in our list
    if (defined($servers{$host})) { $responses++; $responders.="$host "; }
    else {
      # we find a rogue DHCP server.  Danger Will Robinson!
      print "SERVICE STATUS:CRITICAL: Rogue DHCP service running on $host";
      exit $ERRORS{'CRITICAL'}
    }
  }
}
if ($responses == $serversn) {
  # we saw all the servers in our list.  All is good.
  print "SERVICE STATUS:OK: $responses of $serversn Expected Responses to DHCP Broadcast";
  exit $ERRORS{'OK'};
}

if ($responses == 0) {
  # we found no DHCP responses.
  print "SERVICE STATUS:CRITICAL: no DHCP service responded";
  exit $ERRORS{'CRITICAL'}
}

# we found less DHCP servers than we should have. Oh Nos!
$responders =~ s/ $//;
if ($force == 1) {
  print "SERVICE STATUS:WARNING: $responses of $serversn Responses to DHCP Broadcast. Only ($responders) responded. ";
  exit $ERRORS{'WARNING'};
}
else {
  print "SERVICE STATUS:OK: $responses of $serversn Responses to DHCP Broadcast. Only ($responders) responded. ";
  exit $ERRORS{'OK'};
}

The myth of competition and the free market.

Mathematically the free market is not a stable environment.  It quickly becomes an oligopoly and from there it's only a step or two to a monopoly.

The point is that whenever and to the extent that there is a free market, companies must become efficient to compete.

But free markets only exist when companies compete, when they agree to compete, when they cooperate to compete.  The bulk of any company's business is done on a cooperative basis and that cooperation underlies everything we do as a society.

Free markets cannot exist if companies cooperate.  This is why some forms of cooperation have to be made illegal in order to create a "free market" but companies in a competitive environment can still cooperate without an agreement, without even communicating with each other. To state that more plainly: there is no "free market" without government intervention.

As soon as a market has reached the oligopoly stage, there are entrenched players and at this point new players find it very difficult to impossible to enter the market.  An oligopoly company gets far more benefit from cooperating in the market than competing.  Even if there is no overt agreement to cooperate, oligopoly players will benefit more from cooperation than competition.  They will especially cooperate to lock new and potential competitors out of their markets.  They may compete in a desultory way with other oligopoly members but ultimately the oligopoly state benefits them more than competition.

It is the oligopoly companies that are always extolling the virtues of the free market.

The primary enemy, the primary competitor of a company is competition itself.  Without competition companies don't have to be efficient, they can live off the fat of a monopoly or oligarchy.  There is no transparency.  Companies will always fight hardest to not have to compete.   Not having to compete is a massive benefit for them.

Companies in a free market, when it occurs and to the extent it occurs, will compete and will try to end that competition.  Competition will gradually go away for various reasons, because companies go bankrupt, give up in that market,  get bought out or merge and as the major players in a market get bigger, the market becomes an oligopoly and finally a monopoly and competition ceases.  Legislation can lengthen this process, that is can help maintain competition for longer, but cannot really stop it.  As the companies get large enough and rich enough they will lobby governments to change the rules to their benefit or they will go transnational and be out of the reach of governments or apply pressure in other ways.

When companies lobby for changes in law they will always site "the free market" and this is nearly always an indicator of an oligopoly.

ngraph: another random text generator.

In my series of scripts to generate random text for fun and for helping create secure passwords this is my latest.  This new set of scripts generate text based on frequency weighted random choice of ngraphs.  I use it sometimes to generate word-like things  to create passwords.  There may be other uses, for instance creating random text with English characteristics.

ngraph:  Generating text from frequency weighted letter combinations

An ngraph is a group of n consecutive letters occurring in a language.  (This is my definition, there may be another word for it but I couldn't find it.) A set of ngraph frequencies is a set of the number of times each ngraph is used in a group of texts.  So for instance for n=1 we have the frequencies of the letters.  For n=2 we have the set of frequencies of the digraphs, for n=3, the trigraphs etc.  Because it is easy to do on a computer, I include some punctuation (space return - ' , . ; ! ? &).  I use these files among others:
The complete works of Jane Austen and the complete works of William Shakespeare.
Any texts would do.  Gutenburg texts have a few oddities that the script is designed to work with.

There are two main scripts and a subsidiary script.  (ngraph.pl, dbfill.sh and ngraph-db.pl)

The first script is "ngraph.pl": this script has two separate functions.
Firstly it reads a series of files with presumably text in a human language and generates a set of ngraphs for an "n" you specify.
Then ngraph.pl will generate a random set of text based on the ngraphs and their frequency.

Alternatively it can just output the set of ngraph frequencies as text or sql.  The reason for this is that reading the files and creating ngraph frequency tables is a resource intensive process so I decided to create a database of ngraphs and to generate text from that database.  I found a database with n = 1 to 5 to be most useful and above 5 the amount of data gets massive and more actual words are generated. 

The second script "dbfill.sh" is a subsidiary script.  It creates the database and populates it with ngraphs using the first script.

The third script is "ngraph-db.pl"
This uses the database and generates text based on the ngraphs in the database.  Because it has access to a database with ngraphs of say n = 1 to 5 it can generate text from random sized ngraphs as well as a single ngraph.

The generated text can include words but it mostly has word-like things that are a bit memorable but not actual words.  I never use the generated text directly to create passwords but pick and choose bits and let parts of the text inspire a password.

The scripts are available under the GPL here.

Here is a sample output:

$ngraph-db.pl -W -c 1000 -2 -g 1-4

Detecting Rogue DHCP servers

I don't know why there aren't more simple ways to detect rogue DHCP servers.  It can have nasty effects on a network and can be difficult to diagnose and it's easy for it to happen. Someone can plug in a machine set to share its internet connection and bam.

I wrote a perl script to detect rogue DHCP servers in nagios.  I found one on the web here but I wanted it a bit more flexible so the hosts it was testing were in the nagios config files rather than in the script itself.  It's a debian policy thing I like.

So I adjusted the script.  It requires the check_dhcp plugin which for me was part of my ubuntu nagios install.  It takes options "-v" and a list of servers.   In your nagios-plugins/configure file the command looks like:

/usr/lib/nagios/plugins/check_rogue_dhcp.pl!$ARG1$!$ARG3$!$ARG3$

The script can be downloaded from here. (Sorry the link is corrected now!)

Post on new version can be found here.

-----check_rogue_dhcp.pl-------------

#!/usr/bin/perl -w
# nagios: -epn
# the above makes nagios run the script separately.
use POSIX;
use lib "/usr/lib/nagios/plugins";
use utils qw(%ERRORS);

sub fail_usage {
  if (scalar @_) {
    print "$0: error: \n";
    map { print "   $_\n"; } @_;
  }
  print "$0: Usage: \n";
  print "$0 [-v [-v [-v]]] (server-ip) [(server-ip) (server-ip) ....] \n";
  print "$0 [-v [-v [-v]]] [-s] (server-ip) [[-s] (server-ip) (server-ip)....] \n";
  print " \n";
  exit 3 ;
}

my $verbose = 0;
my %servers=();

# examine commandline args
while ($ARGV=$ARGV[0]) {
  my $myarg = $ARGV;
  if ($ARGV eq '-s') {
    shift @ARGV;
    if (!($ARGV = $ARGV[0])) { fail_usage ("$myarg needs an argument"); }
    if ($ARGV =~ /^-/) { fail_usage ("$myarg must be followed by an argument"); }
    if (!defined($servers{$ARGV})) { $servers{$ARGV}=1; }
  }
  elsif ($ARGV eq '-v' ) { $verbose++; }
  elsif ($ARGV eq '-h' or $ARGV eq '--help' ) { fail_usage ; }
  elsif ($ARGV =~ /^-/ ) { fail_usage " invalid option ($ARGV)"; }
  elsif ($ARGV =~ /^\d+\.\d+\.\d+\.\d+$/)
# servers should be ip addresses.  I'm not doing detailed checks for this.
    { if (!defined($servers{$ARGV})) { $servers{$ARGV}=1; } }
  else { last; }
  shift @ARGV;
}

# for some reason I can't test for empty ARGs in the while loop
@ARGV = grep {!/^\s*$/} @ARGV;
if (scalar @ARGV) { fail_usage "didn't understand arguments: (".join (" ",@ARGV).")"; }  

my $serversn = scalar keys %servers;

if ($verbose > 2) {
  print "verbosity=($verbose)\n";
  print "servers = ($serversn)\n";
  if ($serversn) { for my $i (keys %servers) { print "server ($i)\n"; } }
}

if (!$serversn) { fail_usage "no servers"; }
my $responses=0;
my $responders="";
my @check_dhcp = qx{/usr/lib/nagios/plugins/check_dhcp -v};
foreach my $value (@check_dhcp) {
  if ($value =~ /Added offer from server \@ /i){
    $value =~ m/(\d+\.\d+\.\d+\.\d+)/i;
    my $host = $1;
    # we find a server in our list
    if (defined($servers{$host})) { $responses++; $responders.="$host "; }
    # we find a rogue DHCP server.  Danger Will Robinson!
    else {
      print "SERVICE STATUS:CRITICAL: DHCP service running on $host";
      exit $ERRORS{'CRITICAL'}
    }
  }
}
# we saw all the servers in our list.  All is good.
if ($responses == $serversn) {
  print "SERVICE STATUS:OK: $responses of $serversn Expected Responses to DHCP Broadcast";
  exit $ERRORS{'OK'};
}
# we found no DHCP responses.
if ($responses == 0) {
  print "SERVICE STATUS:CRITICAL: no DHCP service responded";
  exit $ERRORS{'CRITICAL'}
}
# we found less DHCP servers than we should have. Oh Nos!
$responders =~ s/ $//;
print "SERVICE STATUS:WARNING: $responses of $serversn Responses to DHCP Broadcast. Only ($responders) responded. ";
exit $ERRORS{'WARNING'};

 

Government control of ideas

From before 1AD to around 1400AD (I believe there is no zero AD because we didn't have zero till much later - an Indian invention!) there were three main centers of civilization - Europe, China and India.  They were of similar population and developed culturally and technically quite similarly.  Then around 1400, Europe started an unprecedented scientific and technological development that continued until recently.  Even though China invented the basis of many of our modern technologies, it never developed them or put them together.  It is interesting to contemplate why this is so. 

My theory is that to really develop technically and scientifically we need two important things in our social environment.  The first is a reasonably stable society and the second is freedom of ideas from government intervention. 

The first part: stable society.  Now I've not read Indian history but I believe that India didn't really provide a stable social environment.  I'm happy to be corrected in this and one day I hope to get time to read Indian history.  China did  have a stable culture, although with interregnums where it dissolved into chaos, but for much of the time it had a stable society.  Europe had a stable society although it was fluid as wars and changes swept through it but overall there was stability.  Europe had geographic divisions which allowed it to maintain diverse political boundaries but with countries remaining connected in a way China and India didn't have.

The second part: freedom of ideas from government intervention shows Europe to be quite different to the others.  For most of history when China was stable, it was also monolithic and uniform with considerable government intervention in ideas and trade.  The monolithic nature of China meant government intervention in ideas was final, there was nowhere to go.

Europe on the other hand was politically and geographically very diverse.  While any one part may not have been stable there were always parts that were stable.  To some extent people could even move to stable parts.  Europe was always more diverse and divided than China. 

The critical point in Europe seems to have been around the time of the reformation.  In the first days of the printing press, the Catholic church had an early and very harsh copyright law.  Allegedly this was to save the "jobs" of monks copying books by hand in scriptoriums but mainly to control and censor books.  The penalty for breaking the copyright law was death.  The law applied more or less in catholic countries, which was to say, much of western Europe.  But from the time of the reformation it was possible for people to by-pass the copyright law by printing in a non-catholic country.  This was how Gallileo's books were published.  His books were banned by the church, and therefore in catholic countries, from around 1633 to 1835.  In fact from the time of the reformation Europe experienced a period of scientific development unprecedented in world history. 

It seems clear to me that the geographic and political diversity and contention in Europe provided a key ingredient here, a way of by-passing government intervention in ideas. 

As Europe consolidates and creates uniform intellectual property laws: ie government intervention in ideas, it will cut off the critical driver that has moved it into the forefront of scientific development.  The US is in the process of doing this. 

Some background:  China and Europe 1500 - 2000

Update: I was reading this article in wired:
Why Black Market Entrepreneurs Matter to the World Economy and it gives an idea of the size, scale, energy and innovation of unregulated economies.

There seems to be a very fine balance between the government control you need to get a stable society and the over-control that stifles and censors ideas.  It seems to me that no single government can get it right.