The basis of randomness

I was thinking today about the basis of randomness.

I'm still thinking about how to write this down so bear with me here.

After the invention of calculus, mathematicians saw the world as a sort of giant clockwork machine.  "Give me the initial conditions and the law of motion, and with calculus I can predict the future -- or better yet, reconstruct the past."  Einstein's spacetime implies the same thing, that all of time and space are fixed, that we can move forward or back, given the mathematical ability.  There is no randomness in this view of the universe.

And so we thought until the discovery of quantum theory.  The problem being that the world is not infinitely divisible.  Eventually everything must collapse into particles.  This collapse introduces a rounding or truncation error, that is the basis of randomness.  The universe is really digital not analog.  The present is the moment that the continuous changes into steps, quanta.  It's the present that changes infinitely divisible potential into particular actuality.  The collapse of the present is the reason we can't go forward or backward - forward - we can't know the future before it arrives, backward - we can't tell exactly what happened from what we know about the present.


Trampolines from oztrampolines online are dangerous. Don't buy them.

Trampolines from oztrampolines online are dangerous. Don't buy them.

Here is my experience.

In March of 2014 bought a 7 x 10ft Rectangular Trampoline from Oztrampolines online.  It was delivered fairly promptly but when I got it here is a picture of what I got. 

Actually a 6ft x 9 ft trampoline with an extra rail 1 foot around the outside.  But this is just disappointing not dangerous.  The legs are complicated and not especially stable, but again not dangerous, just annoying.

From about 18 months after I got the trampoline, the springs started breaking.  They would snap and would fly off and hit the person on the trampoline causing at least bruising.  Thank goodness they haven't hit anyone in the head or eye yet.  This continued on and off since.

5 or 6 have broken this way.  I complained to oztrampolines online and they said that the trampoline had a five year warranty but the springs had only a two year warranty and they would sell me a new set of springs cheaply.  Presumably with a two year warranty.   The complaints went on for a while and they would say: did you install the springs the right way round?  Yes I had.  Did you have the guard pads installed?  Yes I did. 

Everyone seems to believe oztrampolines when they say that the spring guard pads should stop this happening but the physics is quite clear, the pads stop someone landing on the springs and hurting themselves, but they don't stop the springs flying off and hitting the person on the trampoline. 

I went to my government consumer group to dispute the fact that the trampoline I bought was not fit for purpose, but they wouldn't take it any further.  I tried the checkout but they weren't interested. 

A friend had an idea to put cable ties on the springs to stop them flying off.  We did that but they don't work as it has just happened again.

I am now throwing the trampoline out as I believe it is unsafe to use as a trampoline.

1. Who sells dodgy springs in trampolines that break?  I have owned trampolines that are decades old that no spring has ever broken.

2. These springs all break at a similar point, the frame side hook.

The physics:  When you are at the bottom of a trampoline jump, your feet or what ever part of you is contacting the mat are well below the level of the frame.  The mat applies something like 7G* of acceleration to you at the lowest point.  That is the time the springs have the most force on them, and are most likely to break.  It is also the time when the springs are below and not touching the guard pads.  If the springs break at the frame side or anywhere in the middle, the part that is on the mat side will be propelled in the direction of the force, ie towards the person on the mat.

You can clearly see in this picture how the guard mat does not cover the springs at the bottom of a jump.


Here is a sketch of the forces at the bottom of a jump:

And the forces on a spring:
You can see that the guard pad does not cover the spring any more. If the spring breaks before the person reaches the bottom of a jump the force that caused the spring to break will launch the spring above the mat.  In the lower picture you can also see that the hook on the mat when it hits the mat side of the triangular clip will be launched upwards.


Bots, Metadata, big data, linkity and that time I broke the family googles.

I guess this started a long time ago when companies started collecting data from users on the internet.  What triggered me to write linkity was our government deciding to make ISPs collect and store our metadata.

I wrote linkity to create lots of metadata.  It looks at lots of things at random on the internet.  I wanted a program that didn't download much and so cost the user much, and didn't cause much traffic on any remote sites but went to lots of random websites and generated lots of metadata.  I believe it's completely legal to use this and hope it stays that way.

If our government wants to store our metadata, this program helps give them more.  It increases the signal to noise ratio on your metadata.  It's a bit of security by obscurity and probably an annoyance only.  Don't rely on it to do anything much.  It may give you a bit of plausible deniability.  If many people use it, it will create a massive haystack to look for needles in, so I want the program to scale without being too annoying on the internet.

I thought it would be relatively easy at first.  I would run a few searches for random words on google and scrape follow the links. 

That seemed to go well, but while I was just testing and tuning it, google objected and banned my house from searching for using a bot.  My family were not impressed.  I hadn't noticed because I don't use google that much.  They told me "fix the googles!"  Who'd've thought that google didn't like people using bots on google.  Google, who run more and more powerful bots than anyone else in the world or at least the public world.

I decided to stop and rethink my plan.  The googles came back to our house after a day or so.  Family happy again.

I guess google uses its massive machine learning to try and improve its responses to people's questions and runs a lot of checks to make sure it's really people.  I hadn't realised or hadn't thought about the massive numbers of robots and robotic activity out there in the wild internet doing all sorts of things, probably related to making money in good or bad ways. 

I had planned to make scrapers for other search engines, so I bought those plans forward.  I changed the program so it collected the links in a database instead of just viewing them and throwing them away.  Then I found I could harvest links from any page I visited, so I didn't have to do go to search sites very much. 

I did manage to break the googles again but this time their AI only seemed to have blocked my script briefly and not the rest of the household.  Clever googles.

Now I have it working, I can watch how my script wanders around the net. Sometimes it fixates on certain sites.  Sites that have a lot of subdomains.  It's fascinating how websites are networked through links.

Bug reports, ideas and comments welcome.

Oh and lynx developers, why have a non removable error message when you have a non-lynx useragent?  Why are we forced to hand over information about ourselves when we use the web? 

Technical details:
Download: linkity

It runs on linux systems, BSD or Mac.  It uses some unix utilities: lynx, sqlite3, perl and with perl it uses the URI::Escape module. 

On debian/ubuntu systems you can install them like this:
sudo apt-get install sqlite3 lynx liburi-perl

On a mac with MacPorts:  On a Mac with MacPorts:
sudo port install lynx sqlite3
I'm pretty sure MacPorts installs perl5 automatically and URI::Escape is:
sudo port install p5.24-uri

You could use CPAN, to get URI::Escape but I can't really help you with that.


randword: Generating memorable random passwords

This started when I decided to learn python by rewriting one of my old perl scripts in python.  randstring is a script to generate a random string of characters.  I use it sometimes to generate passwords, but password strings or random characters usually can't be remembered, at least not easily.  Passwords like that can be useful at times.  You need to store them in an encrypted password safe.

I have another script that generates more memorable passwords.  Some people I know, have found it useful.  There are always some passwords you need to be memorable.  For instance your login password and the password to your password safe.  randword generates a bunch of words from a dictionary.  XKCD style passwords, if you like.  In the process of examining it, I rewrote it in both perl and python, fixed some bugs and added some features.

In general a bunch of words can be much easier to remember and can be just as difficult or far more difficult  to crack.  I like to generate a bunch and choose a few at random.  4 or 5 or more words is OK.  Hint: misspellings are good but not if you can't remember what you did.  Passwords on websites are a bit mad at the moment with complicated rules, like: "there's an illegal character" or "you must have an upper-case letter and a number", or "that password is too short", or "too long" etc. 

New features of randword:
  • There's a couple of new options about output format, like camel case.   
  • randword can use any dictionary or word frequency lists as long as they have a fairly simple format - ie at least a word and an optional number at the start of each line. 
  • randword can also take a bunch of text and create dictionaries of words that it can use to generate random passwords.  
For word lists, I have used various texts, for instance Jane Austen's complete works, Shakespeare, Mark Twain, Chaucer.  There are many works that can be easily got from Project Gutenburg among other places on the net.  Also word lists and text that can be found at COCA or Lancaster University  etc.Since I only want ascii because I can't type non-ascii characters easily, I used unidecode (python or original perl version) to turn them into ascii.  Python unidecode comes with a command line script.  I wrote a very simple perl script to detect non-ascii characters, (not included) although working out what encoding a page is in is a kind of major headache and you need to know the encoding before unidecode will work, grrrr. 

The links below include word lists from Chaucer, Shakespeare, Mark Twain, and the linux word dictionary.

This is my original blog post on the scripts with all the links to the scripts and associated stuff.

randstring.pl randstring.py
randword.pl randword.py
some word lists
tarred and zipped archive of scripts and wordlists


In memory of my mother, Marjorie Pizer Holburn, 3rd April 1920 - 4th January 2016

Marjorie Pizer died on the 4th of January 2016. She is sadly missed.

It's impossible in a short time and with my limited time and writing skills to summarise her.  Here is my brief message.  In time I will perhaps add links to other people writing about her.  If you knew her please feel free to add a comment.

My mother was a poet, artist, psychotherapist and a deep thinker. She had many unconventional beliefs, which she held passionately. Politically, she almost always sided with the underdog; and she was troubled by the inequities in our world. She was inspired by the beauty of nature, and loved music, poetry and art of all kinds. She read 2 or 3 books a day, every day, on almost every subject imaginable, until near the end of her life. My mother was a member of all the local libraries; municipal, city and state. Brought up in Melbourne, she lived much of her life in Sydney where she went to the beach and swam nearly every day for forty years of her life.

Marje was one of my best friends for my whole life. I was always much more a scientist and she a poet and writer, our discussions were always wide ranging and broadened both our view points. She was and is a reference point for me for ideas and values. Although in later life I agreed with her ideas less, we always had lively and interesting discussions.

When I think of my mother I picture her surrounded by books and works of art and odd found objects from the beach. Pieces of driftwood, and a vase of fresh cut flowers on the table. We would eat a simple meal together, have a cup of tea and discuss the state of the world. This is how I will remember her.

Marjorie's books can be bought at lulu and apple and amazon.  They are published by Pinchgut Press (me). 

Daniela Torsh's obituary from the Sydney Morning Herald: Poetry Marjorie Pizer's vehicle of optimism and understanding. And her speech.

A memory

When my son comes home late,
He sits on my bed
And tells me about his day.
Someday he will remember this
When I will be no more,
When I have had my say
And gone before.
Then I will not exist
As I am now.
This me will be a memory
Of his when I,
Who now am here alone,
Have gone into oblivion.


Conception, the twin problems and souls.

People used to believe that women were like a field and men plowed them and planted a seed and those seeds grew into babies.  There are echoes of this belief in the current right to life movement when they say "life begins at the moment of conception."  Apart from the obvious: "so sperms and eggs aren't alive?"  What does this really mean?  That at the "moment" of conception a fertilised egg suddenly gets a soul?  How long does conception actually take?  What part of that process is the "moment"?  Of course it begs the question, does a fertilised egg/fetus get a "soul" suddenly or gradually.  Does that question even have any meaning?  Why should having a soul be an on/off binary thing. 

Then there are the twin problems. 

The first is that identical twins both originate from the same fertilised egg.  Does this mean they share the same soul?  That they only have half a soul each?  That their soul divides into two?  Can souls do that?  That one of them gets another soul when the fetus divides into two?  Which one gets the new one and which the old one? 

The second twin problem is that sometimes with fraternal twins, one of the twins is absorbed by the other.  The "surviving" twin can become a chimera with both sets of genes.  Does this mean they have two souls?



Allow only one program at a time to access a resource in linux

I had several programs I needed to run but I wanted to make sure only one program got access to a resource at the same time.  So I wrote a short and simple perl script that would do this.  This script needs a directory in /var/run that it can write to.  Since the script doesn't run as root, I have to create this directory after each reboot.

The script is invoked like this:

1nstance.pl -d /var/run/once -n com1 -- my-program my-args

It will run my-program if there is no other script accessing the named resource com1.  If there is another script currently running and using com1 then it will just quit.

The script can be downloaded from here.


#!/usr/bin/perl -w
#(c) copyright 2015 Kim Holburn
# Licensed under GPLv3

use strict;
use Getopt::Long;

my $verbose = 0;
my $help = 0;
my $myname = "";
my $mydir = "/var/run/once";
my $PID = 0;
my $time = 0;
my $timeout = 0;

GetOptions ('verbose+' => \$verbose,
            'directory=s' => \$mydir,
            'name=s' => \$myname,
            'timeout=i' => \$timeout,
            'help|?' => \$help);

if ($help) {
  print <<EOM;
one instance - make sure a program only runs one instance.
usage $0:
$0 [options] -- <command> [ARGS]
    -h|-?|--help - print this screen
    -d|--directory <directory> 
       directory to store run files. 
       Default is /var/run/once
    -t|--timeout <n> 
       time in seconds that a program is allowed to run 
       before being considered hung and will be killed
       -t=0 means no timeout. The running process will 
       be left untouched.
    -v|--verbose - print extra messages
       name of process or resource to be run once
       The default is the name of the program.    



if ($timeout < 0) { die "$0 ERROR: specified timeout less than zero ($timeout)"; }
if (10000 < $timeout) { die "$0 ERROR: ($timeout) too large"; }
if ($mydir !~ m{^/}) { die "$0 ERROR: directory ($mydir) is not an absolute path!"; }
if (! -d $mydir) { die "$0 ERROR: directory ($mydir) does not exist!"; }

# first argument is command to run
my $path = shift;
if (!$path) { die "$0 ERROR: No command "; } 
my $program = $path;
# command must be a full absolute path.  
my $dir = ".";
my $args="";
if ($program =~ m#/#) {
  $program =~ s#^.*/##;
  $dir =~ s#/[^/]*$##;
if (!$program) { die "$0 ERROR: program name invalid.  Must be a filename"; } 
if (! -e $path) { die "$0 ERROR: program ($path) does not exist"; }
if (-d $path) { die "$0 ERROR: program ($path) is a directory"; }
if (! -x $path) { die "$0 ERROR: program ($path) not executable"; }
if (!$myname) { $myname=$program; }

my $run1 = "$mydir/$myname";

sub deleterun {
  if ( -e $run1 ) {
    unlink $run1;
    if ( -e $run1 ) { die "$0 ERROR: cannot delete file ($run1)"; }

if ($verbose) {
  print "debug timeout=($timeout) dir=($mydir) name=($myname) \n";
  print "debug v=($verbose) 1=($run1) prog=($program) args=(";
  print join (")(",@ARGV);
  print ") \n";

if (open(my $fh, '<', "$run1")) {
  while (<$fh>) {
    chomp ;
    if (/^\s*$/) { next; }
    if (/^PID (\d+)$/i) { $PID = $1; }
    elsif (/^TIME (\d+)$/i) { $time = $1; }
  close $fh;
  # no PID or time
  # this shouldn't happen and probably means we have the wrong file
  if (!$PID or !$time) { die "$0 ERROR: run file ($run1) has no PID or time"; }
  chomp (my $proc = `ps hp $PID -o %c`);
  # orphaned run file, delete
  if ($proc eq "") { deleterun; }
  else  {
    # another instance running
    my $timediff = time - $time;
    if (0 == $timeout or $timediff <= $timeout) { die "$0 ERROR: another instance of $myname ($proc) running"; }  
    print STDERR "$0 ERROR: another instance of $myname ($proc) has probably hung\n";
    # another process has probably hung
    # grab its children
    my @PIDS = (`ps h --ppid $PID -o %p`, $PID);
    kill ('TERM', @PIDS);
    chomp ($proc = `ps hp $PID -o %c`);
    # couldn't kill it.  Give up.
    if ($proc) { die "$0 ERROR: can't kill instance ($myname) ($proc) PID ($PID)"; }
    # it died OK.  Delete run file if possible.

if (1 < $verbose) { print STDERR "creating run file ($run1) ... \n"; }
open(my $fh, '>', "$run1") or die "$0 ERROR: opening file ($run1) ($!)" ;
print $fh "PID $$\n";
print $fh "time ", time, "\n";
close $fh;

#If something bad happens delete the run file
sub cleanup {
  if (-e $run1) {
    if (1 < $verbose) { print STDERR "Deleting run file...\n"; }
    unlink $run1;

if ($verbose) { print STDERR "Starting ....\n"; }

system ($path, @ARGV);