randword: Generating memorable random passwords

This started when I decided to learn python by rewriting one of my old perl scripts in python.  randstring is a script to generate a random string of characters.  I use it sometimes to generate passwords, but password strings or random characters usually can't be remembered, at least not easily.  Passwords like that can be useful at times.  You need to store them in an encrypted password safe.

I have another script that generates more memorable passwords.  Some people I know, have found it useful.  There are always some passwords you need to be memorable.  For instance your login password and the password to your password safe.  randword generates a bunch of words from a dictionary.  XKCD style passwords, if you like.  In the process of examining it, I rewrote it in both perl and python, fixed some bugs and added some features.

In general a bunch of words can be much easier to remember and can be just as difficult or far more difficult  to crack.  I like to generate a bunch and choose a few at random.  4 or 5 or more words is OK.  Hint: misspellings are good but not if you can't remember what you did.  Passwords on websites are a bit mad at the moment with complicated rules, like: "there's an illegal character" or "you must have an upper-case letter and a number", or "that password is too short", or "too long" etc. 

New features of randword:

• There's a couple of new options about output format, like camel case.   
• randword can use any dictionary or word frequency lists as long as they have a fairly simple format - ie at least a word and an optional number at the start of each line. 
• randword can also take a bunch of text and create dictionaries of words that it can use to generate random passwords.  

For word lists, I have used various texts, for instance Jane Austen's complete works, Shakespeare, Mark Twain, Chaucer.  There are many works that can be easily got from Project Gutenburg among other places on the net.  Also word lists and text that can be found at COCA or Lancaster University  etc.Since I only want ascii because I can't type non-ascii characters easily, I used unidecode (python or original perl version) to turn them into ascii.  Python unidecode comes with a command line script.  I wrote a very simple perl script to detect non-ascii characters, (not included) although working out what encoding a page is in is a kind of major headache and you need to know the encoding before unidecode will work, grrrr. 

The links below include word lists from Chaucer, Shakespeare, Mark Twain, and the linux word dictionary.

This is my original blog post on the scripts with all the links to the scripts and associated stuff.

Links:
randstring.pl randstring.py
randword.pl randword.py
some word lists
tarred and zipped archive of scripts and wordlists

A Million Monkeys: who owns the copyright?

The random sentence program that I wrote about in my last blog entry can output stories.   It's my version of the million monkeys.

As before, it can be downloaded as a simple perl text file from here or in the zip file with the other programs and the source frequency list from here.  

I will add instructions for creating the database soon.   The database is not necessary for using the program, it just makes it faster.

Each story on this page is created once and is unique, the one you are seeing is a story just for you.   Consider these stories copyrighted!   If you copy and use my program, and I encourage you to do so, it is free but I retain copyrights for all stories created by the program!!   All stories will be cc-sa.

This brings up a question for me: who owns the copyright on words generated by a machine?

Edited:  There was a copyright case recently, a photographer vs. wikipedia in which the court decided that a a photographer couldn't claim copyright of a selfie taken by a monkey because it was the monkey's copyright and monkeys can't own copyright.  So I guess the same goes for a machine.

$randsent -t 5 -p 1000 -g

More random passwords, entropy and words

Following on from my previous blog post I have written a new perl program, this time to create kind of sentence like structures.  It is very simple at the moment but I will continue working on it on and off.  As with the previous programs it is written in perl and like randword it uses Adam Kilgarriff's BNC list from here http://www.kilgarriff.co.uk/BNClists/all.al.gz albeit in a slightly different way - it uses the parts of speech encoded therein.

In looking for the structure of English I found among other things some very interesting pages from this site: http://papyr.com/hypertextbooks/grammar/, specifically the pages on the phrases: the noun phrase, the verb phrase, etc.  What I haven't found yet is anyone who has enumerated the probabilities of the structure: what is the probability, for instance, that the subject phrase of a sentence is a simple pronoun? What is the probability that the main adjective in an adjectival phrase is a verb participle? etc.  I'll probably be working on the way the structure is generated but at the moment the recursivity in the structure seems to have gotten away from me slightly.

This new program called "randsent" creates English-like sentences with a kind of rough grammar so the sentences often seem like English although they may not make a lot of sense.  Of course they're not meant to make sense but to be memorable.  I find it's rather fun to run it and enjoy some of the sentences.  Some of the words from Kilgarriff's list are a bit strange at times.

I've read that this might be done with Markov chains but I'm not familiar with how to do that.  Any interesting ideas, suggestions or rewrites accepted, or encouraged.

As before, it can be downloaded as a simple perl text file from here or in the zip file with the other programs and the source frequency list from here.  


Edit: If you have iframes here is an updating example:

$randsent -t 10 -p 1000

Random passwords, entropy and words

The concepts of passwords and entropy have generated a lot of discussion especially since the xkcd comic.  One of the basic results is that a small additional password length can add as much entropy to a password as adding a larger range of characters and could be much easier to remember.  Think of it like this if you like: each character in a password might be one of say 26 characters or if you add numbers and uppercase and punctuation maybe around 100 possibilities for each letter.  But the number of words in just  the English language gives you a much larger number of possibilities.

A long while ago I wrote a perl program for generating random passwords.  It and the new ones I talk about here are available under the GPL for you to download and use at the end of this article if you're interested.  To use them you will need a computer that runs perl.

My perl random password generator generates a password of random letters.  The command has lots of options to change the make-up of characters and the way it chooses them

$ randstring
 y5tr0nshc_mf
$ randstring -t 5
 yx02w_zlem79
 38rn91schrud
 2vnr2ujty1tt
 oijmmhpjokrg
 lr2v2vknlwws

For me, what started the idea for extending my random password generator to add random words was when someone on the link list sent me these first two links below.

My first try was a variation on the original which generates a series of wordlike things.

$ randstring -w -t 5
 rie hiuv disu coayu
 esxyi sc aim kyw
 kionuj lujyc oni aoahy
 fii ausnzg gad puku
 zna vymiq as mam

Then I decided to write a program that chooses random words from a dictionary.

$ randword -t 5
 yeta fit fot brach
 casave tid oleous pram
 lawyer Ro coto testa
 drawly ras Trapa Ao
 cosec crappo ay hi

Lastly I added a word generator that uses a weighted random choice based on a word frequency list so it has a tendency to choose more often used words.

$ randword -w -t 5
 look and april mr
 home not grass larger
 child that charts lucky
 the the very these
 andrew very she the

This command has many options including an interesting option to limit the word selection to more or less popular words.  In this next example all the words have a frequency of more than 100 in the list.  This cuts the word list in this case from 236660 entries to 26310.

$ randword -w -t 5 -p 100
 upon quest jokes to
 loved scale 's awards
 do in soft pounds
 for and it the
 us most your of

I'm not going to make a web page to do this at the moment, partly because the word-based password generators use quite a lot of memory and CPU resources and partly because someone has already done it (see below) but mainly because it's a silly idea to use a password from a website on the internet.  So for your benefit I am offering these scripts as a GPL program for those interested in using them.

I should note that the word frequency list is based on a 1989 one from Adam Kilgarriff's BNC lists from and thanks to him for permission to use it.
http://www.kilgarriff.co.uk/BNClists/all.al.gz

Links:
http://www.archonmagnus.com/articles/security/passwordEntropy.php
https://secure.wikimedia.org/wikipedia/en/wiki/Password_strength
http://www.baekdal.com/tips/password-security-usability
https://www.grc.com/haystack.htm
https://www.xkcd.com/936/

I've just discovered that someone wrote a webpage which generated "xkcd" style passwords:
http://preshing.com/20110811/xkcd-password-generator
http://passphra.se/

Here are links to download my programs:
randstring
randwords
tgz archive (includes randstring, randword and the word frequency list).

They are, of course, works in progress and any suggestions, bug reports, etc will be gratefully accepted. 

The dictionary version looks for a standard linux, BSD or MacOS dictionary at "/usr/shar/dict/words" with one entry (word) per line of probably ASCII text.  If you don't have that, you may have to get a copy of it.  I have not internationalized the programs.