The Problem of Passkeys

 The Problem of Passkeys

This is not a deep study of passkeys, I am interested in how useful and usable passkeys actually are.  In particular by trying them myself in a limited way.

I have many issues with passkeys.  Asymmetric encryption is hard, technical and not suitable or secure for most normal (ie non-technical) people.  But clearly it's possible to use it to make reasonably secure systems like Signal.

I had a brief look at passkeys a while ago.  When I found that I couldn't use passkeys on MacOS without having my passkeys (my secret keys) copied into the icloud, I decided to give that a miss.  I don't want my secret key on other people's servers.  Apple could have decided to allow passkeys to stay on a device, but no, apparently they wanted to make passkeys easier for people to use.  On all their Apple devices!  

Easy to use is the bane of IT security.

Anyway, since then there have been developments, so I decided to try again.  Apple still allows passkeys on its systems only if you allow it to copy your secret keys to the icloud.  But there are now alternatives.

Apparently Strongbox.app, two versions of which appear to be supplied with my MacOS Tahoe reads and stores password data in a standard Keepass database.  Strongbox appears to be integrated with the MacOS password systems.  Apparently Strongbox can store passkeys but the methods are complicated and oh, require a paid upgrade, that might even work!  Not really interested.  

I found that KeepassXC is yet another Keepass app that reads and writes Keepass format databases.  It also has binaries for MacOS, Windows, and Linux and is free and open source.  It doesn't have an Android or iPhone app yet, I think.  KeepassXC has some interesting new features.  For instance you can use it to generate passphrases.  KeepassXC can store passkeys and it has browser extensions for a number of browsers to autofill passkeys and other stuff. It has browser extensions for Firefox and Brave/Chrome and Edge.   Maybe more in the future? 

It's not exactly simple to get autofill working with KeepassXC.  You have to add the browser extensions to your browsers.  There are settings that need to be set.  I managed to get it to work on Firefox, but not Brave, even though Brave managed to use passkeys without it.

Oh yeah, did I mention: Brave browser can store passkeys internally.  Who knew?

I dislike password wallets that do autofill.  I'm sure that many people find it easy, but frankly easy in computers is the cause of many security vulnerabilities.  I guess you have to use autofill with passkeys.  We can't allow users to copy and paste or drag and drop or generally control their own secret keys.  Also I often need to use different browsers for different sites and to test stuff.  I don't necessarily want all of them to login as the same user or even to login at all.  I definitely don't want one overriding autofill system run by Apple or Microsoft with them deciding where to store my secret keys and personal info.  

After getting passkeys with KeepassXC to work in Firefox, on a test site, I tried it for a real site, ie google.  Somehow it just wouldn't work.  Some part of Firefox or MacOS decided I needed to store my passkey in Apple's icloud and since that was not enabled, no passkey.  The process appears to be controlled by the website and the operating system and is opaque to the user.  I tried another website and it worked.  Mysterious.

One of the problems with passkeys is that most systems appear to take all control away from users.  Websites apparently control how you create your passkeys, how you store them.  If you want to store your passkey in a browser and the browser allows it, you will only be able to use your passkey from that browser.  You may not be able to use other browsers, other devices.  It's hard to see your secret key, hard to move it around or not.  One solution of the problem of moving your passkeys around for yourself to other browsers, other apps, other devices is to put them in a cloud.  Apple's solution and maybe Google's too and possibly Microsoft's.  A cloud is other people's computers.  The big corporates, social media desperately want you to save your secret keys on their cloud, so they can get access to copies of your secret keys. And access to when and how you use your secret keys.  Bonus for them.

It would have been simple for the designers to allow more than one passkey per account.  This would have solved some of those problems.  They could have used just one secret key for all your accounts.  This could have solved some of those problems too.  Having just one secret key is not the same sort of security issue that having the same password is.  I mean, if someone gets your secret key, they can access all your accounts, but if you have all your secret keys on your cloud and they get that, they also have access to all your accounts.

Why can't any of these systems allow users themselves to move their own passkeys between systems?  Of course they might stuff it up.  Of course a compromised system would mean your passkeys would be compromised, but that would be the case if they were stored in a cloud.  

Is it possible for a site to allow log in only with passkeys?  I don't think we are anywhere near to that.  How would we revoke a key then?

I just don't think passkeys are either ready nor has the design been thought through enough for the benefit of what we say in Australia: us mug punters.  Maybe it's almost OK for big companies.  Passkeys are great for a corporate network where the company controls everything.  Is that what we want for us ordinary people?  I think not.

 

A Random Annoyance - magnetic USB connectors

 A few weeks ago I bought a watch, perhaps a smart watch, perhaps a not very smart watch.  Anyway this is not really about the watch.  The watch had an interesting USB charging port.  

The USB "socket" has two plates also surrounded by two magnets.  

 

The USB connector has 2 pins and the pins are surrounded by two magnets.  I found it interesting.  The magnets meant that the connector would only contact and stick on in the right direction.  The socket was completely waterproof.  Cool.  

A couple of weeks later I got a bone conduction headset.  It had the same charging connector.

 

The magnets are buried.

And apparently the same cable and almost the same connector.  

 

Oh, I realised that the magnets were opposite polarity.  Ouch! I carefully marked the cables so as not to get them mixed up.  I checked the polarity of the power pins.  Same electrical polarity but opposite magnetic poles.

It's annoying that there isn't a standard.

Maybe there's a rectifier bridge inside the devices, but these are small devices and I don't want to test this, just in case.

Torch Review: Fenix E30R

I'd like to review my best torch ever.  I've had this torch for several years now.  It is my almost constant companion for working around the house, repairing electrical devices, installing computing equipment, fighting fires at night.  I have no connection with the company other than being a happy customer.  

The Fenix E30R

Small, bright and tough:
This torch is small, not tiny but fits into a pocket easily.  Aluminium makes it light and tough.  It has a maximum brightness of 1600 lumens.  It has only one side button and does lots of things but is fairly easy to get used to.   

Travel lock:
It has a travel feature that works: double clicking the button when the torch is off, puts the torch into and takes it out of travel mode.  In travel mode clicking the button causes the torch to flash twice.  In travel mode you can't burn through your bag or pocket, even if the button is pressed.  

No memory:
One thing that I really like and find essential.  It doesn't have a memory function.  The torch always starts at minimum brightness.   Short presses increase the brightness in 4 steps to maximum and then back to minimum.  Why torch makers insist on having a memory function that turns the torch on to the last brightness level I really don't know!  Some random brightness level I used maybe a day ago or a week ago is unlikely to be right for what I want now.  I have other torches I use sometimes that have this memory function.  One goes to 6000 lumens.  I hardly ever would want it to start on this brightness level.  Yet because of the memory function, every time I use a higher than minimum brightness level I have to cycle through the brightness levels until I get to the minimum before I turn it off.  What a pain.  I can sort of understand why some people might like this function but it would be really good to be able to turn memory off or have a starting brightness level chosen by the user no matter what the previous level was.  I understand that torch makers think it's a great idea to have memory and have worked hard to put it in and don't want to allow users to turn it off.  Grrr.

Black:
Why do so many gadget makers make their gadgets black?  So easy to get stuff lost, in your pocket, under the sofa, in the dark, in your bag with all the other black gadgets, on a burnt fireground.  Make it a bright light colour, how hard is that?

Charger:
The torch has a peculiar and unique magnetic charger.  It doesn't compromise the water proofity (technical term here) of the torch and it works, are the only things that can be said of it.  

Pros: Small, 1600 lumens, no memory

Cons: Black, weird charger, they appear to have discontinued it.  Damn!

That horrible rubbery plastic coating that turns sticky

 I hate that horrible rubbery plastic coating, that they sometimes put on small electronic devices, that turns sticky after a year or three .  If I want to keep using the device, I have to remove the coating which has turned sticky, which is a major undertaking.

To clean the rubbery coating I need to use eucalyptus oil, ti tree oil, citrus oil, alcohol, propyl alcohol or a mixture of one or two of them and some old cloth.  The process is messy and time consuming.

Just stop it. 

Browsing slightly more safely and privately

Companies and governments glean massive amounts of data from users on the web.  This data gets to huge data warehouses where it's matched together and used by many companies and not for your benefit.  Part of the issues are the secrecy and lack of oversight and control by the users themselves.  Often you are paying in multiple ways for them to collect your data.  Remember if it's free on the web, then you are the product, not the customer.  Just because you pay doesn't mean that changes either.

There are a number of things you can do to make web browsing safer and more private and give yourself more control.  I use Firefox because although it has issues, it is flexible and has lots of useful extensions that help make browsing safer.  Just remember that as you get more safety browsing, browsing can become more difficult.  Many sites use scripts from other places to do many things but those scripts often track you and snoop on you.

You can use other browsers but they all have issues.  Chrome is OK but it desperately wants you to sign in to Google and then once you do that, everything you do, every site you go to, every search you make, belongs to Google.  I can't really tell you about Edge as I usually don't use Windows.  I expect Edge browsing is part of the extensive Windows telemetry gathered by Microsoft.

Defaults:

Don't leave browsers on their default settings.  Although they make it hard, you can change important things like the default search engine.  Most browsers are paid in some way by search engines for the privilege to be the default search engine on that browser.  Or like Edge and Chrome they have a default from the company that supplies them.  I should mention here that you should try and avoid Google or Bing for your searches.  For a long time I used duckduckgo.com but I learned a while ago that they use a lot of data from bing although I think they have their own indexing, and some links redirect through bing.   It is still my fallback search engine and rarely I still use Google too.  Learn how to change your default search engine.  I prefer to have a separate navigation bar and search bar and not to allow searching from my navigation bar.  I actually sometimes type out full URLs.  That's all harder on a phone though.  Don't use google search if you're logged into google.  All that data goes straight into your Google history.   Remember you don't always have to use the same search engine.  Sometimes specialised search engines are much more useful, for example wikipedia, youtube, google maps, etc.

My favourite search engine at the moment is Searx Randomizer.  This sends your search to a random searx instance. Searx is an open source meta-search engine.  Official Searx website Searx Wikipedia Entry  Your search is relayed to many other search engines but without information about you, the searcher being relayed.  It often gives me unusual and interesting search results and useful, did I mention useful?  Google tends to show you results it thinks you want or it thinks you should have.  Also note that searx instances are quite fluid, coming and going occasionally.  In part, because the big search engines don't appear to like non-humans doing searches.  Google, who has robots trawling every web page on the planet, apparently doesn't like anyone doing Google searches without it knowing who is doing it.  Bing ditto but they have taken to using special Bing links that redirect through Bing.  Because of this, sometimes I'll have to repeat a searx search because it fails the first time.  Slow down. 

Fingerprinting.  Fingerprinting is a way tracking companies have of identifying you using features of your browser.  Things like languages, fonts, page size, operating system, IP address, colours of pixels on your monitor.  Why do browsers even give out this information?  Some of the extensions that follow make fingerprinting much harder.

Firefox extensions

Firefox has a number of useful extensions for safety and privacy.  

Adblockers: 

Apart from getting rid of objectionable and sometimes unsafe and even malware filled ads, these help lower your bandwidth.  There is a huge war between advertisers and adblockers that has been going on for a while.  

I use uBlock Origin. uBlock Origin website I'm sure there are many other adblockers that work. 

No Mining:

No Coin Stop scripts that use your browser to mine bitcoin for someone else. Sigh. 

Fingerprint Blocker:

CanvasBlocker  This add-on allows users to prevent websites from using some Javascript APIs to fingerprint them. Users can choose to block the APIs entirely on some or all websites (which may break some websites) or fake its fingerprinting-friendly readout API.

Tracker Blockers:

I tend to use a few of these.  There is just so much tracking at the moment. 

Privacy Badger by The Electronic Frontier Foundation.

Ghostery is a tracker blocker.  It is a commercial company but I've found it's quite good and easy to use.

Duckduckgo Privacy Essentials  A whole bunch of privacy features.  Also blocks fingerprinting.

Decentralize Protects you against tracking through "free", centralized, content delivery. It prevents a lot of requests from reaching networks like Google Hosted Libraries, and serves local files to keep sites from breaking. Complements regular content blockers.

AdNauseum not only blocks ads, it obfuscates browsing data to resist tracking by the online ad industry. To throw ad networks off your trail AdNauseam “clicks” blocked and hidden ads, polluting your data profile and injecting noise into the economic system that drives online surveillance. Just a bit of fun.

TrackMeNot An artware browser add-on to protect privacy in web-search. By issuing randomized queries to common search-engines, TrackMeNot obfuscates your search profile and registers your discontent with surreptitious tracking.  Just a bit of fun too.

Cookie Autodelete When a tab closes, any cookies not being used are automatically deleted. Keep the ones you trust (forever/until restart) while deleting the rest. Containers Supported. 

Facebook Container Prevent Facebook from tracking you around the web. The Facebook Container extension for Firefox helps you take control and isolate your web activity from Facebook.  Facebook is a surveillance company, it's not what you thought.

F.B Purity While you are logged onto Facebook, this lets you hide all the Facebook Ads, Suggested Posts / Related Posts / Sponsored Posts / Sponsored Posts / Upcoming Events / Games your Friends are playing / Games You May Like / Similar To / Related Articles / More Like / More From etc, etc.

Cross-site script blocking:

uMatrix Prevents cross-site scripting.  Warning: This extension can make it harder to use webpages.  You need to understand how to use it and you often have to enable scripts to get the page to work.  If you're prepared to deal with the hassle, it's very good.   Unfortunately even IT professionals find this one difficult to use, other alternative suffer from the same issues.

Smart Referer Every time you click on a link, your browser helpfully tells the website the link takes you to, what web page you came from.  This extension stops that. In tech speak: Automatically hide HTTP Referer and JavaScript document.referrer for cross-domain requests!

Redirector Some pages have links that redirect through their own site so they see what you click on.  Google and Bing both do this.  This extension might help with that, but it requires a bit of work.  May not be worth it.

Making pages more readable:

Remove/Crop to Selection Sometimes you may want to print or save only a part of a web page. With this add-on you can select a part of a web page (text, images, etc), right click on the selection.  Remove parts of a webpage (it's not permanent, just reload the page).  Remove annoying animations etc.

Kill Sticky Remove fixed headers or buttons that obscure or limit content on a web page.  Again, non-permanent but can be very useful.  Based on Alisdair McDermid's Kill Sticky.  This is a javascript bookmarklet not an extension as such.

Lots of tabs users:

Tab Session Manager Save all your tabs and restore them.

Tree Style Tab This extension provides the ability to work with tabs as "trees".  What can I say, I usually have a lot of tabs open.

Update (2025-04-12):

Since I wrote this post there are a couple of updates.  Apple has disallowed extensions on browsers other than Safari and Google has stopped adblocking extensions on Chrome.

iphones and ipads:

Unfortunately on IOS, Apple has forbidden browsers other than its own browser Safari to have extensions.  On iphones and ipads you can install Adguard extension for Safari.

Chrome:

Google has stopped extensions being able to block ads, use Brave instead.  Brave is based on Chrome and has an adblocker built in.  It works on IOS, MacOS, Windows and Linux. 

Review of Neal Stephenson's "Fall, or Dodge in Hell" (spoilers)

Neal Stephenson - Fall, or Dodge in Hell
Warning Spoilers

I really love some Neal Stephenson books, but some I find unreadable.  I think his book "The Diamond Age" one of the best science fiction books I have ever read.  I loved Anathem, although I had issues with the ending.  His endings are often strange.  I can't actually read his "Baroque Cycle" trilogy - only ever got a couple of pages into it.  I find many of his books just OK, Snow Crash - OK; Cobweb - a good potboiler, Cryptonomicon - ditto; Reamde - meh, OK if you like 'Murrican Ayn Randian libertarian cyber westerns; Seveneves - couldn't read, got about a third of the way through.

I liked Fall but some things just gave me the screaming irrits.

Warning Spoilers ahead:

The book is a continuous story but I'll divide it into 4 different sections, where there are different main characters, and scenarios. 

Fall has like Stephenson's other books, a lot of technical detail.  Interesting if you like that sort of thing, and I usually do.  IT people are often good at technical detail but not so good with biology (non-technical biology), sociology, or economics.  For instance, there are no discussions about different ways of making societies.  Everyone has two options.  An Ayn Randian libertarian feudal structure with Dodge as top alpha male superman and an Ayn Randian libertarian feudal structure with El as top alpha male superman. 

Section 1

In the first section Dodge is still alive and then dies and has his head frozen.  This section ends around when his brain is destructively scanned and uploaded into a computer.

Section 2

Dodge's brain in the computer creates a virtual world.

My first real gripe with the story is that the second section, really doesn't make much sense, here's why:  Dodge's brain has been scanned and uploaded into a computer.  Just his brain.   Actually, just his neurone connecting structure.  There appears to be no communication between the uploaded brain and all the living people around it in the real world.  Putting a living person into complete sensory deprivation for even a short length of time will make them quite insane and surely that's what they did. He is just a brain with no body, no senses and they made him alive in the simulation that way.  Instead of going mad, Dodge, after some time, creates a virtual world in the computer.  A kind of multiplayer online role-playing game all by himself with not even senses, just his brain.  But still no communication with the outside.  They can sort of vaguely see the virtual world that he's created but not communicate with Dodge.  I know it's a story artefact done to divide the current world and the virtual world to keep the story going but it just doesn't really make much sense. All that technology and they can't even make vibrations in his ears or listen to vibrations in the air of the world?  How do the virtual inhabitants communicate with each other?

Section 3

Then more and more people are uploaded into the computer network that grows over time.  No-one apparently thinks of having more than one virtual world.  Virtual inhabitants don't appear to remember their "previous" lives. Dodge's world grows much more complex and Dodge becomes the creator and alpha male king or god of his world with a pantheon of uploaded characters who have special powers and many ordinary uploaded people.  Dodge's world has a very simple societal structure - a feudal hierarchy with an alpha male at the top.  One of his pantheon, a woman, manages to create self-reproducing life and eventually manages to create native self-reproducing humans.  (It had to be a woman to do this?  Like women are better at biology?)  Dodge's nemesis El doesn't like Dodge's world, in part because he has to pay for the computer resources to maintain the self-reproducing native life, but rather than create his own world, he makes plans to take over Dodge's world.  El, dies and gets uploaded with lots of people who have paid him to be uploaded, some of whom become powerful "angels".  El and his angels take over Dodge's world and kick him out, Dodge ends up a prisoner in a distant corner of his world (the "Fall").

Uploaded people appear to be able to die in the virtual world but come back, at least some times, a sort of virtual reincarnation.

Section 4

A complicated quest among friends of Dodge in the El dominated world eventually allows Dodge to come back and fight El and somehow kill him.  Will El reincarnate?  Not known.  So it comes down to a giant shoot out at high noon between the alpha males, which Dodge wins.  Just a change to the alpha-male at the top.  The feudal society with all its faults goes on.

The basis of randomness

I was thinking today about the basis of randomness.



I'm still thinking about how to write this down so bear with me here.

After the invention of calculus, mathematicians saw the world as a sort of giant clockwork machine.  "Give me the initial conditions and the law of motion, and with calculus I can predict the future -- or better yet, reconstruct the past."  Einstein's spacetime implies the same thing, that all of time and space are fixed, that we can move forward or back, given the mathematical ability.  There is no randomness in this view of the universe.

And so we thought until the discovery of quantum theory.  The problem being that the world is not infinitely divisible.  Eventually everything must collapse into particles.  This collapse introduces a rounding or truncation error, that is the basis of randomness.  The universe is really digital not analog.  The present is the moment that the continuous changes into steps, quanta.  It's the present that changes infinitely divisible potential into particular actuality.  The collapse of the present is the reason we can't go forward or backward - forward - we can't know the future before it arrives, backward - we can't tell exactly what happened from what we know about the present.

Update: I have been thinking about this and I think the point of collapse is simply the present.  I am not sure what the present is exactly, when considered in the light of Einstein's general relativity, but then as I said before, spacetime and quantum collapse don't seem compatible anyway.  The idea that the point of collapse is the present does not require an observer or multiple universes, although I do wonder if the collapse releases energy and/or information.

Trampolines from oztrampolines online are dangerous. Don't buy them.

Trampolines from oztrampolines online are dangerous. Don't buy them.

Here is my experience.

In March of 2014 bought a 7 x 10ft Rectangular Trampoline from Oztrampolines online.  It was delivered fairly promptly but when I got it here is a picture of what I got. 

Actually a 6ft x 9 ft trampoline with an extra rail 1 foot around the outside.  But this is just disappointing not dangerous.  The legs are complicated and not especially stable, but again not dangerous, just annoying.


From about 18 months after I got the trampoline, the springs started breaking.  They would snap and would fly off and hit the person on the trampoline causing at least bruising.  Thank goodness they haven't hit anyone in the head or eye yet.  This continued on and off since.

5 or 6 have broken this way.  I complained to oztrampolines online and they said that the trampoline had a five year warranty but the springs had only a two year warranty and they would sell me a new set of springs cheaply.  Presumably with a two year warranty.   The complaints went on for a while and they would say: did you install the springs the right way round?  Yes I had.  Did you have the guard pads installed?  Yes I did. 

Everyone seems to believe oztrampolines when they say that the spring guard pads should stop this happening but the physics is quite clear, the pads stop someone landing on the springs and hurting themselves, but they don't stop the springs flying off and hitting the person on the trampoline. 

I went to my government consumer group to dispute the fact that the trampoline I bought was not fit for purpose, but they wouldn't take it any further.  I tried the checkout but they weren't interested. 

A friend had an idea to put cable ties on the springs to stop them flying off.  We did that but they don't work as it has just happened again.

I am now throwing the trampoline out as I believe it is unsafe to use as a trampoline.

1. Who sells dodgy springs in trampolines that break?  I have owned trampolines that are decades old that no spring has ever broken.

2. These springs all break at a similar point, the frame side hook.

The physics:  When you are at the bottom of a trampoline jump, your feet or what ever part of you is contacting the mat are well below the level of the frame.  The mat applies something like 7G* of acceleration to you at the lowest point.  That is the time the springs have the most force on them, and are most likely to break.  It is also the time when the springs are below and not touching the guard pads.  If the springs break at the frame side or anywhere in the middle, the part that is on the mat side will be propelled in the direction of the force, ie towards the person on the mat.

http://iopscience.iop.org/article/10.1088/0031-9120/50/1/64/meta

You can clearly see in this picture how the guard mat does not cover the springs at the bottom of a jump.

*http://iopscience.iop.org/article/10.1088/0031-9120/50/1/64/meta

Here is a sketch of the forces at the bottom of a jump:

And the forces on a spring:

You can see that the guard pad does not cover the spring any more. If the spring breaks before the person reaches the bottom of a jump the force that caused the spring to break will launch the spring above the mat.  In the lower picture you can also see that the hook on the mat when it hits the mat side of the triangular clip will be launched upwards.

Bots, Metadata, big data, linkity and that time I broke the family googles.

I guess this started a long time ago when companies started collecting data from users on the internet.  What triggered me to write linkity was our government deciding to make ISPs collect and store our metadata.

I wrote linkity to create lots of metadata.  It looks at lots of things at random on the internet.  I wanted a program that didn't download much and so cost the user much, and didn't cause much traffic on any remote sites but went to lots of random websites and generated lots of metadata.  I believe it's completely legal to use this and hope it stays that way.

If our government wants to store our metadata, this program helps give them more.  It increases the signal to noise ratio on your metadata.  It's a bit of security by obscurity and probably an annoyance only.  Don't rely on it to do anything much.  It may give you a bit of plausible deniability.  If many people use it, it will create a massive haystack to look for needles in, so I want the program to scale without being too annoying on the internet.

I thought it would be relatively easy at first.  I would run a few searches for random words on google and scrape follow the links. 

That seemed to go well, but while I was just testing and tuning it, google objected and banned my house from searching for using a bot.  My family were not impressed.  I hadn't noticed because I don't use google that much.  They told me "fix the googles!"  Who'd've thought that google didn't like people using bots on google.  Google, who run more and more powerful bots than anyone else in the world or at least the public world.

I decided to stop and rethink my plan.  The googles came back to our house after a day or so.  Family happy again.

I guess google uses its massive machine learning to try and improve its responses to people's questions and runs a lot of checks to make sure it's really people.  I hadn't realised or hadn't thought about the massive numbers of robots and robotic activity out there in the wild internet doing all sorts of things, probably related to making money in good or bad ways. 

I had planned to make scrapers for other search engines, so I bought those plans forward.  I changed the program so it collected the links in a database instead of just viewing them and throwing them away.  Then I found I could harvest links from any page I visited, so I didn't have to do go to search sites very much. 

I did manage to break the googles again but this time their AI only seemed to have blocked my script briefly and not the rest of the household.  Clever googles.

Now I have it working, I can watch how my script wanders around the net. Sometimes it fixates on certain sites.  Sites that have a lot of subdomains.  It's fascinating how websites are networked through links.

Bug reports, ideas and comments welcome.

Oh and lynx developers, why have a non removable error message when you have a non-lynx useragent?  Why are we forced to hand over information about ourselves when we use the web? 

Technical details:
Download: linkity

It runs on linux systems, BSD or Mac.  It uses some unix utilities: lynx, sqlite3, perl and with perl it uses the URI::Escape module. 

On debian/ubuntu systems you can install them like this:
sudo apt-get install sqlite3 lynx liburi-perl

On a mac with MacPorts:  On a Mac with MacPorts:
sudo port install lynx sqlite3
I'm pretty sure MacPorts installs perl5 automatically and URI::Escape is:
sudo port install p5.24-uri

You could use CPAN, to get URI::Escape but I can't really help you with that.

randword: Generating memorable random passwords

This started when I decided to learn python by rewriting one of my old perl scripts in python.  randstring is a script to generate a random string of characters.  I use it sometimes to generate passwords, but password strings or random characters usually can't be remembered, at least not easily.  Passwords like that can be useful at times.  You need to store them in an encrypted password safe.

I have another script that generates more memorable passwords.  Some people I know, have found it useful.  There are always some passwords you need to be memorable.  For instance your login password and the password to your password safe.  randword generates a bunch of words from a dictionary.  XKCD style passwords, if you like.  In the process of examining it, I rewrote it in both perl and python, fixed some bugs and added some features.

In general a bunch of words can be much easier to remember and can be just as difficult or far more difficult  to crack.  I like to generate a bunch and choose a few at random.  4 or 5 or more words is OK.  Hint: misspellings are good but not if you can't remember what you did.  Passwords on websites are a bit mad at the moment with complicated rules, like: "there's an illegal character" or "you must have an upper-case letter and a number", or "that password is too short", or "too long" etc. 

New features of randword:

• There's a couple of new options about output format, like camel case.   
• randword can use any dictionary or word frequency lists as long as they have a fairly simple format - ie at least a word and an optional number at the start of each line. 
• randword can also take a bunch of text and create dictionaries of words that it can use to generate random passwords.  

For word lists, I have used various texts, for instance Jane Austen's complete works, Shakespeare, Mark Twain, Chaucer.  There are many works that can be easily got from Project Gutenburg among other places on the net.  Also word lists and text that can be found at COCA or Lancaster University  etc.Since I only want ascii because I can't type non-ascii characters easily, I used unidecode (python or original perl version) to turn them into ascii.  Python unidecode comes with a command line script.  I wrote a very simple perl script to detect non-ascii characters, (not included) although working out what encoding a page is in is a kind of major headache and you need to know the encoding before unidecode will work, grrrr. 

The links below include word lists from Chaucer, Shakespeare, Mark Twain, and the linux word dictionary.

This is my original blog post on the scripts with all the links to the scripts and associated stuff.

Links:
randstring.pl randstring.py
randword.pl randword.py
some word lists
tarred and zipped archive of scripts and wordlists

In memory of my mother, Marjorie Pizer Holburn, 3rd April 1920 - 4th January 2016

Marjorie Pizer died on the 4th of January 2016. She is sadly missed.

It's impossible in a short time and with my limited time and writing skills to summarise her.  Here is my brief message.  In time I will perhaps add links to other people writing about her.  If you knew her please feel free to add a comment.

My mother was a poet, artist, psychotherapist and a deep thinker. She had many unconventional beliefs, which she held passionately. Politically, she almost always sided with the underdog; and she was troubled by the inequities in our world. She was inspired by the beauty of nature, and loved music, poetry and art of all kinds. She read 2 or 3 books a day, every day, on almost every subject imaginable, until near the end of her life. My mother was a member of all the local libraries; municipal, city and state. Brought up in Melbourne, she lived much of her life in Sydney where she went to the beach and swam nearly every day for forty years of her life.

Marje was one of my best friends for my whole life. I was always much more a scientist and she a poet and writer, our discussions were always wide ranging and broadened both our view points. She was and is a reference point for me for ideas and values. Although in later life I agreed with her ideas less, we always had lively and interesting discussions.

When I think of my mother I picture her surrounded by books and works of art and odd found objects from the beach. Pieces of driftwood, and a vase of fresh cut flowers on the table. We would eat a simple meal together, have a cup of tea and discuss the state of the world. This is how I will remember her.


Marjorie's books can be bought at lulu and apple and amazon.  They are published by Pinchgut Press (me). 

Daniela Torsh's obituary from the Sydney Morning Herald: Poetry Marjorie Pizer's vehicle of optimism and understanding. And her speech.

A memory

When my son comes home late,
He sits on my bed
And tells me about his day.
Someday he will remember this
When I will be no more,
When I have had my say
And gone before.
Then I will not exist
As I am now.
This me will be a memory
Of his when I,
Who now am here alone,
Have gone into oblivion.

Conception, the twin problems and souls.

People used to believe that women were like a field and men plowed them and planted a seed and those seeds grew into babies.  There are echoes of this belief in the current right to life movement when they say "life begins at the moment of conception."  Apart from the obvious: "so sperms and eggs aren't alive?"  What does this really mean?  That at the "moment" of conception a fertilised egg suddenly gets a soul?  How long does conception actually take?  What part of that process is the "moment"?  Of course it begs the question, does a fertilised egg/fetus get a "soul" suddenly or gradually.  Does that question even have any meaning?  Why should having a soul be an on/off binary thing. 

Then there are the twin problems. 

The first is that identical twins both originate from the same fertilised egg.  Does this mean they share the same soul?  That they only have half a soul each?  That their soul divides into two?  Can souls do that?  That one of them gets another soul when the fetus divides into two?  Which one gets the new one and which the old one? 

The second twin problem is that sometimes with fraternal twins, one of the twins is absorbed by the other.  The "surviving" twin can become a chimera with both sets of genes.  Does this mean they have two souls?

 

Allow only one program at a time to access a resource in linux

I had several programs I needed to run but I wanted to make sure only one program got access to a resource at the same time.  So I wrote a short and simple perl script that would do this.  This script needs a directory in /var/run that it can write to.  Since the script doesn't run as root, I have to create this directory after each reboot.

The script is invoked like this:

1nstance.pl -d /var/run/once -n com1 -- my-program my-args

It will run my-program if there is no other script accessing the named resource com1.  If there is another script currently running and using com1 then it will just quit.

The script can be downloaded from here.

--------------------------1nstance.pl-------------------

#!/usr/bin/perl -w
#
#(c) copyright 2015 Kim Holburn
# Licensed under GPLv3

use strict;
use Getopt::Long;

my $verbose = 0;
my $help = 0;
my $myname = "";
my $mydir = "/var/run/once";
my $PID = 0;
my $time = 0;
my $timeout = 0;

Getopt::Long::Configure('require_order');
GetOptions ('verbose+' => \$verbose,
            'directory=s' => \$mydir,
            'name=s' => \$myname,
            'timeout=i' => \$timeout,
            'help|?' => \$help);

if ($help) {
  print <<EOM;
one instance - make sure a program only runs one instance.
usage $0:
$0 [options] -- <command> [ARGS]
  options:
    -h|-?|--help - print this screen
    -d|--directory <directory> 
       directory to store run files. 
       Default is /var/run/once
    -t|--timeout <n> 
       time in seconds that a program is allowed to run 
       before being considered hung and will be killed
       -t=0 means no timeout. The running process will 
       be left untouched.
    -v|--verbose - print extra messages
    -n|--name
       name of process or resource to be run once
       The default is the name of the program.    

EOM

  exit;
}

if ($timeout < 0) { die "$0 ERROR: specified timeout less than zero ($timeout)"; }
if (10000 < $timeout) { die "$0 ERROR: ($timeout) too large"; }
if ($mydir !~ m{^/}) { die "$0 ERROR: directory ($mydir) is not an absolute path!"; }
if (! -d $mydir) { die "$0 ERROR: directory ($mydir) does not exist!"; }

# first argument is command to run
my $path = shift;
if (!$path) { die "$0 ERROR: No command "; } 
my $program = $path;
# command must be a full absolute path.  
my $dir = ".";
my $args="";
if ($program =~ m#/#) {
  $program =~ s#^.*/##;
  $dir =~ s#/[^/]*$##;
}
if (!$program) { die "$0 ERROR: program name invalid.  Must be a filename"; } 
if (! -e $path) { die "$0 ERROR: program ($path) does not exist"; }
if (-d $path) { die "$0 ERROR: program ($path) is a directory"; }
if (! -x $path) { die "$0 ERROR: program ($path) not executable"; }
if (!$myname) { $myname=$program; }

my $run1 = "$mydir/$myname";

sub deleterun {
  if ( -e $run1 ) {
    unlink $run1;
    if ( -e $run1 ) { die "$0 ERROR: cannot delete file ($run1)"; }
  }
}

if ($verbose) {
  print "debug timeout=($timeout) dir=($mydir) name=($myname) \n";
  print "debug v=($verbose) 1=($run1) prog=($program) args=(";
  print join (")(",@ARGV);
  print ") \n";
}

if (open(my $fh, '<', "$run1")) {
  while (<$fh>) {
    chomp ;
    if (/^\s*$/) { next; }
    if (/^PID (\d+)$/i) { $PID = $1; }
    elsif (/^TIME (\d+)$/i) { $time = $1; }
  }
  close $fh;
  # no PID or time
  # this shouldn't happen and probably means we have the wrong file
  if (!$PID or !$time) { die "$0 ERROR: run file ($run1) has no PID or time"; }
  chomp (my $proc = `ps hp $PID -o %c`);
  # orphaned run file, delete
  if ($proc eq "") { deleterun; }
  else  {
    # another instance running
    my $timediff = time - $time;
    if (0 == $timeout or $timediff <= $timeout) { die "$0 ERROR: another instance of $myname ($proc) running"; }  
    print STDERR "$0 ERROR: another instance of $myname ($proc) has probably hung\n";
    # another process has probably hung
    # grab its children
    my @PIDS = (`ps h --ppid $PID -o %p`, $PID);
    kill ('TERM', @PIDS);
    chomp ($proc = `ps hp $PID -o %c`);
    # couldn't kill it.  Give up.
    if ($proc) { die "$0 ERROR: can't kill instance ($myname) ($proc) PID ($PID)"; }
    # it died OK.  Delete run file if possible.
    deleterun;
  }
}

if (1 < $verbose) { print STDERR "creating run file ($run1) ... \n"; }
open(my $fh, '>', "$run1") or die "$0 ERROR: opening file ($run1) ($!)" ;
print $fh "PID $$\n";
print $fh "time ", time, "\n";
close $fh;

#If something bad happens delete the run file
sub cleanup {
  if (-e $run1) {
    if (1 < $verbose) { print STDERR "Deleting run file...\n"; }
    unlink $run1;
  }
  exit;
};
$SIG{'INT'}=\&cleanup;
$SIG{'TERM'}=\&cleanup;
$SIG{'QUIT'}=\&cleanup;

if ($verbose) { print STDERR "Starting ....\n"; }

system ($path, @ARGV);

cleanup;

Detecting Rogue DHCP servers 2

So my script detecting rogue DHCP servers worked and worked well but having two or three DHCP servers covering the same scope is somewhat problematical.  So we eventually gave in and changed configuration to having a main DHCP server and failover servers.  I had to change the logic slightly to get the script to cover that.  The new version has a mode where it will only alert if it detects a rogue DHCP server it has not been told about, or gets no response from any of the servers in its valid server list.  One or more valid servers and all is right with the world.

As before, this script works in nagios.  I run nagios on ubuntu.  The scripts in the nagios package reside in /usr/lib/nagios/plugins.   Maybe there's a good place to put your own scripts but I put mine in there too (/usr/lib/nagios/plugins/check_rogue_dhcp.pl).

This script uses the nagios builtin DHCP checker: /usr/lib/nagios/plugins/check_dhcp.

Then you need a plugin command config file.  I edited the dhcp.cfg command file (/etc/nagios/plugins/) and added these lines:

# 'check_rogue_dhcp' command definition
define command{
   command_name check_rogue_dhcp
   command_line /usr/lib/nagios/plugins/check_rogue_dhcp.pl -f '$ARG1$' '$ARG2$' '$ARG3$'
}

Then to actually invoke the check you need to define a nagios object where you give the command the IP addresses of the servers (12.34.12.34 etc).  That may need a hostgroup or some other object depending on how you have nagios set up

#check that no rogue dhcp services are running
define service {
   service_description rogue-dhcp
   check_command check_rogue_dhcp!12.34.12.34!12.34.12.45
   use generic-service
   notification_interval 0 ; set > 0 if you want to be renotified
}

There's various ways to do this and I'm not great at strategic configuration of nagios, so I'll leave that to you.

The script can be downloaded from here.

The script:

------------check_rogue_dhcp.pl-------------------

#!/usr/bin/perl -w
# nagios: -epn
# the above line makes nagios run the script as a separately.
# rather than as part of nagios.
use POSIX;
use lib "/usr/lib/nagios/plugins";
use utils qw(%ERRORS);

sub fail_usage {
  if (scalar @_) {
    print "$0: error: \n";
    map { print "   $_\n"; } @_;
  }
  print "$0: Usage: \n";
  print "$0 [<options>] <server> [<server> [<server>]] \n";
  print "$0 [<options>] [-s <server> [-s <server> [-s <server>]]] \n";
  print "    options:  \n";
  print "      [-v [-v [-v]]] (verbose) \n";
  print "      [-t  <secs>] (wait this number of seconds)   \n";
  print "      [-f] (fuzzy - ok if one or more of the designated servers answer)   \n";
  print "      [-F] (force (default) - all the designated servers must answer)   \n";
  print " \n";
  exit 3 ;
}

my $verbose = 0;
my %servers=();
my $opt = "-t 5";
my $time = 5;
my $force=1;

## for some reason I can't test for empty ARGs in the while loop
@ARGV = grep {!/^\s*$/} @ARGV;

# examine commandline args
while ($ARGV=$ARGV[0]) {
  my $myarg = $ARGV;
  if ($ARGV eq '-s') {
    shift @ARGV;
    if (!($ARGV = $ARGV[0])) { fail_usage ("$myarg needs an argument"); }
    if ($ARGV =~ /^-/) { fail_usage ("$myarg must be followed by an argument"); }
    if (!defined($servers{$ARGV})) { $servers{$ARGV}=1; }
  }
  elsif ($ARGV eq '-t') {
    shift @ARGV;
    if (!($ARGV = $ARGV[0])) { fail_usage ("$myarg needs an argument"); }
    if ($ARGV =~ /^-/) { fail_usage ("$myarg must be followed by an argument"); }
    if ($ARGV !~ /^(\d+)$/) { fail_usage ("$myarg must be followed by an number"); }
    $time = $1;
    $opt = "-t $time";
  }
  elsif ($ARGV eq '-f' ) { $force=0; }
  elsif ($ARGV eq '-F' ) { $force=1; }
  elsif ($ARGV eq '-h' or $ARGV eq '--help' ) { fail_usage ; }
  elsif ($ARGV =~ /^-/ ) { fail_usage " invalid option ($ARGV)"; }
  elsif ($ARGV =~ /^\d+\.\d+\.\d+\.\d+$/)
    # servers should be ip addresses.  I'm not doing detailed checks for this.
    { if (!defined($servers{$ARGV})) { $servers{$ARGV}=1; } }
  else { last; }
  shift @ARGV;
}

if (scalar @ARGV) { fail_usage "didn't understand arguments: (".join (" ",@ARGV).")"; }  
my $serversn = scalar keys %servers;

if ($verbose > 2) {
  print "verbosity=($verbose)\n";
  print "servers = ($serversn)\n";
  if ($serversn) { for my $i (keys %servers) { print "server ($i)\n"; } }
}

if (!$serversn) { fail_usage "no servers"; }
my $responses=0;
my $responders="";
my @check_dhcp = qx{/usr/lib/nagios/plugins/check_dhcp -v $opt};
foreach my $value (@check_dhcp) {
  if ($value =~ /Added offer from server \@ /i){
    $value =~ m/(\d+\.\d+\.\d+\.\d+)/i;
    my $host = $1;
    # we find a server in our list
    if (defined($servers{$host})) { $responses++; $responders.="$host "; }
    else {
      # we find a rogue DHCP server.  Danger Will Robinson!
      print "SERVICE STATUS:CRITICAL: Rogue DHCP service running on $host";
      exit $ERRORS{'CRITICAL'}
    }
  }
}
if ($responses == $serversn) {
  # we saw all the servers in our list.  All is good.
  print "SERVICE STATUS:OK: $responses of $serversn Expected Responses to DHCP Broadcast";
  exit $ERRORS{'OK'};
}

if ($responses == 0) {
  # we found no DHCP responses.
  print "SERVICE STATUS:CRITICAL: no DHCP service responded";
  exit $ERRORS{'CRITICAL'}
}

# we found less DHCP servers than we should have. Oh Nos!
$responders =~ s/ $//;
if ($force == 1) {
  print "SERVICE STATUS:WARNING: $responses of $serversn Responses to DHCP Broadcast. Only ($responders) responded. ";
  exit $ERRORS{'WARNING'};
}
else {
  print "SERVICE STATUS:OK: $responses of $serversn Responses to DHCP Broadcast. Only ($responders) responded. ";
  exit $ERRORS{'OK'};
}

The myth of competition and the free market.

Mathematically the free market is not a stable environment.  It quickly becomes an oligopoly and from there it's only a step or two to a monopoly.

The point is that whenever and to the extent that there is a free market, companies must become efficient to compete.

But free markets only exist when companies compete, when they agree to compete, when they cooperate to compete.  The bulk of any company's business is done on a cooperative basis and that cooperation underlies everything we do as a society.

Free markets cannot exist if companies cooperate.  This is why some forms of cooperation have to be made illegal in order to create a "free market" but companies in a competitive environment can still cooperate without an agreement, without even communicating with each other. To state that more plainly: there is no "free market" without government intervention.

As soon as a market has reached the oligopoly stage, there are entrenched players and at this point new players find it very difficult to impossible to enter the market.  An oligopoly company gets far more benefit from cooperating in the market than competing.  Even if there is no overt agreement to cooperate, oligopoly players will benefit more from cooperation than competition.  They will especially cooperate to lock new and potential competitors out of their markets.  They may compete in a desultory way with other oligopoly members but ultimately the oligopoly state benefits them more than competition.

It is the oligopoly companies that are always extolling the virtues of the free market.

The primary enemy, the primary competitor of a company is competition itself.  Without competition companies don't have to be efficient, they can live off the fat of a monopoly or oligarchy.  There is no transparency.  Companies will always fight hardest to not have to compete.   Not having to compete is a massive benefit for them.

Companies in a free market, when it occurs and to the extent it occurs, will compete and will try to end that competition.  Competition will gradually go away for various reasons, because companies go bankrupt, give up in that market,  get bought out or merge and as the major players in a market get bigger, the market becomes an oligopoly and finally a monopoly and competition ceases.  Legislation can lengthen this process, that is can help maintain competition for longer, but cannot really stop it.  As the companies get large enough and rich enough they will lobby governments to change the rules to their benefit or they will go transnational and be out of the reach of governments or apply pressure in other ways.

When companies lobby for changes in law they will always site "the free market" and this is nearly always an indicator of an oligopoly.