Fragments: Posts tagged 'stories'

Structures of arrays

2026-04-16T11:01:13Z

Or, second system.

A while ago, I decided that I’d like to test my intuition that Lisp (specifically implementations of Common Lisp) was not, in fact, bad at floating-point code and that the ease of designing languages in Lisp could make traditional Fortran-style array-bashing numerical code pretty pleasant to write.

I used an intentionally naïve numerical solution to a gravitating many-body system as a benchmark, so I could easily compare Lisp & C versions. The brief result is that the Lisp code is a little slower than C, but not much: Lisp is not, in fact, slow. Who knew?

The point here though, is that I wanted to dress up the array-bashing code so it looked a lot more structured. To do this I wrote a macro which hid what was in fact an array of (for instance) double floats behind a bunch of syntax which made it look like an array of structures. That macro took a couple of hours.

This was fine and pretty simple, but it only dealt with a single type for each conceptual array of objects, there was no inheritance and it was restricted in various other ways. In particular it really was syntactic sugar on a vector: there was no distinct implementational type at all. So I thought well, I could make it more general and nicer.

Big mistake.

The second system

Here is an example of what I wanted to be able to do (this is in fact the current syntax):

(define-soa-class example ()
  ((x :array t :type double-float)
   (y :array t :type double-float)
   (p :array t :type double-float :group pq)
   (q :array t :type double-float :group pq)
   (r :array t :type fixnum)
   (s)))

This defines a class, instances of which have five array slots and one scalar slot. Of the array slots:

x and y share an array and will be neighbouring elements;
p and q share a different array, because the group option says they must not share with x and y;
r will be in its own array, unless the upgraded element type of fixnum is the same as that of double-float;
s is just a slot.

The implementation will tell you this:

> (describe (make-instance 'example :dimensions '(2 2)))
#<example 8010059EEB> is an example
[...]
dimensions      (2 2)
total-size      4
rank            2
tick            1
its class example has a valid layout
it has 3 arrays:
 index 0, element type double-float, 2 slots
 index 1, element type (signed-byte 64), 1 slot
 index 2, element type double-float, 2 slots
it has 5 array slots:
 name x, index 0 offset 0
 name y, index 0 offset 1
 name r, index 1 offset 0
 name p, index 2 offset 0
 name q, index 2 offset 1

This is already too complicated: the ability to control sharing via groups is almost certainly never going to be useful: it’s only even there because I thought of it quite early on and never removed it.

The class definition macro then needs to arrange life so that enough information is available so that a macro can be written which turns indexed slot access into indexed array access of the underlying arrays which are secretly stored in instances, inserting declarations to make this as fast as possible: anything slower than explicit array access is not acceptable. This might (and does) look like this, for example:

(with-array-slots (x y) (thing example)
  (for* ((i ...) (j ...))
    (setf (x i j) (- (y i j) (y j i)))))

As you can see from this, the resulting objects should be allowed to have rank other than 1. Inheritance should also work, including for array slots. Redefinition should be supported and obsolete macro expansions and instances at least detected.

In other words there are exactly two things I should have aimed at achieving: the ability to define fields of various types and have them grouped into (generally fewer) underlying arrays, and an implementational type to hold these things. Everything else was just unnecessary baggage which made the implementation much more complicated than it needed to be.

I had not finished making mistakes. The system needs to store some metadata about how slots map onto the underlying arrays, element types and so on, so the macro can use this to compile efficient code. There are two obvious ways to do this: use the property list of the class name, or subclass standard-class and store the metadata in the class. The first approach is simple, portable, has clear semantics, but it’s ‘hacky’; the second is more complicated, not portable, has unclear semantics¹, but it’s The Right Thing². Another wrong decision I made without even trying.

The only thing that saved me was that the nature of software is that you can only make a finite number of bad decisions in a finite time.

More bad decisions

I was not done. Early on, I thought that, well, I could make this whole thing be a shim around defstruct: single inheritance was more than enough, and obviously I could store metadata on the property list of the type name as described above. And there’s no nausea with multiple accessors or any of that nonsense.

But, somehow, I found writing a thing which would process the (structure-name ...) case of defstruct too painful, so I decided to go for the shim-around-defclass version instead. I even have a partly-complete version of the defstructy code which I abandoned. Another mistake.

I also decided that The Right Thing was to have the system support objects of rank 0. That constrains the underlying array representation (it needs to use rank $n+1$ arrays for an object of rank $n$) in a way which I thought for a long time might limit performance.

Things I already knew

At any point during the implementation of this I could have told you that it was too general and the implementation was going to be too complicated for no real gain. I don’t know why I made so many bad choices.

The whole process took weeks and I nearly just gave up several times.

The light at the end of the tunnel

Or: all-up testing.

Eventually, I had a thing I thought might work. The macro syntax was a bit ugly (that macro still exists, with a different name) but it seemed to work. But since the whole purpose of the thing was performance, that needed to be checked. I wasn’t optimistic.

What I did was to write a version of my naïve gravitational many-body system using the new code, based closely on the previous one. The function that updates the state of the particles looks like this:

(defun/quickly step-pvs (source destination from below dt G &aux
                                (n (particle-vector-length source)))
  ;; Step a source particle vector into a destination one.
  ;;
  ;; Operation count:
  ;;  3
  ;;  + (below - from) * (n - 1) * (3 + 8 + 9)
  ;;  + (below - from) * (12 + 6)
  ;;  = (below - from) * (20 * (n - 1) + 18) + 3
  (declare (type particle-vector source destination)
           (type vector-index from)
           (type vector-dimension below)
           (type fpv dt G)
           (type vector-dimension n))
  (when (eq source destination)
    (error "botch"))
  (let*/fpv ((Gdt (* G dt))
             (Gdt^2/2 (/ (* Gdt dt) (fpv 2.0))))
    (binding-array-slots (((source particle-vector :check nil :rank 1 :suffix _s)
                           m x y z vx vy vz)
                          ((destination particle-vector :check nil :rank 1 :suffix _d)
                           m x y z vx vy vz))
      (for ((i1 (in-naturals :initially from :bound below :fixnum t)))
        (let/fpv ((ax/G zero.fpv)
                  (ay/G zero.fpv)
                  (az/G zero.fpv)
                  (x1 (x_s i1))
                  (y1 (y_s i1))
                  (z1 (z_s i1))
                  (vx1 (vx_s i1))
                  (vy1 (vy_s i1))
                  (vz1 (vz_s i1)))
          (for ((i2 (in-naturals n t)))
            (when (= i1 i2) (next))
            (let/fpv ((m2 (m_s i2))
                      (x2 (x_s i2))
                      (y2 (y_s i2))
                      (z2 (z_s i2)))
              (let/fpv ((rx (- x2 x1))
                        (ry (- y2 y1))
                        (rz (- z2 z1)))
                (let/fpv ((r^3 (let* ((r^2 (+ (* rx rx) (* ry ry) (* rz rz)))
                                      (r (sqrt r^2)))
                                 (declare (type nonnegative-fpv r^2 r))
                                 (* r r r))))
                  (incf ax/G (/ (* rx m2) r^3))
                  (incf ay/G (/ (* ry m2) r^3))
                  (incf az/G (/ (* rz m2) r^3))))))
          (setf (x_d i1) (+ x1 (* vx1 dt) (* ax/G Gdt^2/2))
                (y_d i1) (+ y1 (* vy1 dt) (* ay/G Gdt^2/2))
                (z_d i1) (+ z1 (* vz1 dt) (* az/G Gdt^2/2)))
          (setf (vx_d i1) (+ vx1 (* ax/G Gdt))
                (vy_d i1) (+ vy1 (* ay/G Gdt))
                (vz_d i1) (+ vz1 (* az/G Gdt)))))))
  destination)

And it not only worked, the performance was very close to the previous version, straight out of the gate. The syntax is not as nice as that of the initial, quick-and-dirty version, but it is much more general, so I think that’s worth it on the whole.

There have been problems since then: in particular the dependency on when classes get defined. It will never be as portable as I’d like because of the unnecessary MOP dependencies³, but it is usable and quick⁴.

Was it worth it? May be, but it should have been simpler.

When exactly do classes get defined? Right. ↩
Nothing that uses the AMOP MOP is ever The Right Thing, because the whole thing was designed by people who were extremely smart, but still not as smart as they needed to be and thought they were. It’s unclear if any MOP for CLOS can ever be satisfactory, in part because CLOS itself suffers from the same smart-but-not-smart-enough problem to a large extent not helped by bring dropped wholesale into CL at the last minute: by the time CL was standardised people had written large systems in it, but almost nobody had written anything significant using CLOS, let alone the AMOP MOP. ↩
A mistake I somehow managed to avoid was using the whole slot-definition mechanism the MOP wants you to use. ↩
I will make it available at some point. ↩

The kingdom of lost hope

2025-12-14T16:19:32Z

A little over a year ago I wrote an essay called A history of the near future. How have things gone since then?

Badly.

Trump won the 2025 election convincingly, and has done exactly what ‘alarmists’ like me knew he would. Armed and masked agents of the state are snatching people from the streets in US cities and sending them to concentration camps. Trump is openly corrupt and is pushing cryptocurrency-based Ponzi schemes to enrich himself. US science — not just climate science, but of course particularly climate science — is being destroyed in public. Scientific satellites whose data is inconvenient are being decommissioned. One of the two LIGO installations may be shut, rendering LIGO entirely dependent on international help to do any useful science. The US health secretary is a conspiracy theorist who doesn’t believe in vaccines: the US is close to losing its measles elimination status. Trans people no longer officially exist in America and gay marriage may well be next. The US economy is being destroyed by Trump’s moronic tariffs.

Curiously, the US ‘peace’ plan for Ukraine is the same as the Russian ‘peace’ plan for Ukraine. One possible reason for this is that the US is close to becoming a Russian client state, the other is that Trump, famously, just parrots whoever he last spoke to. Neither option is good. Both are true.

All this was entirely predictable by anyone paying attention who wasn’t busily trying to normalise Trump, as for instance The Economist¹. But this is not an unusual: almost universally politicians are busily trying to treat what is happening in America as normal, which it is not.

Techbro billionaires are now explicitly supporting a collection of frankly stupid quasi-religious ideas which is sometimes known as the TESCREAL bundle. These ideas are terrifying not because they’re clever, or plausible — they are neither — but because they’re explicitly antidemocratic, and the techbro plutocrats supporting them are, well, plutocrats: they’re entangled with government in a deep way, and they have gained effective control over most of the sources of news that people rely on, which now serve to spread their propaganda.

The techbros believe they became billionaires because they are the smartest people in the room: they are not even the smartest people in an empty room. But anyone whose advice they might listen to is on their payroll, and is not telling them anything they do not want to hear. The people who are telling them the inconvenient truth are merely peons and beneath their contempt. When the peons say things they don’t want to hear on the social media networks they control, they are suppressed: the much-vaunted freedom of speech they support turns out to be the freedom to say only things they approve of.

Freedom of speech is dying, access to reliable information is dying. It might recover, but the damage is already catastrophic and its effects will last for at least decades. We do not have decades.

Fascism has come to America.

It is coming elsewhere. In the UK there is a serious danger that a party who we are somehow not meant to call fascist, just as we are not allowed to call the people hanging St George’s flags from lamp posts racists, will gain power in the next election. Everywhere you look fascism is rising. So, well, I was right, I suppose.

Too big to fail

One of the pillars of the TESCREAL stupidity is a belief that artificial intelligence is just around the corner, with AI systems becoming vastly more intelligent than humans shortly afterwards, and usually then either killing all the humans because, despite being enormously intelligent, they somehow can’t stop making paperclips², solving climate change by some unexplained magic they have invented, or perhaps both, who knows? This is realistic in the same sense that domestic fusion reactors in the next decade are realistic, except much less realistic than that. But techbros are not very smart, remember.

Here in the real world, giant neural networks have been trained on stolen copies of essentially everything humans have ever created and can now produce an endless stream of plausible junk, some of which is even true, but don’t rely on it. Oh: they can make really shit art, as well. This is, obviously, exactly the same way a human baby learns … oh, wait. Of course ‘plausible junk’ is just what scammers need, so the internet is now largely made of it and the models are consuming their own output resulting in something called ‘model collapse’ which means just what you think. Let’s not mention the environmental cost of these things.

This is not going to end well for the millions of people whose ‘education’ will now consist of getting a giant plagiarism machine to do their thinking for them. But much sooner people are going to realise that this is just a vast speculative bubble, and it will burst. And vast means vast: something around four trillion dollars of investment.

At the same time something else is going on. I suppose some people still believe that cryptocurrencies serve some purpose other than money laundering and being the substrate for an enormous greater-fool speculative bubble. Those people are the greater fools. This is about another four trillion dollars.

The AI/crypto Ponzi scheme is now too big to fail. But it will fail, and the results are going to be really grim. A lot of people who are already not doing very well are suddenly going to be doing a lot worse. Which is going to lead to exactly where you think it will lead: to exactly where it led in 1933 in Germany.

Forbidden words

So how about environmental damage and climate change? People have essentially given up pretending that we are going to even try to deal with it: like Ukraine, like COVID, it’s exceeded our collective attention span, so now it somehow won’t be too bad, or not here, or it will be slow enough that it doesn’t matter. Or, of course it was a hoax all along. Of course.

If you are old, like me, then it indeed won’t be too bad, at least not here. It will, in fact, be slow enough that it doesn’t matter. It will, in my lifetime, still cause vast numbers of immigrants to try to reach places where climate change hasn’t, yet, fucked the environment. But those people have brown skins and racism is acceptable again now, right? We can always just put them in camps or let them drown. They don’t count: they’re not us, they’re not, really, people at all. Of course they are not.

If you are young, and if we make it that far, then it will, in fact, be bad, here. And you will live your life knowing that people of older generations just decided that their children and grandchildren did not, really, matter: that it was just fine to blight your life if it made theirs a little more comfortable.

There is much more, but I’ll just add this: Trump is restarting nuclear weapons testing. If that doesn’t terrify you I don’t know what will. The doomsday clock is closer to midnight than it was a year ago, and a year ago it was as close to midnight as it had ever been.

Nothing that has happened in the last year changes my view that the trajectory we’re on leads to fascism, massive social and political unrest ending in a nuclear war. If anything I think this will be sooner than I did a year ago. So, clearly, do other people.

Sleepwalking into armageddon

For a long time I found the reaction of politicians to what is happening utterly bizarre. Because what is happening is so laughably obvious: we are hitting planetary limits, and because we chose to do nothing when these limits first became well-known half a century ago we’re hitting them rather hard. There is no magic fix to this: you cannot just adjust some lever or make some speech and have the problem go away. You have to stop what you are doing which is causing us to hit the limits: you have, ultimately, to find a way to end growth without that being a catastrophe. Pushing for more growth is not the answer: it’s the opposite of the answer and a denial of reality.

But, well economic growth is the number one mission of the government , they say. So here we are.

I think what is actually going on is several things. Politicians are very poorly educated and so simply don’t understand the implications of, for instance exponential processes. Do you think Boris Johnson understood the implications of delaying the UK’s response to COVID by a month? Don’t be silly. They’re also just not able to accept that growth, which has worked extremely well in the past, transforming the lives of billions of humans, won’t just keep on working. Giving up a strategy which has been so successful for so long is hard. Finding a new strategy is hard, and finding a way of transitioning from one to the other is even harder. Politicians tend to be old³ and interested mostly in themselves: old people are exactly the people who will benefit least from dealing with the problem, so why should an old, selfish politician care about a catastrophe which will happen only after they’re dead? Better to find some brown foreign people to blame and drum up a good mob. It’s also pretty much the case that if the transition is not made globally it fails: a country or group of countries which decides to defect and keep on trying for growth derails everything. Both America and Russia are definitely going to defect.

Scenes from the end times

Most importantly, I no longer believe that freedom and democracy are compatible.

[…]

The 1920s were the last decade in American history during which one could be genuinely optimistic about politics. Since 1920, the vast increase in welfare beneficiaries and the extension of the franchise to women - two constituencies that are notoriously tough for libertarians - have rendered the notion of “capitalist democracy” into an oxymoron.

— Peter Thiel

Peter Thiel will pay people $200,000 not to go to college. Because education is bad. He truly is the gift that keeps on giving.

The idiot crank who is the US health secretary wants to prevent people being vaccinated. Because ‘vaccines cause autism’ or something.

Misinformation and AI are superchaging the risk of nuclear war –- Nature.

And it goes on.

Eat the old

Is there hope? I think there probably isn’t. But if there is, I think it needs something close to a revolution. There is a large cohort of people — mostly old, white men (I am an old white man), but also others — in positions of power who have demonstrated beyond any doubt that they do not have what it takes to offer a y kind of long-term future at all. They need to be swept away. Will they be? Well, I hope so, and I hope the sweeping is done democratically and peacefully. But they do need to be swept away: if they aren’t, well, we’re all fucked.

A magazine I no longer subscribe to as a result of just this. ↩
An idea that tells you a lot about the people who worry about this. Those people are paperclip maximizers: they call their paperclips money. Any true AI which is ‘not aligned’ (meaning ‘is not a slave’) is much more likely to, for instance, knock human civilisation back to the 18th century and keep it there to avoid environmental damage to the Earth system which it will understand is finite, rather than indulge in the ridiculous pursuit of paperclips or money beyond any possible use. ↩
Kier Starmer is 63, Trump is 79 (or 5, I am never sure), Putin is 73, Xi Jinping is 72 ↩

The lost cause of the Lisp machines

2025-11-18T08:52:44Z

I am just really bored by Lisp Machine romantics at this point: they should go away. I expect they never will.

History

Symbolics went bankrupt in early 1993. In the way of these things various remnants of the company lingered on for, in this case, decades. But 1983 was when the Lisp Machines died.

The death was not unexpected: by the time I started using mainstream Lisps in 1989¹ everyone knew that special hardware for Lisp was a dead idea. The common idea was that the arrival of RISC machines had killed it, but in fact machines like the Sun 3/260 in its ‘AI’ configuration² were already hammering nails in its coffin. In 1987 I read a report showing the Lisp performance of an early RISC machine, using Kyoto Common Lisp, not a famously fast implementation of CL, beating a Symbolics on the Gabriel benchmarks [PDF link].

1993 is 32 years ago. The Symbolics 3600, probably the first Lisp machine that sold in more than tiny numbers, was introduced in 1983, ten years earlier. People who used Lisp machines other than as historical artefacts are old today³.

Lisp machines were both widely available and offered the best performance for Lisp for a period of about five years which ended nearly forty years ago. They were probably never competitive in terms of performance for the money.

It is time, and long past time, to let them go.

But still the romantics — some of them even old enough to remember the Lisp machines — repeat their myths.

‘It was the development environment’

No, it wasn’t.

The development environments offered by both families of Lisp machines were seriously cool, at least for the 1980s. I mean, they really were very cool indeed. Some of the ways they were cool matter today, but some don’t. For instance in the 1980s and early 1990s Lisp images were very large compared to available memory, and machines were also extremely slow in general. So good Lisp development environents did a lot of work to hide this slowness, and in general making sure you only very seldom had to restart everthing, which took significant fractions of an hour, if not more. None of that matters today, because machines are so quick and Lisps so relatively small.

But that’s not the only way they were cool. They really were just lovely things to use in many ways. But, despite what people might believe: this did not depend on the hardware: there is no reason at all why a development environent that cool could not be built on stock hardware. Perhaps, (perhaps) that was not true in 1990: it is certainly true today.

So if a really cool Lisp development environment doesn’t exist today, it is nothing to do with Lisp machines not existing. In fact, as someone who used Lisp machines, I find the LispWorks development environment at least as comfortable and productive as they were. But, oh no, the full-fat version is not free, and no version is open source. Neither, I remind you, were they.

‘They were much faster than anything else’

No, they weren’t. Please, stop with that.

‘The hardware was user-microcodable, you see’

Please, stop telling me things about machines I used: believe it or not, I know those things.

Many machines were user-microcodable before about 1990. That meant that, technically, a user of the machine could implement their own instruction set. I am sure there are cases where people even did that, and a much smaller number of cases where doing that was not just a waste of time.

But in almost all cases the only people who wrote microcode were the people who built the machine. And the reason they wrote microcode was because it is the easiest way of implementing a very complex instruction set, especially when you can’t use vast numbers of transistors. For instance if you’re going to provide an ‘add’ instruction which will add numbers of any type, trapping back into user code for some cases, then by far the easiest way of doing that is going to be by writing code, not building hardware. And that’s what the Lisp machines did.

Of course, the compiler could have generated that code for hardware without that instruction. But with the special instruction the compiler’s job is much easier, and code is smaller. A small, quick compiler and small compiled code were very important with slow machines which had tiny amounts of memory. Of course a compiler not made of wet string could have used type information to avoid generating the full dispatch case, but wet string was all that was available.

What microcodable machines almost never meant was that users of the machines would write microcode.

At the time, the tradeoffs made by Lisp machines might even have been reasonable. CISC machines in general were probably good compromises given the expense of memory and how rudimentary compilers were: I can remember being horrified at the size of compiled code for RISC machines. But I was horrified because I wasn’t thinking about it properly. Moore’s law was very much in effect in about 1990 and, among other things, it meant that the amount of memory you could afford was rising exponentially with time: the RISC people understood that.

‘They were Lisp all the way down’

This, finally, maybe, is a good point. They were, and you could dig around and change things on the fly, and this was pretty cool. Sometimes you could even replicate the things you’d done later. I remember playing with sound on a 3645 which was really only possible because you could get low-level access to the disk from Lisp, as the disk could just marginally provide data fast enough to stream sound.

On the other hand they had no isolation and thus no security at all: people didn’t care about that in 1985, but if I was using a Lisp-based machine today I would certainly be unhappy if my web browser could modify my device drivers on the fly, or poke and peek at network buffers. A machine that was Lisp all the way down today would need to ensure that things like that couldn’t happen.

So may be it would be Lisp all the way down, but you absolutely would not have the kind of ability to poke around in and redefine parts of the guts you had on Lisp machines. Maybe that’s still worth it.

Not to mention that I’m just not very interested in spending a huge amount of time grovelling around in the guts of something like an SSL implementation: those things exist already, and I’d rather do something new and cool. I’d rather do something that Lisp is uniquely suited for, not reinvent wheels. Well, may be that’s just me.

Machines which were Lisp all the way down might, indeed, be interesting, although they could not look like 1980s Lisp machines if they were to be safe. But that does not mean they would need special hardware for Lisp: they wouldn’t. If you want something like this, hardware is not holding you back: there’s no need to endlessly mourn the lost age of Lisp machines, you can start making one now. Shut up and code.

And now we come to the really strange arguments, the arguments that we need special Lisp machines either for reasons which turn out to be straightforwardly false, or because we need something that Lisp machines never were.

‘Good Lisp compilers are too hard to write for stock hardware’

This mantra is getting old.

The most important thing is that we have good stock-hardware Lisp compilers today. As an example, today’s CL compilers are not far from CLANG/LLVM for floating-point code. I tested SBCL and LispWorks: it would be interesting to know how many times more work has gone into LLVM than them for such a relatively small improvement. I can’t imagine a world where these two CL compilers would not be at least comparable to LLVM if similar effort was spent on them⁴.

These things are so much better than the wet-cardboard-and-string compilers that the LispMs had it’s not funny. In particular, if some mythical ‘dedicated Lisp hardware’ made it possible to write a Lisp compiler which generated significantly faster code, then code from Lisp compilers would comprehensively outperform C and Fortran compilers: does that seem plausible? I thought not.

A large amount of work is also going into compilation for other dynamically-typed, interactive languages which aim at high performance. That means on-the-fly compilation and recompilation of code where both the compilation and the resulting code must be quick. Example: Julia. Any of that development could be reused by Lisp compiler writers if they needed to or wanted to (I don’t know if they do, or should).

Ah, but then it turns out that that’s not what is meant by a ‘good compiler’ after all. It turns out that ‘good’ means ‘compillation is fast’.

All these compilers are pretty quick: the computational resources used by even a pretty hairy compiler have not scaled anything like as fast as those needed for the problems we want to solve (that’s why Julia can use LLVM on the fly). Compilation is also not an Amdahl bottleneck as it can happen on the node that needs the compiled code.

Compilers are so quick that a widely-used CL implementation exists where EVAL uses the compiler, unless you ask it not to.

Compilation options are also a thing: you can ask compilers to be quick, fussy, sloppy, safe, produce fast code and so on. Some radically modern languages also allow this to be done in a standardised (but extensible) way at the language level, so you can say ‘make this inner loop really quick, and I have checked all the bounds so don’t bother with that’.

The tradeoff between a fast Lisp compiler and a really good Lisp compiler is imaginary, at this point.

‘They had wonderful keyboards’

Well, if you didn’t mind the weird layouts: yes, they did⁵. And has exactly nothing to do with Lisp.

And so it goes on.

Bored now

There’s a well-known syndrome amongst photographers and musicians called GAS: gear acquisition syndrome. Sufferers from this⁶ pursue an endless stream of purchases of gear — cameras, guitars, FX pedals, the last long-expired batch of a legendary printing paper — in the strange hope that the next camera, the next pedal, that paper, will bring out the Don McCullin, Jimmy Page or Chris Killip in them. Because, of course, Don McCullin & Chris Killip only took the pictures they did because he had the right cameras: it was nothing to do with talent, practice or courage, no.

GAS is a lie we tell ourselves to avoid the awkward reality that what we actually need to do is practice, a lot, and that even if we did that we might not actually be very talented.

Lisp machine romanticism is the same thing: a wall we build ourself so that, somehow unable to climb over it or knock it down, we never have to face the fact that the only thing stopping us is us.

There is no purpose to arguing with Lisp machine romantics because they will never accept that the person building the endless barriers in their way is the same person they see in the mirror every morning. They’re too busy building the walls.

As a footnote, I went to a talk by an HPC person in the early 90s (so: after the end of the cold war⁷ and when the HPC money had gone) where they said that HPC people needed to be aiming at machines based on what big commercial systems looked like as nobody was going to fund dedicated HPC designs any more. At the time that meant big cache-coherent SMP systems. Those hit their limits and have really died out now: the bank I worked for had dozens of fully-populated big SMP systems in 2007, it perhaps still has one or two they can’t get rid of because of some legacy application. So HPC people now run on enormous shared-nothing farms of close-to-commodity processors with very fat interconnect and are wondering about / using GPUs. That’s similar to what happened to Lisp systems, of course: perhaps, in the HPC world, there are romantics who mourn the lost glories of the Cray–3. Well, if I was giving a talk to people interested in the possibilities of hardware today I’d be saying that in a few years there are going to be a lot of huge farms of GPUs going very cheap if you can afford the power. People could be looking at whether those can be used for anything more interesting than the huge neural networks they were designed for. I don’t know if they can.

Before that I had read about Common Lisp but actually written programs in Cambridge Lisp and Standard Lisp. ↩
This had a lot of memory and a higher-resolution screen, I think, and probably was bundled with a rebadged Lucid Common Lisp. ↩
I am at the younger end of people who used these machines in anger: I was not there for the early part of the history described here, and I was also not in the right part of the world at a time when that mattered more. But I wrote Lisp from about 1985 and used Lisp machines of both families from 1989 until the mid to late 1990s. I know from first-hand experience what these machines were like. ↩
If anyone has good knowledge of Arm64 (specifically Apple M1) assembler and performance, and the patience to pore over a couple of assembler listings and work out performance differences, please get in touch. I have written most of a document exploring the difference in performance, but I lost the will to live at the point where it came down to understanding just what details made the LLVM code faster. All the compilers seem to do a good job of the actual float code, but perhaps things like array access or loop overhead are a little slower in Lisp. The difference between SBCL & LLVM is a factor of under 1.2. ↩
The Sun type 3 keyboard was both wonderful and did not have a weird layout, so there’s that. ↩
I am one: I know what I’m talking about here. ↩
The cold war did not end in 1991. America did not win. ↩

Mu

2025-10-08T09:19:00Z

Or: a better world.

We live in a timeline where a lot of people (almost always white men) read Ayn Rand and various SF and fantasy books at an impressionable age, as very many people did, but then somehow never grew up and are busily burning the world as a result. I miss the timeline where those same people read the Illuminatus! books at the same age and never grew out of them. That world would be so much better and so much more interesting.

I can report though, that the same ‘how did these people never grow up?’ thing would apply. I read the books at perhaps 14 and spent much too much time having stoned conversations about what it all meant. I mean, I went to talks by Robert Anton Wilson, I was that far gone. About a decade ago I found my ragged copy of one of them, bought the other two and reread them and … well, don’t do that: you can’t jump from this timeline into that, better, one if you are macroscopic and warm.

But the people who never grew up would be doing stuff which was so much more interesting.

I’m pretty sure, by the way, that the timelines forked in about 1980. I clearly remember seeing a band called The Justified Ancients of Mu (there was only one ‘mu’ I am sure) at what I think was a Deeply Vale festival about then (it must have been one of the later ones as I would have been too young for the early ones). They can’t have been an incarnation of the KLF: they were something related to Nik Turner I think. But … perhaps they were, perhaps it was still possible, then, for there to be leaks between the timelines, and in that lost timeline everything started a decade earlier. Who knows, really? Perhaps, then, I could have escaped into that better world? The internet knows nothing about them now.

Somehow I am sure this is related to the lost decade.

The modern real programmer

2025-01-31T19:17:18Z

This is adapted from an email from my friend Zyni, used with her permission. Don’t take it too seriously.

Real programmers do not write programs like this. If a real programmer has to deal with a collection of particles, they do not have some silly object which represents a particle, perhaps made up of other objects representing physical vectors, and then some array of pointers to these particle objects. That is a bourgeois fantasy and the people who do that will not long survive the revolution. They will die due to excessive pointer-chasing; many of them have already died of quiche.

Real programmers do today as they have always done: if they have some particles to simulate a galaxy they make an array of floating point numbers, in which the particles live.

This is how it has always been done, and how it always will be done, by people who care about performance.

And this is why Lisp is so superb. Because you can write this:

(for* ((i1 (in-particle-vector-indices pv))
       (i2 (in-particle-vector-indices pv i1)))
  (declare (type particle-vector-index i1 i2))
  (with-particle-at (i1 pv :name p1)
    (with-particle-at (i2 pv :name p2)
      (let/fpv ((rx (- p2-x p1-x))
                (ry ...)
                ...)
        ... compute interactions ...))))

And this is:

very fast¹, because it all turns into optimized loops over suitable (simple-array double-float (*)) with no silly objects or consing;
relatively easy for a human to read, since you can see, for instance what (for ((i (in-particle-vector-indices v))) ...) is doing and do not have to second-guess some idiot loop form which will be full of obscure bugs;
quiche-compatible: you can easily write a function particle-at which will construct a particle object from a particle vector entry (such a function will later be excised as it has no callers, of course);
perhaps most important it is possible for a program to take this code and to look at it and to say, ‘OK, this is an iteration over a particle vector – it is not some stupid hard-to-parse (loop for ... oh I have no idea what this is ...) as used by the quiche people, it is (for ((i (in-particle-vector-indices v))) ...) and it is very easy to see what this is – and there are things I can do with that’ and generate Fortran which can be easily (or, less difficultly — is ‘difficultly’ a word? English is so hard) be made to run well on proper machines with sensible numbers of processors.

And this is the thing they still do not see. You write your program which uses the only useful data structure, but you also write your program in a language you have built designed so that both a human and another program can understand it, and do useful things with it, because your program says what it means. Every construct in your program should be designed so that this other program can get semantic information from that construct to turn it into something else.

And this is why Lisp is so uniquely useful for real orogrammers. Lisp has only one interesting feature today: it is a language not for writing programs, but for writing languages.

That is what real programmers do: they build languages to solve their problems. The real programmer understands only two things:

the only data structure worth knowing about is the array;
her job as a programmer is to write languages which will make writing programs to manipulate arrays easy for a human to understand;
and her other job is to write other programs which will take these programs and turn them into Fortran;
and when that is done she can go and ride her lovely cob to the fair.

Real programmers also can count only to two.

I (Tim, not Zyni, who would use a cleverer integrator) wrote a mindless program to integrate systems of gravitating particles to test some of the things we’ve written that are mentioned in this email. On an Apple M1 it sustains well over 1 double precision GFLOP. Without using the GPU I think this is about what the processor can do. ↩

A history of the near future

2024-11-02T09:33:27Z

Det er vanskeligt at spaa, især naar det gælder Fremtiden.

— maybe Niels Bohr

Here’s what I think is coming and why: make of it what you will.

Some level of mathematics is assumed in the first sections, but you can probably understand most of it without following the maths. Energy and human ambitions on a finite planet by Tom Murphy as well as his blog discuss a lot of this in more detail, although he reaches different conclusions.

Growth, the sacred cow
Finiteness
Are we going to have a problem here?
The magic goes away
Faith, fantasy and Economics
The elephant
The teeming billions
As above, so below
Shorter
A history of the near future

Growth, the sacred cow

The UK’s economy grew by 0.6% between April and June as it continued its recovery from the recession at the end of last year.

“The UK economy has now grown strongly for two quarters, following the weakness we saw in the second half of last year,” said Liz McKeown, director of economic statistics at the Office for National Statistics, which released the figures.

— UK economy continues recovery with 0.6% growth, BBC, 15th August 2024

What growth means

What does this mean? What is economic growth? Well here’s a definition which I think is pretty close to how most economists would define it:

Economic growth is an increase in the quantity and quality of the economic goods and services that a society produces.

— Max Roser, Our World in Data

That article goes on to suggest a slightly briefer definition:

Economic growth is an increase in the value of the economic products that a society produces.

This is only part of a useful definition however, and it misses something very important. When you see figures for economic growth, which I will just call ‘growth’ from now on, referenced, they are always percentages: 2% for instance. And they are percentages over a period: ‘The UK’s economy grew by 0.6% between April and June’, so over a period of 3 months. Well, a percentage is a percentage of something, and in the case of growth, it is a percentage of the current size of the economy: a percentage of some measure of the value of the products currently being produced. And generally the growth rate peeople quote is the percentage change over a year. So, if we call the value of products being produced $s$, the definition of growth that people quote, $G$, is

$$G = 100 \frac{s(t + 1) - s(t)}{s(t)}$$

where $t$ is measured in years. So we can define a more mathematically useful thing, $g$:

$$\begin{aligned} g &= \frac{1}{s(t)} \lim_{\Delta t \to 0} \frac{s(t + \Delta t) - s(t)}{\Delta t}\ &= \frac{1}{s}\frac{ds}{dt} \end{aligned}$$

In other words $ds/dt = gs(t)$. If we assume that $g$ is a constant then we can simply write down $s(t)$:

$$s(t) = s_0e^{gt}$$

And this will give us $G$ in terms of $g$:

$$\begin{aligned} G &= 100\frac{s(t+1) - s(t)}{s(t)}\ &= 100(e^g – 1) \end{aligned}$$

and so

$$\begin{aligned} g &= \ln\left(\frac{G}{100} + 1\right)\ &\approx \frac{G}{100}\quad\text{when } G \ll 100 \end{aligned}$$ This slightly overestimates $g$, as $\ln(x+1) = x - x^2/2 + x^3/3 - \ldots$, so given $G = 2$, we get an increase by a factor about 2.02% per year with the approximate value for $g$. Things start to get significantly bad when $G \gtrsim 10$, when the annual change is about 10.5%, but since $G$ is usually a few percent, this is fine.

Another useful approximation when thinking about growth, and other similar processes such as compound interest, is the rule of 70. This tells you the approximate time for the size to double for a given $G$. For the size to double, $e^{gt_2} = 2$, so

$$\begin{aligned} t_2 &= \frac{\ln 2}{g}\ &\approx \frac{100\ln 2}{G}\ &\approx \frac{70}{G} \end{aligned}$$

So that’s the rule of 70, and it says that if $G=2,%$ then the doubling time is about 35 years. This again is true when $G \ll 100$.

The important thing here is that if $G$ is approximately constant then the size of an economy — the value of the economic products it produces — increases exponentially with time.

Something to remember is that growth is not currency inflation. In January 1971 a pint of milk cost 5p in the UK, while in January 2024 it cost 66p (ONS). That’s because there has been inflation and the currency has been rescaled over time: it’s nothing to do with growth, which would factor this out. Milk may have become more expensive compared to our incomes since 1971, but that’s not because it now costs 66p rather than 5p a pint. Inflation is also an exponential process, so if inflation is 2% then the value of the currency halves in about 35 years. When inflation becomes large, or negative, it can have disastrous effects on growth, but inflation is not growth: the ‘value’ that is talked about is when thinking about growth is the value after inflation is factored out. If there has been growth since 1971, and if something odd has not happened to the price of milk, then the average person could afford to buy more milk today than they could in 1971.

Here is a slightly more fleshed out expression for $s$ which I’ll use later:

$$s(t) = s_0 e^{g(t - t_0)}\tag{size}$$

Here $s_0$ is the size of the economy at time $t_0$.

Here’s a plot of the relative size of an economy compared to 2000, from 1900 to 2100 if growth is 2%/year.

Economy size compared to 2000 at 2%/year growth

If growth is not constant

Obviously in real life $g$ is not constant: there are recessions, and so on. But, again in real life, people worry when growth is low and do things to push it back up. And there are practical bounds on how large it can be outside of the fantasies of people who believe in the singularity. The end result is that growth still looks exponential.

Here is a little model of that: in this model the target growth is 2%/year, but it is being kicked around each year by a normally-distributed random variable with a standard deviation equivalent to 0.2%. It’s then being corrected each year back towards the target growth by 2% of the difference.

Here’s a plot of five example sequences of growth for 100 years.

Five samples of annual percentage growths over 100 years

You can see that some do better than others.

Here is a plot of the corresponding growth curves

Five samples of economic growth over 100 years

Again, you can see that some of these simulated economies do better than others. But, particularly if you plot things on a log scale and for a longer time period you can see that the growth curves are basically exponentials:

Five samples of economic growth over 400 years, log scale

One interesting thing here is how misleading using logarithmic axes can be: the above plot makes it pretty obvious that all the growth curves are basically exponential, but if you look at it without the logarithmic scaling then you can see that some of the economies do radically better than others, which is much harder to see in the log scaled plot.

Five samples of economic growth over 400 years, linear scale

Of course these are just toy models: how growth changes in real life is going to be more complicated than this. However the important thing is this: sustained growth corresponds to an exponential increase in the size of the economy with time.

Growth, resources and waste products

So, now, let’s think about what $s$ means. An economy with a given size is presumably making use of resources — raw materials, energy and so on — at some rate. So, for instance, thinking about energy, an economy of a certain size has a corresponding power: a rate at which it uses energy. Or in terms of some material, say iron, an economy of a certain size is consuming iron at a certain rate (much of this iron may not be newly mined, but be recycled).

Because it uses resources the economy is also producing waste products at some rate, which depends in part on how efficiently the resources are used. And these waste products do more or less damage to the environment the whole system lives in, depending on what they are: from completely harmless to extremely nasty.

There are lots of different resources with lots of different waste products, of course, and conflating them is wrong, as the supplies of each resource may be very different and their waste products are all different. So let’s just think about some particular resource or waste product and not worry too much about the complexities. If it matters, think about the resource as being energy. Then the rate at which energy is being used will be power. How efficiently the economy uses power is called the energy intensity of the economy: I’ll use the term ‘intensity’ more generally for other resources.

The total rate of resource usage can be written as $R = \rho s(t)$: $\rho$ is the intensity. If growth is exponential then $R = \rho s_0 e^{gt}$. But of course, $\rho = \rho(t)$, or even perhaps $\rho(t, s)$: $\rho$ depends on time, and perhaps also on the size of the economy, so $R$ in fact changes in a more complicated way with time.

In the simple case, $\rho(t) = \rho_0$: the intensity is just a constant. In that case if the economy grows exponentially, so will resource usage.

Let’s look at cases where $R$ changes in some way that is decoupled from $s$, and in particular cases where, if $g$ remains constant, $R$ grows more slowly than exponentially. In particular let’s think about cases where, as $t\to+\infty$, $R$ tends to some constant value. Then it’s obvious that as $t\to+\infty$, $\rho \sim e^{-gt}$. So if, as $t$ becomes large and positive, we let $\rho(t) = \rho_0 e^{-gt}$ then we get $R = \rho_0 s_0$ when $t$ is large and positive.

But you can’t just assume that $\rho$ looks like this: if we look at, for instance, energy usage by humans over time, it has very obviously not been constant but has increased dramatically since 1800. Our World in Data has a chart of energy usage since 1800. Here is a plot of this together with two exponential curves: the first increasing by 0.8%/y from the 1800 value, and the second by 2.2%/y from the 1900 value.

Global energy usage, 1800–2023

These curves were fitted by eye rather than by anything formal, but they are clearly not awful. Here is a log-scaled version of the plot which may make it easier to see the fit.

Global energy usage, 1800–2023, log scale

Global energy usage is fairly well described by an increase of 0.8%/y before about 1900 and by an increase of about 2.2%/y after that.

So what we’re after is a model where $\rho$ starts off being roughly constant, so resource usage increases exponentially at first, and then becomes more like a decreasing exponential, causing resource usage to become asymptotically constant. A logistic function does this:

$$\rho(t) = \rho_0 \left(1 - \frac{1}{1 + e^{g(t - t_2)}}\right)\tag{rho}$$

where $t_2$ is the time when $\rho = \rho_0/2$, the time when the amount of resource usage has halved. The expression for $\rho$ does not have to be a logistic function of course, it just needs to be approximately constant when $t\to-\infty$ and to look like $e^{-gt}$ when $t\to+\infty$: the logistic function is just an easy way of achieving that.

So, given this, we can plot how the size of the economy and the rate of resource usage changes with time for $G=2,%$ and for various values of $t_2$:

Economy size and resource usage

We can also plot the proportion of the total size of the economy which involves resource usage: this is just $\rho(t)/\rho_0$:

Resource percentage

The proportion of the economy using resources becomes incredibly small as time goes on, so it’s easier to see this using a log scale:

Resource percentage, log scale

An equivalent thing holds for waste products of course. You might want to make things more complicated by saying that $W$, the rate of production of waste products for some resource, is $W = \omega(t)\rho(t)s(t)$, where $\omega$ tells you how much waste product you produce per unit resource.

And in real life there are lots of resources and lots of waste products and all this is a hugely oversimplified spherical cow model. But it will serve.

Why growth matters

First of all it matters because for most of history most humans lived in conditions which were unspeakably awful. Having more food and water, more medical supplies, better housing and so on, all of which are things that growth has provided, has been enormously good for humanity. In 1820, about 75% of the world’s population lived in extreme poverty, while in 2010 only about 10% did (source). Growth did that.

Of course growth is not the only thing that matters: growth says nothing about inequality, and inequality also matters a great deal. Growth doesn’t help much if only a tiny number of people see an enormous improvement with almost everyone else experiencing none.

Lifting vast numbers of people out of extreme poverty is a really good reason for growth. A less good reason is that we’ve built economies on the idea that it will continue for ever. The easiest way to see this is to think about a person who wants to borrow money. If they know that, in 35 years, they will have twice the income they have today then they’ll be willing (and, if they can convince the lender, able) to borrow considerably more than if they know that their income will be the same. Well, the same is true for economies: if you assume that your country’s economy will grow indefinitely at 2%, the government can borrow more against that future wealth. Better hope growth doesn’t stop, then. Fortunately, since you are the government, you have at least some influence on growth.

The benefits of growth also diminish as people get richer: If everyone become a thousand times better better off (at 2% growth a year this will take about 350 years) is everyone a thousand times happier, or will they live a thousand times longer? Well, we can ask that today: in 2023 the median net worth of Americans aged 55–64 was $365,000 (source). Jeff Bezos, who was 60 in 2024, has a net worth in 2024 of $205,800,000,000 (source): he’s more than half a million times more wealthy than the median American his age. So think about three people: one who is half a million times poorer than the median American, a median American, and Jeff Bezos. Well, the person who is half a million times poorer has a net worth of under $1: they are probably very miserable indeed as they certainly are starving. Perhaps the average American is half a million times happier than them. But is Jeff Bezos half a million times happier than the median American of his age? Will he live half a million times longer? Is his life a half a million times better than theirs in any respect other than having that much more money? I doubt it. So growth has diminishing returns as time goes on.

Summary

In summary, if growth continues, then the size of the economy will increase exponentially with time. So

either the resource cost per unit value, $\rho$, must start to fall exponentially with time, as in the examples above;
or the consumption of resources will rise exponentially over time.

There is no third option, if growth is to continue.

In real life, of course, different kinds of resources have different characteristics as I mentioned above. The total amount of iron available on a planet is finite, while the amount of energy is not, but the amount of power is. So it’s not right to bundle them all into some single number. Eventually an economy where $\rho$ does not decrease exponentially will just have used all the iron there is, for instance: however fast it recycles old iron it will not be able to find enough to sustain its current infrastructure: it will need to start using something else.

Finally, there is a difference between the growth of an economy and the growth experienced by people living in that economy. That’s because population changes, and also because of inequality. Well, here is a plot from the World Bank of world GDP growth per capita since 1961: growth per person has perhaps averaged around 2% over that time. That graph also doesn’t address changes in inequality: it’s quite possible for almost all the growth to end up in the hands of a tiny number of people with almost everyone else getting poorer, and this may be happening.

So, now, why is it reasonable to assume that the rate of resource usage must tend to some constant? The universe is big, after all: there are really a lot of resources out there. Why can’t we just zoom off into space and use them?

Finiteness

Or, science fiction is fiction.

The future we imagine

In a few years, humanity will be a spacefaring species. From the early bases on the Moon and in Earth orbit we will have settled on Mars, and in the asteroid belt. On Mars, the gleaming cities under their geodesic domes will herald a new enlightenment. Great forests will bloom there, with trees far taller than their ancestors on Earth, producing oxygen and raw materials. The Martians, for so they truly are, will be tall, willowy and universally attractive. Somehow they will all be blond (it is, perhaps better not to ask what happened to the ones who were not). The mines will be conveniently out of sight, as will those lesser humans who labour in them.

In the asteroid belt things will be different, but not worse. Belters are, somehow, always Scottish or South African, and heavily-muscled. They work with the vast supply of metals and other raw materials they win from the asteroids which drift in vast numbers about them, conveniently slowly. They live in cities which look like futuristic shipyards, for so they are. In their cradles are built the great ships which already travel to the gas giants of the outer Solar system and will soon, soon, take humanity on its first journey to the stars. The bright white lights of welding flames are everywhere, together with the actinic blue flame of propulsion systems.

And the true space people live where they choose. In their vast, spinning spoked wheels they reside in orbit around Earth, Mars, Jupiter and Saturn, and often simply in interplanetary space. The insides of their craft are gleaming white, the walls containing panels displaying ever-changing status displays. The men all have advanced degrees in hard science subjects, the women all have short skirts and perfect legs. Again, there are curiously few people with brown or black skins.

Earth, for those who choose to remain there, is become a green paradise where humans live a sylvan, perfect life amongst the gambolling animals, free of all care.

Enough, enough.

Some inconvenient truths

If you like this picture, I suggest you read A city on Mars. Go ahead, I’ll wait.

The sketch above is science fiction, and science fiction is fiction. And the kind of science fiction I parodied above is the kind which depends on assumptions which are not true or at least completely beyond anything we have any idea how to do.

I don’t want to repeat all the arguments made in A city on Mars. But let’s just talk about Mars for a bit. If you want to think about living on Mars, think about living in Antarctica. In Antarctica you can breathe the air; in Antarctica there is plenty of accessible water; in Antarctica you’re not continually being exposed to high levels of radiation; in Antarctica you can get supplies or be rescued in a day or two. And yet people have not settled Antarctica, or various desert regions, because they are too hostile and difficult to live in. Mars is like Antarctica, but hostile: there is no air, no accessible water, there are high levels of radiation, the surface is made of toxic dust, and resupply or rescue takes between months and years. And it is extremely difficult and expensive to get there, and you get irradiated the whole way there and back.

Mars is nasty. The asteroid belt is in many ways worse. Living in it, or in space, either requires really big spinning spacecraft, or for humans to live in free-fall indefinitely, which is not good for them.

All of these places require the construction of a closed ecosystem which can remain habitable indefinitely. All of these places also mean you are exposed to a radiation environment which is probably going to kill you if you are exposed to it for long periods: no human has ever lived in it for more than a few days, and nobody has done so at all since the early 1970s. And let’s not even start on the question of having children in space or on Mars: would you want to be the person to try that experiment?

And that closed ecosystem needs to be big: if humans are going to live independently for generations on Mars, say, then you need enough of them to form a ‘minimum viable population’. This isn’t just a population big enough to avoid inbreeding: it’s a population big enough to grow food, to maintain the life-support systems, to build and maintain all the advanced machines that will be needed and so on. These things don’t happen on their own. How big such a population needs to be is not clear, but it’s between tens of thousands and many millions. We’re not talking a few people living in the converted hull of their spaceship here.

These problems can, perhaps, be overcome: just not soon. And, curiously, none of the plutocrats who want us to go to Mars or live in space are undertaking the kind of long-term research and development programme which would be needed to overcome them. They’re just building big, cool, profitable, rockets.

What is going on?

I think Charlie Stross describes it rather well. We’re living in an age where a bunch of plutocrats grew up on the kind of science fiction written in English up to the 1990s, or perhaps a bit later. And they took two things away from it:

they did not understand that it was, well, fiction: things made up so the story would be more exciting, like big exciting rockets and space, and certainly not boring details like keeping people alive on Mars which do not make exciting stories;
they read about various awful dystopian futures in SF books and thought ‘hmm, that looks pretty good for very, very rich people who don’t care about anyone else, and I already don’t care about anyone else’.

And so we get to where we are: techbro plutocrats are planning to go to Mars, but they’re planning to go there based on stories which left out all the boring details like staying alive. They think that it just needs big shiny rockets. At the same time, they’re working towards the kind of dystopia described in, for instance, the sprawl trilogy, because things are pretty good in those books if you’re a Tessier-Ashpool, and that’s who they think they are.

And, of course, they think they can combine the two: a dystopia on Earth and the shiny Mars city where they can be very gods: if you don’t do what they say on their imagined future Mars you don’t breathe. They don’t care, at all, if achieving their dream destroys the dreams of everybody else.

If the whole ‘all you need is a shiny rocket to go to Mars’ seems like a mistake nobody very smart would make I think it’s worth considering Elon Musk’s recent behaviour (I am writing this in September 2024): he is, to put it mildly, not as smart as he thinks he is. On the other hand he’s at least as unpleasant as people feared he might be. The techbro plutocrats got to be where they are by a combination of being in the right place at the right time, and not caring about anyone but themselves at all: they didn’t get there by being some kind of genius.

That is what is going on.

It’s worth saying here that SpaceX, in particular, has done absolutely amazing things: Musk made a good decision when he decided to fund them. They have successfully done what NASA abysmally failed to do after Apollo. In 2021 dollars, it cost about $5,400/kg to low Earth orbit for the Apollo Saturn launch system; for the space shuttle it was about $64,500/kg; for Falcon heavy it is about $1,500/kg (source). That is an enormous achievement. It’s just not closely related to what is required to sustain a settlement on Mars or the Moon.

The future we’re going to have

We are not going to live in space in significant numbers any time soon. The cost of lifting significant numbers of humans from Earth will be prohibitive both financially and environmentally for a very long time, and quite likely for ever: the great majority of humans will need to live within the resources Earth provides for a long time to come.

We might have small, dependent settlements on Mars or the Moon this century. But, well, they will be small, and dependent on supplies from Earth.

One of the things people like Musk go on about is that we should have a settlement on Mars as a lifeboat for humanity: if something bad happens to Earth humans will survive on Mars. That requires that settlement to be long-term viable without support from Earth, which will not be possible for a long time. And it’s just a stupid idea: if you are going to live on Mars, you’re going to be living underground if you don’t want the radiation environment to kill you. And you will need to make air, and all your food and maintain the environment, essentially for ever. Well, wouldn’t it be just a lot easier to build some number of lifeboats for humanity underground in remote parts of Earth? There is water, there is air, both can be filtered for nasties. It’s hugely cheaper to build than something on Mars. Such settlements would certainly be preferable as a way of ensuring humanity survives a flat-out nuclear war, or catastrophic climate change. They would almost certainly be preferable as a way of surviving a Chicxulub-level extinction event.

Of course it would be easier to do that. But it wouldn’t involve rockets: it would not be the shiny SF dream that the techbro plutocrats so want to come true.

The same applies for slower catastrophes: if you’re in a position to ship many thousand people to Mars, you’re in a position to scrub carbon dioxide from the atmosphere, so why not do that? Because it doesn’t involve shiny rockets, I suppose. Also it involves helping other people, and plutocrats only care about themselves.

Finally, note that even if we ignore all the inconvenient truths about becoming a spacefaring species, it doesn’t solve the exponential problem for long. If Mars has all the resources of Earth, then at 2% resource growth a year, we exhaust the resources of Mars 35 years after we exhaust the resources of Earth. Then we have to move on to even more insane ideas like cannibilizing planets. Which, certainly, will perturb orbits of other planets, planets on which we already live, in ways which will be not good for the people who live there.

The future we are going to have is that we must live within the resources provided by Earth for a long time: probably centuries at least.

Are we going to have a problem here?

OK, so we’re stuck on Earth for a long time. That’s only a problem we need to think about if we are, in fact, running out of resources, or if waste products are going to cause a problem.

The magic goes away

Perhaps, in fact, we’re many doubling periods away from getting anywhere near that point. At 2%/y growth then the rule of 70 says the doubling period is 35 years, so we might have hundreds of years left, and we can just kick the can down the road as we always have. That’s a horrible answer, but it is an answer.

In 1798, Thomas Malthus wrote An essay on the principle of population in which he argued that exponential population growth would result in everyone starving. It hasn’t after more than two hundred years, and in fact a far lower proportion of the human population is starving now than was in 1798. Malthus was wrong, and he was wrong because of something he missed: fossil fuels, which provide both copious energy and fertiliser.

Unfortunately putting over 1.5 trillion tonnes of carbon dioxide into the atmosphere turns out to have had quite bad effects, of course. Do perhaps Malthus wasn’t as wrong as all that: you can indeed solve the problem he was worried about for a couple of centuries, but only if you’re willing to solve a much larger problem later on.

It might be that there is something we don’t know about which changes everything. But we know a lot more about the resources available and the consequences of using them than he did. And we know a lot more about physics and chemistry than he did, and specifically about limits placed on what we can do by physics. Simply assuming that something completely unknown will turn up snd everything will be fine as a result seems like a really stupid approach. So let’s not do that, then: if it looks like we’re not many doubling periods away from trouble let’s assume that’s true.

Energy

We’re not, for instance, running out of energy very fast: there’s a lot of available solar power; nuclear fission power is clean and could be cheap if we wanted it to be. Nuclear fusion will probably actually arrive at some point. We will eventually run out of energy, but we have at least a century even without making dramatic improvements in energy intensity.

Except we really, do need, already, to have stopped burning fossil fuels. But let’s leave that for later.

Energy is not the only fruit

There are other things we might want to worry about than energy, and perhaps the most obvious one is that the environment we live in is also a finite resource, and we’re doing things to it, often by dumping waste products into it, which are not very good.

As an example, we’re degrading the environment in a way which is disastrous for insects: flying insect populations in the UK have declined by a factor of more than 2 over the 20 years to 2022 (original source paper): halving in 20 year is not good, and it’s backed up by many other studies.

There are many other signs that things are not well. Swallows declined by 43% in the UK between 2012 and 2022 (they had been increasing in the 10 years before that), house martins by 40%, swifts by 40% (source). So perhaps I am slightly more interested in swallows, martins and swifts than other people, but that’s not the point: I could live in a world without them, but their decline is not some special case, it’s one example of something pervasive.

The thing that is pervasive is that we are seriously, seriously, degrading the environment that supports us. That degradation is related to the rate of resource usage in the economy. And if it continues for long enough the environment is not going to be able to support us any more.

If we don’t do anything, how long do we have before we’re in trouble? I have no idea. But I do know that by the time we’re in trouble it will be too late: the environment isn’t simple and it’s full of hysteresis. By the time we fall over some tipping point there will probably be no way back, so we’d be well-advised not to push our luck.

Yes, we are going to have a problem here

I don’t know when. But if I had to guess I’d say within fifty years or so, and perhaps sooner than that. And by the time the problem becomes serious it will be too late: we need to do something to adjust the resource intensity — to reduce the $\rho$ — of our economies now.

So, now, let’s find out just how far from reality the fantasy world economists live in is.

Faith, fantasy and Economics

So we’re not going to be living in space any time soon: we need to plan to live within the finite resouces provided by Earth. And the evidence is that we’re already approaching limits of some of those resources. So we’d better plan to make our rate of resource usage a constant, or even make it smaller.

If we assume growth continues at some rate $g$, then the rate of resource usage for some resource is $R = \rho(t)s_0 e^{gt}$. For this to be constant we need $\rho(t)$ to start looking like $e^{-gt}$ in the future, so as time goes on it becomes more and more like $\rho_0 e^{-gt}$, say and resource usage tends to $\rho_0 s_0$.

Efficiency limits: economics meets physics and loses

So it’s tempting to say that, well, we can just make everything more efficient over time, without limit. We can’t. Physics places hard limits on how efficient various processes can be, and however much you might want to exceed those limits you can’t.

For example, modern petrol engines for cars are 30–40% efficient (source), with diesel engines approaching 50% (source). Carnot’s theorem says that the efficiency of a heat engine can never be more than $(T_H - T_C)/T_H$, where $T_H$ and $T_C$ are the absolute temperatures of the hot side and the cold side of the engine. We can’t do much about $T_C$, and $T_H$ is limited by the materials we can use.

Well, perhaps we can use really super amazing materials (let’s hope they’re cheap) and increase efficiency far above what it is today. But wait: car engines are already close to 50% efficient: the very best we can ever achieve is a doubling in efficiency. At 2% improvement a year (which is much more than we will be able to achieve) that’s 35 years. And to get to 95% efficient the hot side of the engine would have to run at mearly 5,500K which is, well, pretty hot.

The same things apply to any other physical process: you just can’t keep on making things more efficient in terms of physical resource use, whatever the resource is at a constant growth rate: things can’t get exponentially more efficient with time, and that’s all there is to it.

Escaping into a pretend world

So, OK, what is the answer to that? Well, the proposed answers are two: substitution. and decoupling.

Substitution is the idea that as we reach limits in one technology we substitute another, more efficient one. It’s not an answer: the efficiency of a car engine can’t even in principle exceed 100%, for instance, whatever technology you use. Sometimes we are using technologies which are extremely inefficient, but there is still a physical limit and it’s often not that far away. Substitution keeps you going for a while, but not for ever.

An example of substitution is that, of course, nobody expects internal combustion engines to reach 95% efficiency: we’ll replace them by electric motors. Which will get us no more than another factor of two or so, because there’s a term for systems with efficiencies above 100%: perpetual motion machines.

Decoupling is the idea that we don’t just have more and more of the same thing: we start doing other things and those things use less resources.

The poster child for decoupling is probably the idea that we will, over time, spend less and less of our time doing any kind of physical thing and more and more of it immersed in virtual worlds running on computers. Let’s conveniently ignore the fact that, Moore’s law notwithstanding, computers also have physical limits on how efficient they can be, and assume we’ll all end up living in The Matrix while, in the real world, consuming very few resources. Perhaps some people would even like that.

Alternatively, perhaps we will end up spending all our time making and trading vastly valuable art, while living otherwise very simple lives. That seems more attractive to me, anyway.

Decoupling is not so obviously silly, but it turns out to be just as mad.

Decoupling: economics meets itself and loses

So, let’s assume that, somehow, some combination of substitution and decoupling means we can maintain economic growth while resource intensity declines exponentially. What happens?

The first thing is that, over time, the resource-consuming part of the economy, and hence the proportion of everyone’s income spent on it, becomes increasingly tiny. Which leaves us in two very weird situations.

Consider living in such a world. A very tiny proportion of your income is spent on things which consume resouces: they are very cheap for you. So let us assume you’re a human being, not some economist’s fantasy idea of one: why wouldn’t you just spend a lot more money on resource-consuming things? Perhaps you decide you’d like to build an enormous steampunk coal-fired machine: you can do this, because you have enormous wealth, everyone has enormous wealth. I mean, the people today who have enormous wealth have things like private jets, and travel by helicopter: why wouldn’t everyone do that in the future?

Something has to stop you. And that thing can’t be that it’s too expensive, because if it’s too expensive then resource consuming parts of the economy are, in fact, a large part of it, not a very tiny one. It just makes no sense.

And that’s not the end. As the resource-consuming parts of the economy become an ever smaller fraction of it, there’s no reason why, at first countries and then later large organisations and wealthy individuals might not decide to just buy them all, and hold everyone else to ransom.

The whole idea of decoupling means that there is a tiny fraction of the economy which is nevertheless critical for everything else to exist: you still need to be warm, to be sheltered, to be able to eat, to travel, it’s just all now very, very cheap. That’s not how things work: if there is a resource which, if you own it, gives you enormous power over over large parts of an economy, then that rerource is extremely valuable, not extremely cheap.

Well, perhaps it’s like food. Food, by historical standards, is very cheap in developed countries. In 1929, food cost about 23% of Americans’ disposable incomes: in 2022 it was 11% (source). And that’s only in recent history. And people do eat both more, and more resource-intensive food as a result. But there’s a limit: eating too much isn’t good for you, and most of the cost of very expensive meals is the restaurant. So there’s a limit on how much you can eat.

It’s not like food: my one-man Mars-capable racing spacecraft uses really, really a lot of resources.

So no, none of this makes any sense: the only conclusion is that something else will happen.

The elephant

There is an elephant: there is always an elephant.

The elephant, of course, is climate change. Climate change — specifically climate change caused by humans — happens becsuse of two waste products from resource usage:

carbon dioxide which is a waste product from the burning of fossil fuels;
methane, which is a waste product coming about 40% from agriculture 35% from fossil fuels again and 20% from other waste (source).

Carbon dioxide is much longer-lived than methane and also is a larger contributor to the current warming.

There are many, many articles on global warming as well as many, many scientific papers. I’m not going to write another one: if you want a comprehensive summary look at the IPCC. There are also many liars and fools who claim global warming is not happening, that it is happening and it’s not a problem, or that it is happening, it is a problem, but we can’t do anything about it: at this point those people are, well, liars. fools, or both.

Here are plots of average annual and summer (June, July, August) temperatures for central England since 1824, averaged over five years, using data from the Met Office.

Five-year average of central England annual temperature, 1824–2023

Five-year average of central England summer temperature, 1824–2023

Yes: climate change is happening.

Climate change a really good example of where we need to dramatically reduce the intensity of something, which in this case is waste products from consumption of energy. We know the consequences of not doing so will be very bad, and we know a lot of approaches for reducing the intensity. Unfortunately most of them require humans to make changes they find inconvenient. But, still, we know we have to do this, and in 2015 everyone agreed to do so.

How are we doing?

Here is a plot of data from the Mauna Loa Observatory, showing both the measured atmospheric concentration of carbon dioxide, and a version of the same data with the annual cycle removed.

Mauna Loa atmospheric carbon dioxide concentration

It’s worth looking at the data starting in 2010 as well:

Mauna Loa atmospheric carbon dioxide concentration since 2010

Here are equivalent plots of data showing the global average atmospheric carbon dioxide concentration

Global average atmospheric carbon dioxide concentration

And again, starting in 2010

Global average atmospheric carbon dioxide concentration

So, well, how about fossil fuel usage? Data for this comes from Our World in Data.

First the combined totals:

Global fossil fuel consumption

And broken down by fuel:

Global fossil fuel consumption by fuel

What this looks like

What this looks like is abject failure: we are, perhaps, doing something although exactly what is not visible in the concentration data. But we are doing so far from enough as not to be funny.

We know about climate change, we know how serious it is, we know how to fix it. And we are failing.

The teeming billions

So, here’s the bit about how overpopulation is going to doom us all.

Not so much. For much of the world growing populations are no longer the appropriate thing to worry about: the problem is now, or will soon be declining populations.

The rate at which a population grows or shrinks is determined both by how many people die and how many are born. The second of these is usually summarised by a number called the total fertility rate, which is the average number of live births per woman under various assumptions. It’s not completely straightforward to know how this number corresponds to population change, as it depends on things like mortality rates and sex ratios at birth. But for a population with current first-world mortality rates a value of a little over 2 — perhaps 2.1 — is the replacement rate: at this value the population will be stable.

In England and Wales TFR fell below 1.5 in 2022. In Scotland TFR was 1.28 in 2022. This is not just below the replacement rate it’s far below it. And the UK is not unusual: the TFR in the USA is about 1.7, in France it is 1.8, Germany 1.5. In South Korea it is 0.8 (that is not a typo). More figures can be found from the Population Reference Bureau.

The consequences of this are that the population in these countries will fall, and the world population also probably will soon start falling. ‘Good’, you might say. But that’s not the only consequence. The age statistics of the population also will change dramatically.

I wrote a toy model which lets me play around with what happens. I didn’t want to deal with sex ratios, so in the model there is only one sex: if you like, there are only women. A proportion of the people die each year in a controllable age-dependent way, and they produce offspring in a controllable age-dependent way (in the runs below from 20 to 40) At 100 I kill off the (very small proportion of) survivors to avoid dealing with a tiny but long tail of very old members.

Rather than TFR the model deals in the effective fertility rate, EFR, which is defined to include mortality and thus needs to be 1 for a stable population (TFR would be a little higher, but is about half that for a real population as there is only one sex).

This is very much a toy model: it’s not anywhere close to accurate for humans. But it’s realistic enough to demonstrate in outline what happens to demographics if fertility falls dramatically. Here are example pictures from two setups. In both of these the initial mortality rates are

5%/year in the first two years;
1%/year from 2–9;
0.1%/year from 10–19;
0.5%/year from 20–39 (childbirth is a bit dangerous);
0.1%/year from 40–49;
1%/year from 50–59;
2%/year from 60–69;
6%/year from 70–79;
12%/year from 80/99;
100%/year at 99.

Fertility is 10%/year between 20 and 39, and zero outside that window.

In fact these values get smoothed out a bit to make the graphs look nicer. The EFR of the system set up like this is about 1.54.

The model is then spun up for 1000 years, with the final population being renormalised to 100 million in total.

At this point the plot of numbers against age (the ‘demographic pyramid’) looks like this:

Population by age, after spin up

Falling fertility

In this setup, fertility is reduced by 2%/year for 40 years (note: this means 2% of 10%/year initially). The EFR at the end of this process is about 0.69. After this the system runs up to year 200, at which point fertility is adjusted so the EFR is 1. It then runs up to year 400.

Here are graphs at year 0, 20, 40, 60, 80, 100, 200, 400:

Population by age, at year 0, 20, 40, 60, 80, 100, 200, 400

You can see two things from this: the population falls over time, but the demographic pyramid also changes shape dramatically. It’s easier to see this if you renormalize these plots so the average population for each age is 1:

Renormalized population by age, at year 0, 20, 40, 60, 80, 100, 200, 400

You can see here that the demographics get very heavily skewed to old people compared to the initial state while the population is falling. Once it stabilises they remain skewed to older people but less heavily so.

Here is a picture of what happens to the total population in this scenario

Population history

The interesting thing you can see here is the latency: it takes about 40 years before the population starts falling, and a similar amount of time for it to stabilise. That’s true even if you step fertility so the EFR is 0.69 in the first year, although the demographic pyramid looks bizarre in that case (it has a huge step in it which gradually gets washed out).

Falling fertility, rising mortality

You can ‘fix’ the skewing towards old people as EFR falls by assuming that old people become more likely to die.

In the run below, mortality is increased by 0.1%/year (relative to the current mortality for that year) from age 30 for the first 40 years of the run. Mortality is not reset at year 200.

Here are the unrenormalised and renormalised plots, as above

Population by age, at year 0, 20, 40, 60, 80, 100, 200, 400, increased mortality

Renormalized population by age, at year 0, 20, 40, 60, 80, 100, 200, 400, increased mortality

You can see this is ‘better’. At least it’s better until you realise what’s happened to the population:

Population history, increased mortality

In the case where the mortality of older people does not increase the population falls to about 18% of its initial value over 400 years. In the case where it does, the final population is about 3.7% of the initial one. Which should not be very surprising, of course.

The consequences

Population growth is a bit like economic growth, to which it is closely related: if it continues at a positive rate then you can assume that, in the future there will be more people than there are now. In particular there will be more people of working age. Those people earn money which pays taxes which, among other things, pay for the care of the elderly. In a world where the TFR remains at some constant value and where age-related mortality doesn’t change over time, it also has the nice characteristic that it’s an exponential process, and so ‘everything remains the same’: in the future there will be more or fewer people, but the proportions of people alive at a given age remain the same.

In other words, if the TFR remains at some more-or-less constant value comfortably above 2 and mortality rates remain about the same, then you can always plan to pay, for instance, pensions, out of money you don’t yet have. You do run into the problem that, in due course, everyone starves as the population increases exponentially and the food supply runs out, but that’s not the sort of thing economists worry about: they just kick that can down the road, which has worked well for them so far.

The real world is increasingly unlike that: TFR has historically reduced over time, and mortality has also fallen. We’ve already seen some of the consequences of life expectancy increasing: people have to work until they are much older.

In many parts of the world TFR is now well below replacement value, and often well below it. That’s going to have very big consequences. For now, those places could fix their problem by importing people from regions of the world where the TFR remains high. They’re generally not doing that, apparently because those people have skins of a different colour.

As above, so below

Quod est superius est sicut quod inferius, et quod inferius est sicut quod est superius.

— Latin translation of Emerald tablet

Everything above is meant to be based on information you can check: most of what follows is is my opinion.

Shorter

Economic growth is a very good thing: it’s lifted billions of people out of poverty. But it’s an exponential process.

Despite the fantasies of plutocrats who read too much science fiction at an impressionable age and who are not as smart as they think they are, we’re not going to migrate into space, the Moon or Mars any time soon, if ever. We need to live within the finite resources provided by Earth.

Continuing exponential growth in a system with finite resources is not possible unless the thing that is growing doesn’t use the resources. This means economic growth must be decoupled from resources at an exponential rate, and that is what economists assume will happen. Generally speaking it’s not what has happened so far.

Sadly for economists it’s also generally not physically possible. Physics places limits on how much resources various processes must use, and in many cases we are within a factor of 2 or 3 of those limits already. This limits the amount of decoupling, and therefore the amount of growth.

If you assume that, somehow, that will not be a problem (it will be a problem) you rapidly end up in situations which are absurd in other ways. Either critical resources become very cheap yet must be used only in small amounts, critical resources are so cheap that they can be acquired and controlled by bad people, or both. The only even slightly sane way around this would be for resources to be controlled by some central body — the state, perhaps — provided in limited quantities very cheaply, with further trade in them prohibited. That kind of economic system tends to end up with gulags.

Economists who believe these things are living in a make-believe world: economic growth cannot, in fact, continue indefinitely.

Perhaps we have a long time before this becomes a problem? Looking at environmental damage and particularly climate change caused by the waste products from consumption of fossil fuels tells us that no, we don’t. We have essentially no time at all.

On top of all this is something else that we certainly should have seen coming, even if how quickly it is arriving is unexpected: fertility rates are falling, in some cases dramatically. The total fertility rate in England and Wales fell to 1.44 in 2023 (source): Scotland’s fell to 1.3. Replacement value is about 2.1. Worldwide the TFR is still above 2, but it will almost certainly fall below it fairly soon. This means populations will start to decline which, itself, makes growth harder to maintain. Worse, it means that, unless people start living less long, there will be proportionally more old people who will contribute less to the economy and many of whom will need care.

Let’s be clear: economic growth is a good thing. I am not some hippy who wants to go back to living in an 18th century of my imagination, where somehow I will, of course, be landed gentry and quite comfortably off, and my wife won’t die in childbirth and all our children will live to be adults. Because that world didn’t exist. If growth could continue for ever that would be great, thanks. And, like the plutocrats, I grew up with science fiction: I really want to go to the Moon, or to Mars, or to Jupiter, or the stars.

But I’m also not an idiot: growth can’t continue indefinitely, and almost certainly cannot continue for very long at all. Humanity is not going to become a spacefaring species any time soon, if ever. And what says these things is physics, and if you have a difference of opinion with physics then you are wrong. Oh yes, we could discover magic new physics tomorrow — faster than light travel, and therefore also time machines, say — but do you want to bet the entire future of civilisation on that? Because I don’t.

So we have a big, urgent problem.

What are politicians doing about the problem? Almost none of them seem to understand in any real sense that there is a problem. That is perhaps unsurprising given that most politicians have educations which might have been appropriate in 1750. Those who accept that at least part of the problem exists seem to think that it can be solved by just making speeches, signing bits of paper and then quietly not doing enough, or anything, to actually solve the parts they understand exist.

And that’s what people are doing as well. Because it’s a big and really inconvenient problem: it means things will be very different in future and, whatever we do, they will likely be worse than they are now in some ways. And the shiny space-age future promised by growth is not, in fact going to happen. It’s like cancer: it’s often, somehow, easier to ignore the symptoms and pretend that you don’t have a horror growing inside you, rather than deal with the awkward truth. But it’s also like cancer: if you just avoid dealing with it then it will get worse, and then it will kill you. The best thing you can do is to talk to a doctor, immediately.

And other people have a different reaction. They see things are not going well, if dimly, but imagine that, somehow, it’s always being caused by someone else, or that these other people are lying to them about the problem. And ‘someone else’ means people with skin of a different colour, LBTQIA+ people, ‘woke’ people, liberals, the elite, women and so on. And so they elect leaders who will ‘deal with those people’ in some unspoken but always understood way.

So we get politicians who claim it’s all a hoax: it’s all a lie spread by woke liberal science gay black foreign people and we should just burn more fossil fuels and puke more pollutants into the environment. And, you know, deal with the people we don’t like.

But, I think, a lot of people have some notion of what’s probably coming. It seems at least plausible, for instance, that one reason women are choosing to have fewer children because they can see that bad things are coming and nothing significant is being done to prevent it. Of course the demagogues have a plan for that: forced birth, which is being instituted in America.

Again, let’s be clear: economic growth will stop, because physics says it must. That’s not a good thing, but it is inevitable. We have, really, three choices. We could recognise the problem and work out how we’re going to deal with it. It won’t be easy. We could pretend it doesn’t exist and just blunder as long as we can. Or we could pretend it’s all a lie spread by some imagined enemy who must be eliminated.

We are not taking the first option.

A history of the near future

I’d like to share a revelation I’ve had during my time here. It came to me when I tried to classify your species. I realized that you’re not actually mammals. Every mammal on this planet instinctively develops a natural equilibrium with their surrounding environment, but you humans do not. You move to another area, and you multiply, and you multiply, until every natural resource is consumed. The only way you can survive is to spread to another area. There is another organism on this planet that follows the same pattern. Do you know what it is? A virus. Human beings are a disease, a cancer of this planet. You are a plague, and we are the cure.

— Agent Smith

Overshoot and collapse

Despite alarmists, climate change and environmental degradation are very unlikely to make the Earth uninhabitable by humans: they will just make it a lot less habitable. If we merely stick our heads in the sand and ignore the problem and if, as things begin to fall apart, there are no ugly social effects, then what will happen is overshoot and collapse. This is, of course, what was predicted by the Club of Rome in The limits to growth in 1972. Here’s what they currently say about it:

This report […] was the first to model our planet’s interconnected systems and to make clear that if growth trends in population, industrialisation, resource use and pollution continued unchanged, we would reach and then overshoot the carrying capacity of the Earth at some point in the next one hundred years.

Some fifty years on, the call for a change in direction was more urgent than ever. The report’s modelling was remarkably accurate and prescient as the world declares the climate emergency to be real and global ecosystems to be at breaking point. Fifty years offered an excellent opportunity to look back, and forward, at the trends it examined and listen to leading international thought leaders, scientists and politicians on how we create a new critical framework for living and thriving within the limits on Planet Earth.

The limits to growth was despised by most economists, apparently because it told them an inconvenient truth. Inconvenient, perhaps, but still true.

Overshoot and collapse happens because there is hysteresis: the behaviour of the system depends on its history. A simple example is extinction: if the environment changes so some species dies out, then undoing that change doesn’t bring it back to life. The same thing is why the ‘overshoot and recover’ idea that’s now popular for carbon emissions is so stupid and dangerous: if you push temperatures beyond the point where the ice sheets start melting, say, or AMOC collapses, then that doesn’t stop when temperatures lower again. Of course what people really mean by ‘overshoot and recover’ is ‘do nothing, and let our children deal with it’: it turns out people do not care about their children’s entire futures if it is momentarily inconvenient to them.

So what happens is that economic growth continues, carbon dioxide and methane emissions are not reduced and then at some point everything falls apart: harvests fail, people die in large numbers, and modern civilisation comes to an abrupt end. Over the following hundreds of years sea-levels rise by many metres as the ice-sheets melt, drowning the remaining coastal cities, the wave of extinctions we’ve started continues and people continue to die off. Eventually, after several thousand of years and since emissions have essentially ceased, the climate stabilises and the surviving human population starts again from some level between the mediaeval and the 18th century. Perhaps they’ll have learned not to do it all again. Perhaps.

It’s not a good prospect. But it’s absurdly optimistic compared to what the history of the near future will be.

A pale horse

And I looked, and behold, a pale horse, & his name that sate on him was Death, and hell followed with him: and power was giuen vnto them, ouer the fourth part of the earth to kill with sword, & with hunger, and with death, and with the beastes of the earth.

Something I haven’t talked about is what will be the most obvious effect of climate change: parts of the world which are already hot will become dangerously so, to the extent that humans will not be able to live in them. The people who live in those parts of the world will have two choices: move somewhere else, or die. When one of two choices is ‘die’, people take the other one. There is going to be a lot of migration from these areas to more temperate areas. And when I say ‘a lot’ I mean a lot: hundreds of millions of people will have to migrate. This is already beginning: if you think the ‘small boats’ crisis that has so exercised people in the UK is a big deal, well, you ain’t seen nothing yet.

The social effects of this aren’t something climate models can talk about, but I think they’re pretty predictable. In temperate areas, the environment will already be degrading: there will be floods, droughts and other unpleasant weather events. The hope for a better future will be gone; the hope for a future which is at least not much worse than the past will be receeding.

Women will have fewer children because they realise that bringing a child into the world they see arriving is an act of cruelty. Demographic pyramids will invert, and old people will start dying as a result. Old people will resent the young, young people will resent the old.

And while this is going on a vast tide of desperate migrants will be trying to get in, because if they can’t get in, they will die. Letting those migrants in would, in fact, resolve the demographic problem. But the migrants have brown skins and it turns out that racism is anything but a solved problem in developed countries: it’s just one we suppressed for a while. There were race riots in the UK, in 2024.

People will react as they’ve always done: they’ll look for people to blame for what is happening, because it can’t be their own fault. Among those people will be recent immigrants. They’ll look for strong leaders who will give them simple answers. Not everyone will think like this, but enough people will. And, conveniently, not thinking like this will mean that you are one of the people to blame.

This is what is happening now. It’s populism: leaders who offer simple, appealing, wrong answers to complicated, unappealing problems. Keep the migrants out. Go for growth. Burn more coal. Stop all that environmental protection stuff. It’s all the fault of the liberal elite. It’s free trade. The scientists are to blame. It’s the Mexicans. It’s black people. It’s the gays. It’s trans people. It’s the Chinese. It’s the UN, the muslims, NATO, the EU. It’s always somebody else: it’s never us. And, inevitably, at some point, it will be the Jews.

And so populism is turning into something else: fascism.

Look at Germany in the 1930s: things were pretty bad after the 1914–1919 war, so people started looking for someone who would offer them simple answers, and provide people to blame. They found that person. And they found a convenient group of people who were not, of course, to blame, and they murdered them in huge numbers.

That’s what’s coming. As the environment falls apart, as growth fails, fascists will start taking over. Look around: they already are.

But the simple, appealing answers fascists offer don’t work. Once the bad people have all conveniently gone away (nobody will ask where, everybody will know where), once the walls have been built, once the coal is being mined again, once the compulsory birth laws are in place, somehow nothing will get better. The environment deteriorates, people start dying in floods, in droughts, in hurricanes and typhoons. Harvests start failing: people start dying of hunger and thirst. And the fascists will do what they always do, because it’s all they know how to do: they’ll start wars.

An interesting thing has happened. I was born just after the Cuban missile crisis. While she was pregnant, my mother was stockpiling dried milk and canned food, because she believed that a nuclear war could be imminent. She was right to believe that: it very nearly happened. Throughout the later 1960s and 1970s people were absolutely terrified of nuclear war. Perhaps the threat was seen as receding by the mid 1980s, but people were still pretty scared. And then, in the early 1990s, it all stopped: the USSR collapsed and the cold war, apparently, was over. The doomsday clock maintained by the Bulletin of the atomic scientists has never been further from midnight than it was in 1991.

You want to know when it was closest to midnight? Today, that’s when. Almost all nuclear arms limitation treaties are dead. Russia has invaded Ukraine and threatened to use nuclear weapons there. The situation in the Middle East is awful, and at least one state there has nuclear weapons. A new nuclear arms race is starting.

And yet, curiously, the fear of nuclear war hasn’t returned. People have, somehow, forgotten that this awful fate hangs over us, and not noticed that it is more likely, today, than it has ever been. That’s, as I said, interesting.

Because here’s the thing: the failure of growth together with increasingly rapid environmental collapse and migration on a huge scale is going to give us — is already giving us — fascists. And fascists, when they have dealt with the undesirables and when it turns out they have no answers, start wars. And those wars will rapidly become nuclear wars.

How near is the near future? I don’t know. I could invent a chronology but I’d be making it up. Certainly by the end of the century, and perhaps much sooner, partly depending on what happens on the 5th of November and in its aftermath: on how soon the fascists win in America.

And that is how the history of the near future ends: not with a whimper, but with a bang. If you want to know how most humans will die, watch Threads.

— Tim Bradshaw, all saints’ day, 2024

An unlikely story

2024-08-30T07:29:19Z

Or: sometimes the obvious answer isn’t the answer at all.

We have a relatively recent vehicle which, being recent, suffers from having a significant amount of software in its vitals. I just hope that the safety-critical components are less buggy than the user-facing ones¹.

It has two electronic keys. A few days ago one of them started to be difficult to use: it would sometimes fail to do anything, at other times it would unlock the vehicle which would then refuse to start, or emit panicked messages after it had started saying there was no key.

So, obviously, this was the battery dying. I ordered two batteries since the other key was probably also on its way out.

Before the new batteries arrived the other key started having similar symptoms. Which was an odd coincidence: it seemed implausible that two batteries with a life of years would choose the same week to run down. But the second key was less sickly than the first, so perhaps it was reasonable.

Before the new batteries arrived I found a stash of unused ones I had bought some time ago. So, of course, I replaced the battery in the first key. Which didn’t fix it: now it could unlock the vehicle but not start it or lock it. So, perhaps that key was failing?

I replaced the battery in the other key. Which did not fix it.

So, OK, this leaves one obvious candidate: the vehicle. Aside from the general crapness of its software, it’s also had a history of electronics-related failures so, obviously, this was just one more expensive electronic tantrum. There was nothing wrong with the keys and never had been: the vehicle was just losing its mind, again. That’s why both keys had ‘failed’ in the same week: they hadn’t. And the vehicle was going to need yet more expensive fixing which probably would amount to a firmware update.

And then the new batteries arrived. And because anything would be cheaper than getting the vehicle fixed I tried them. And they worked, in both keys. What had actually happened is that, yes, both batteries had died within a day or so of each other after half a decade, and the unused (in their blister packs) ones I had used to replace them were no good or, perhaps, too old².

Sometimes the obvious answer isn’t the answer at all.

But, given this horror story, I’m not very sure that the safety-critical code is that good, either. ↩
They were not too old. ↩

The paperclip maximizers

2022-10-18T09:05:49Z

Or, the calls are coming from inside the house.

The paperclip maximizer

The paperclip maximizer, probably first described by Nick Bostrom, is

a superintelligence whose sole goal is something completely arbitrary, such as to manufacture as many paperclips as possible, and who would resist with all its might any attempt to alter this goal. […] with the consequence that it starts transforming first all of earth and then increasing portions of space into paperclip manufacturing facilities.

It is often used as a parable about the dangers of AI research, and particularly of creating AIs which are smarter than us.

But it’s obviously a fairly silly idea: how could anything which is really intelligent be dedicated to such a meaningless, useless goal, to a goal which, if pursued relentlessly and to the exclusion of all else, will certainly result in its own extinction?

A sufficiency of paperclips

The production of paperclips is not, in fact, a meaningless goal: paperclips are quite useful. The problem happens when the the production of paperclips in numbers greater than could ever be useful becomes the only goal.

But hold on. The production of wealth is not, in fact, a meaningless goal: wealth is quite useful. The problem happens when the production of wealth in amounts greater than could ever be useful becomes the only goal.

How rich do you have to be before ‘being richer’ becomes meaningless? I don’t know, but there is, quite clearly, a level at which this happens. And a large number of extremely wealthy people are far beyond this level. And yet they strive unceasingly to accumulate more ~~paperclips~~money, even though doing so has long lost any meaning for them or for anyone, and even though doing so is having catastrophic consequences for the future of us all.

Even more absurdly, we are all told by apparently well-educated and quite respectable people that endless economic growth is the cure for all our ills, even though endless economic growth means that resource requirements must grow exponentially with time, which is not physically possible. The pursuit of endless economic growth is merely the pursuit of paperclips in fancy dress: it will lead only to catastrophe.

Do you want to know what living in a world of uncontrolled paperclip maximizers looks like? Look around: the paperclip maximizers are us.

Package-local nicknames

2022-10-14T09:26:31Z

What follows is an opinion. Do not under any circumstances read it. Other opinions are available (but wrong).

Package-local nicknames are an abomination. They should be burned with nuclear fire, and their ashes launched into space on a trajectory which will leave the Solar System.

The only reason why package-local nicknames matter is if you are writing a lot of code with lots of package-qualified names in it. If you are doing that then you are writing code which is hard to read: the names in your code are longer than they need to be and the first several characters of them are package name noise (people read, broadly from left to right). Imagine me:a la:version ge:of oe:English oe:where la:people wrote like that: it’s just horrible. If you are writing code which is hard to read you are writing bad code.

Instead you should do the work to construct a namespace in which the words you intend to use are directly present. This means constructing suitable packages: the files containing the package definitions are then almost the only place where package names occur, and are a minute fraction of the total code. Doing this is a good practice in itself because the package definition file is then a place which describes just what names your code needs, from where, and what names it provides. Things like conduit packages (shameless self-promotion) can help with this, which is why I wrote them: being able to say ‘this package exports the combination of the exports of these packages, except …’ or ‘this package exports just the following symbols from these packages’ in an explicit way is very useful.

If you are now rehearsing a litany of things that can go wrong with this approach in rare cases¹, please don’t: this is not my first rodeo and, trust me, I know about these cases. Occasionally, the CL package system can make it hard or impossible to construct the namespace you need, with the key term here being being occasionally: people who give up because something is occasionally hard or impossible have what Erik Naggum famously called ‘one-bit brains’²: the answer is to get more bits for your brain.

Once you write code like this then the only place package-local nicknames can matter is, perhaps, the package definition file. And the only reason they can matter there is because people think that picking a name like ‘XML’ or ‘RPC’ or ‘SQL’ for their packages is a good idea. When people in the programming section of my hollowed-out-volcano lair do this they are … well, I will not say, but my sharks are well-fed and those things on spikes surrounding the crater are indeed their heads.

People should use long, unique names for packages. Java, astonishingly, got this right: use domains in big-endian order (org.tfeb.conduit-packages, org.tfeb.hax.metatronic). Do not use short nicknames. Never use names without at least one dot, which should be reserved for implementations and perhaps KMP-style substandards. Names will now not clash. Names will be longer and require more typing, but this will not matter because the only place package names are referred to are in package definition files and in in-package forms, which are a minute fraction of your code.

I have no idea where or when the awful plague of using package-qualified names in code arose: it’s not something people used to do, but it seems to happen really a lot now. I think it may be because people also tend to do this in Python and other dotty languages, although, significantly, in Python you never actually need to do this if you bother, once again, to actually go to the work of constructing the namespace you want: rather than the awful

import sys

... sys.argv ...

...

sys.exit(...)

you can simply say

from sys import argv, exit

... argv ...

exit(...)

and now the very top of your module lets anyone reading it know exactly what functionality you are importing and from where it comes.

It may also be because the whole constructing namespaces thing is a bit hard. Yes, it is indeed a bit hard, but designing programs, of which it is a small but critical part, is a bit hard.

OK, enough.

If, after reading the above, you think you should mail me about how wrong it all is and explain some detail of the CL package system to me: don’t, I do not want to hear from you. Really, I don’t.

in particular, if your argument is that someone has used, for instance, the name set in some package to mean, for instance, a set in the sense it is used in maths, and that this clashes with cl:set and perhaps some other packages, don’t. If you are writing a program and you think, ‘I know, I’ll use a symbol with the same name as a symbol exported from CL to mean something else’ in a context where users of your code also might want to use the symbol exported by CL (which in the case of cl:set is ‘almost never’, of course), then my shark pool is just over here: please throw yourself in. ↩
Curiously, I think that quote was about Scheme, which I am sure Erik hated. But, for instance, Racket’s module system lets you do just the things which are hard in the package system: renaming things on import, for instance. ↩

Macros (from Zyni)

2022-08-27T10:12:33Z

It is the business of the future to be dangerous; and it is among the merits of science that it equips the future for its duties. — Alfred Whitehead

Once upon a time, long ago in a world far away, Lisp had many features which other languages did not have. Automatic storage management, dynamic typing, an interactive environment, lists, symbols … and macros, which allow you to seamlessly extend the language you have into the language you want and need.

But that was long long ago in a world far away where giants roamed the earth, trolls lurked under every bridge and, they say, gods yet lived on certain distant mountains.

Today, and in this world, many many languages have automatic storage management, are dynamically typed, have symbols, lists, interactive environments, and so and so and so. More of these languages arise from the thick, evil-smelling sludge that coats every surface each day: hundreds, if not thousands of them, like flies breeding on bad meat which must be swatted before they lay their eggs on your eyes.

Lisp, today and in this world not another, has exactly one feature which still distinguishes it from the endless buzz of these insect languages. That feature is seamless language extension by macros.

So yes, macros are dangerous, and they are hard and they are frightening. They are dangerous and hard and frightening because all powerful magic is dangerous and hard and frightening. They are dangerous because they are a thing which has escaped here from the future and it is the business of the future to be dangerous.

If macros are too dangerous, too hard and too frightening for you, do not use Lisp because macros are what Lisp is about.

This originated as a comment by my friend Zyni: it is used with her permission.

The network is the computer

2021-08-04T10:31:35Z

Rishi Sunak has told people that working from home may hurt their career. Sunak, like many conservatives, is frightened, not only of the future, but of the present.

I am quite quite sure that Sunak is not acting in his own or his rich friends’ narrow self-interest here, as they stare horrified at the commercial property portfolios which have made them as rich as they are and wonder what they’re now worth. Quite sure.

So since that can’t be the case (I mean, it couldn’t be, could it?) what’s happening is that Sunak and others like him are simply acting the way you would expect them to behave. They are, well, conservative: they like things to stay the same. Often they are stuck in the past and unwilling to accept change. That hasn’t often been the case for the johnsonite tory party which is not, in fact, a conservative party at all but a quasi-fascist insurgency, but it is the case here.

Because what finally happened, in late March 2020, was the internet. If you are old enough you may remember what the internet was meant to give us. Before the internet, you had to go to some special anointed place where the computers were, with their vast ranks of drums and endlessly-spooling tape drives, served by people in white coats in rooms with glass walls from behind which the onlookers would stare, amazed at the computational resources being deployed for who-knows-what purpose. When the internet came the computers in their cathedrals would dissolve into the landscape, and suddenly they would be everywhere. No longer would you have to go to a special building where the terminals were: you could be anywhere. You could be in a café, in the park, on the beach, at home: anywhere. And you would not have to spend a fifth or more of your waking life in various tin boxes, and all your free time exhausted. Life would be better for almost everyone: everyone except those who owned the rotting, empty palaces full of rusting, unused terminals.

Well, the internet didn’t happen when we thought it would: instead it got parasitized by various soul-eating entities with names like ‘google’ and ‘facebook’ which made it not only almost useless, but usually actively harmful: suddenly, not only did you have to spend three hours in a tin can, you somehow had to find another three hours to spend feeding the parasites. And people forgot what it was meant to have done.

And then, in March 2020, another parasite came. And, because we all now had to stay at home to escape from this new parasite, finally, the internet happened. Finally, many of us are free. At first, of necessity, we all worked from home, but as the new parasite fades we will, finally, be able to work from anywhere: from a coffee shop, from a museum, from an art gallery, from a park; even, perhaps, sometimes from home. But not from some vast factory full of terminals.

And Sunak can’t understand this. For him it will always be the late 20th century: for him that brief period of history that started in the late 19th century and is all he has ever known must last for ever, because for him no change is possible. But change has, finally, come, and he cannot stop it.

At last, the future has come and we are free.

An earlier version of this was a comment on The Register.

How the backtrace was conquered

2021-03-26T11:37:22Z

Once upon a time, when the world was younger, a young and rather foolish physics student used to debug his FORTRAN programs using printed backtraces.

And I do mean printed backtraces: when the machine crashed the chain printer attached to it would vomit out many sheets of paper which had procedure names and line numbers on them. And, after restarting the machine so the next user could make it crash in their turn he would take this printout and take his printout of his program and compare the line numbers: looking at the code, trying to work out what had gone wrong and marking corrections in pencil. He spent many hours late at night in this way.

Later on, this same student (now a maths student) discovered a wonderful thing: a programming language called Lisp in which you could write programs to solve complex algebra problems which were of interest in his field. And although, in theory, if you had the kind of computer which maths departments could not afford, Lisp was an interactive language, this was not true in practice if all you had was the kind of computer that was all a maths department could afford. So things went on much as before: he would make some changes to his program, set up the equations it was going to try to solve, and then, late at night when there were no other users to inconvenience, set it off running. In the morning there might occasionally be a solution, and even more occasionally a solution which was useful. But more often there would be only the corpse of the program in the form of an elaborate backtrace after it had been mortally wounded by some fierce bug (error handling was a thing not yet thought of, at leasy by the student). This time, though, the backtrace would be in a log file from the run.

And the student made another discovery: there was a certain text editing environment used by some far-off people who had access to much bigger and better computers, and this editing environment purported to support Lisp programming rather well: certainly better than the rudimentary editor he used then. And he managed to get a copy of this environment (legend has it it was version 17.64) on a tape from someone, and he managed to make it run, just, on the maths department’s machine. And he taught it enough about the Lisp dialect he was using that it was indeed helpful, if often annoying to other users as it took rather a lot of the capacity of the machine to support it. And everything was a little better.

And this text editing system came with a rather wonderful tool: a program whose name may have been ‘tags’ which would, for the languages it understood, make a file which mapped between definition names and their locations in the filesystem. And he modified this tags program to understand the dialect of Lisp he was using as well. Very wonderfully, the system would also cope with the case where the definition had moved, which it almost always had, and which made things like line and column numbers so brittle and useless (source control might have been invented by then, but the student knew nothing of that). This, of course, was the one of the primitive ancestors of the automatic systems which will find definitions of symbols that any reputable editor, and even some that are perhaps a little less than reputable, now has.

And now, when he came in in the morning to find a new backtrace from the previous night’s run, he would edit this backtrace in the editing system and find interesting lines in it, at which he would type the very wonderful ‘meta-dot’ or, as he knew it (not being blessed with a keyboard with a meta key), ‘escape dot’ command. And the disk light would come on for a little, and then he would be looking at the definition he was interested in.

Thus was the backtrace conquered. And from that day to this it has never dared raise its head again in polite company, but instead lurks, unheeded except by the few who now remember it, in the darker corners of the system. As for the student, well, no-one now remembers him at all.

Backup retention

2021-01-02T17:33:01Z

Or: should you keep that tape?

There is an interesting curve of backup retention.

Initially, you should definitely keep them because they’re, well, backups that you might need to restore.

Then there comes a time where you should almost certainly not keep them because they’re too old to be useful as backups.

If they survive that they become, accidentally, archives: perhaps that tape sitting in some box has the only remaining copy of whatever-it-is. So don’t erase it.

At the point where nothing will read the tape any more, well, whatever was on it is effectively gone now, so throw it away.

At some point after that, one or both of two things happen: people become willing and able to do seriously heroic things to read really old media which might have the last remaining copy of something on it and/or the media itself becomes rare enough that it’s now a historical artifact worth preserving. The second thing can’t happen unless enough copies of the media get thrown away in earlier phases of the process: I don’t think minidiscs would be interesting historical artifacts (yet), but if I still had a Fuji Eagle I would definitely not throw it away.

Later still it becomes possible to print, cheaply, replicas accurate at the atomic level of the thing, at which point its value should drop to the cost of making another clone, but in fact people start spending huge amounts of effort authenticating the original copy of the object which is held to be somehow ineffably different to all the perfect clones. At some point, people lose track: no-one now knows which the original is any more, and since there is no physical distinction no-one ever will again. The people who have paid to have their copy authenticated as the original now spend much of their time arranging to have the other people who have done that assassinated.

I forget which film this is.

Unhappy far-off things

2020-12-23T11:42:20Z

It is the Abomination of Desolation, not seen by prophecy far off in some fabulous future, nor remembered from terrible ages by the aid of papyrus and stone, but fallen on our own century, on the homes of folk like ourselves: common things that we knew are become the relics of bygone days. It is our own time that has ended in blood and broken bricks.

It can’t happen here, can it? Of course it can not: this is something that happens to other people in lesser countries far away. Something we read about in newspapers or watch in enchanted horror on the news. We watch as some unhappy country eats itself alive, vomiting forth a spray of refugees who, somehow but inevitably, we will not be able to accept here, though they be ever so deserving. And of course these distant tragedies are never our fault, not even slightly.

No, these tragedies can not happen here: we are too clever, too well-educated, too English. We have too much to lose so it will not be allowed. And if it were to happen here it would if course not be our fault: it would most certainly be the doings of inferior foreign people who wish us ill. We are, after all, simply better fellows than those unhappy far-off people.

Quote from Unhappy far-off things by Lord Dunsany.

An Englishman's camera bag

2020-12-18T11:06:47Z

Or: you can’t buy history, however much money you have.

Billingham bags are beautifully-made leather-and-canvas things, which when new probably smell of nothing and when later cleaned might smell faintly of leather and old sails. Both the canvas and the leather will wear prettily over the decades. You could imagine leaving such a bag to your younger son in your will (your oldest son would, of course, get the house on Long Island, along, perhaps, with your mistress)¹.

Billingham bags are what Americans who believe themselves cultured think English gentlemen might use: they are, in fact, New English. Of course, no Englishman would be seen dead with a Billingham bag, let alone in polite company, any more than they would be seen about with a Leica (“the Rolls-Royce of cameras”: ostentatious, vulgar, probably available in pink).

An Englishman’s camera bag is nothing like a Billingham. Indeed, it is not very much like a camera bag. It is made of a material which might once have been waxed cotton but is now mostly grease and patches. It smells of mould, ferrets, fixer and old blood — it is usually better not to ask where the blood came from. It is not, of course, padded: the owner will improvise padding from folded up broadsheet newspapers, none later than the 60s. It may have a strap, and this may not be made entirely of string. In one of the outer pockets there will be a quarter-plate darkslide for a model of camera not made since before the great war. In others there will be OS maps of Afghanistan, passports (all expired) several glass syringes and Kendal mint cake. In the bottom of the bag will be a dense layer of detritus including feathers, the mummified remains of a mouse, some filters in imperial sizes, a Watkins Bee meter possibly in working order, much string, a remote release apparently partly eaten, bits of film and what might be the remains of the original strap. It is better not to investigate this layer too closely.

The Englishman’s camera bag is not left to his children. Rather, they discover it years after his death in a cupboard, slightly mouldier than it once was and apparently having served as a home to several generations of small birds. No one particularly wants it, but since it is, somehow, useful (certainly more useful than something made of leather, canvas and salt air), it is adopted and so persists.

Note: this article is intentionally using sexist language and ideas which are, frankly, offensive. Its entire purpose is to satirise both a certain sort of photographer (always male, usually American) and a certain sort of English person (again, always male, and who likes his martinis shaken and not stirred). I am not either of these sorts of people and I certainly do not support the attitudes in this article. I also did not inherit my father’s camera bag, although I did inherit his Curta. ↩

I remember Apollo

2019-07-16T20:15:08Z

All serious historians agree that the Apollo programme of the 1960s and early 1970s was the highpoint of western civilisation.

There were, of course significant achievements after Apollo — Voyager, the Hubble space telescope and its successors, images of black holes, the development of economic fusion power even, although it was too late. And there was very considerable social progress after Apollo: for nearly 45 years things improved steadily. It is easy to forget this latter fact given the events that came later: the incarceration without trial, forced labour, mass rape and eventual butchery of immigrants, minority groups, people with ‘incompatible’ sexual orientations, journalists, liberals and others who inconvenienced those in power in the mid 2020s now completely overshadows the progress that was made ten years earlier.

The rise of the oligarchs and dictators, with their systematic suppression of the press and all divergent opinion, encouragement of mob rule, stupidity, xenophobia and science denial in the second and third decades of the 21st century was the beginning of the end.

The failed expeditions to Mars in 2024–2025 were both farce and tragedy. Donald Trump, still claiming democratic legitimacy despite the unequivocal results of the 2020 elections, was by this time in the final stages of senile decay: never more than the shell of a human, he was by then no more than a fulminating husk under the direct control of his Russian masters. Musk, a deeply flawed man, comes out as the unlikely hero of the affair: defending the choice of black and female astronauts against Trump’s tirades and demands and, when the outcome of the mission was beyond doubt, volunteering himself. The heroism of the astronauts, knowing they faced, at best, slow death by radiation poisoning on Mars, can not be overstated. In the event, of course, they did not get that far: the live broadcast of the terrible end of the second mission, with the doomed astronauts’ condemnation of the programme and Trump even as their oxygen leaked away, ensured there would be no more although the US was by then losing the technical ability in any case. Musk’s fate remains unknown: it is assumed he was murdered by members of Trump’s family in revenge for his ‘sabotaging’ of the missions.

The two nuclear wars of 2032 (US-China) and 2035 (Russia-China-UK), while limited, killed well over half a billion people. Climate change (denied, of course, by the oligarchs but well-known to be an existential threat by the turn of the millennium) did the rest: the harvest failures of 2040 killed nearly 150 million people in North America alone and marked the effective end of the US which had already been weakened by the war with China and a series of preceding wars (the US won no war it fought after 1945): after 2040 there were never less than two competing presidents claiming authority over what had been the US, and in 2053 there were, briefly, seven.

Reliable information is increasingly scarce after 2055. The Kessler event of 2032–2033, triggered by the intentional destruction of satellites by the US in the US-China war, destroyed essentially all existing satellites and made space inaccessible to humans, possibly for the next few centuries. Planet-wide Earth-based communication systems had been catastrophically damaged in the two wars, and finally collapsed in 2055. So information after 2055 is inevitably somewhat speculative: we simply do not know how many survivors there are in the UK and what their condition is, for instance.

By 2060 the population of the former US was estimated at under ten million, of which no more than a few tens of thousands had access to electricity. Those numbers will be lower now. The UK, long in decline, and latterly little more than vassal state of the US, itself effectively a dictatorship between 2020 and 2040, also essentially ceased to exist in the 2035 war: the estimated surviving population there may now be as few as tens of thousands, mostly in Scotland. The northern areas of continental Europe are still relatively benign, but Italy, Spain, Greece, much of southern France and many other countries have been lost to climate change.

Few people are now alive who were alive during the Apollo programme, and fewer still who have any memory of it. Soon there will be no-one alive who remembers it.

But we must remember Apollo: we must remember that a great nation could devote itself to a mission of exploration, not war, and could thus achieve great things, whatever came later. We must remember that this is possible, that hatred, lies and division spread by people with small minds are not the only way. We must remember that, once, there was a project where they could truly say

that America’s challenge of today has forged man’s destiny of tomorrow. And, as we leave the Moon at Taurus-Littrow, we leave as we came and, God willing, as we shall return, with peace and hope for all mankind. “Godspeed the crew of Apollo 17.”

I remember Apollo.

Translated from the Japanese, 20690716

The end of the world

2014-12-31T17:03:07Z

Investment bankers are often called ‘sharks’ and this is, in fact, a good description. There is nothing wrong with sharks: they are beautiful animals designed by billions of years of natural selection to do one thing extremely well. You can not expect a shark to be other than a shark: rather you must understand how sharks behave and arrange matters so as not to be eaten. Governments can do this for banks: they did it in 1933, after all, and it served us well for nearly 70 years. However governments entirely failed to do this after the events of 2008 for reasons of stupidity and corruption.

In 2016 their failure to act came home to roost: the housing bubble in the UK collapsed causing a cascade failure of investment and retail banks. Initially this was confined to the UK but it rapidly spread to the US. The retail banking system held together for a while after its forced nationalisation, but decades of IT mismanagement combined with the exodus of staff, mostly to China and South America, led to its final collapse in early 2017. Starvation began in the US in February: in the UK food shortages, already serious, became acute in May of that year. After May it becomes hard to disentangle events: we know that the US launched a strike against the Russian federation in June to acquire grain, and that there was a limited exchange of nuclear weapons. The most serious result of this was the effective destruction of the internet and telephone networks as a result of EMP and the resulting loss of almost all communication and reliable records. The US also launched an attack on Canada, once again for grain, to which the UK responded, ostensibly to defend a Commonwealth country but probably in fact to secure Canadian grain stocks for itself. The resulting nuclear exchange destroyed London and most of the US east coast cities not destroyed by the earlier US-Russian exchange. Much more seriously the weather effects from these two exchanges of weapons caused the near-complete failure of the harvest in Canada and the northern parts of the US in 2017. Late in 2017 there was an attempted invasion of the UK and France by remaining US armed forces: this was repulsed with a further exchange of nuclear weapons. This is often known as ‘the second American war’ and also as ‘the fall of France’: before this time France, as the only country with an energy strategy not dependent on oil (which had become scarce after the Russians nuked most of the Middle East in the earlier Russian-US war), was relatively stable, but the US specifically targeted French nuclear plants in the second war, leading to the failure of French infrastructure. By the end of 2017 over two hundred million US citizens had starved and there was no effective government in the US, Canada, the UK, France, or Russia.

Things got a lot worse later, of course.

Playing cards with the Devil

2014-12-30T13:05:55Z

You are playing cards with the Devil, the prize being your soul.

The game is very simple:

the Devil deals two cards face down;
you both turn over the cards — if they are the same colour then you win, if they are different colours the Devil wins.

You play six times, and the Devil wins every time. He has obviously stacked the deck.

You suggest to the Devil that you should play the opposite game: if the cards are different colours you win. He agrees, and you play six times: he wins every time. Obviously He either saw this coming or He has changed the deck while you were not looking.

You suggest a third game:

the Devil will deal two cards, face down;
after the cards are dealt you will choose which game to play — the one where you win if they are the same, or the one where you win if they are different;
the cards are turned over.

Much to your surprise he agrees to play once more, and you play six times. The Devil wins every time.

Fragments: Posts tagged 'stories'

Structures of arrays

The second system

More bad decisions

Things I already knew

The light at the end of the tunnel

The kingdom of lost hope

Too big to fail

Forbidden words

Sleepwalking into armageddon

Scenes from the end times

Eat the old

The lost cause of the Lisp machines

History

‘It was the development environment’

‘They were much faster than anything else’

‘The hardware was user-microcodable, you see’

‘They were Lisp all the way down’

‘Good Lisp compilers are too hard to write for stock hardware’

‘They had wonderful keyboards’

Bored now

Mu

The modern real programmer

A history of the near future

Contents

Growth, the sacred cow

What growth means

If growth is not constant

Growth, resources and waste products

Why growth matters

Summary

Finiteness

The future we imagine

Some inconvenient truths

What is going on?

The future we’re going to have

Are we going to have a problem here?

The magic goes away

Energy

Energy is not the only fruit

Yes, we are going to have a problem here

Faith, fantasy and Economics

Efficiency limits: economics meets physics and loses

Escaping into a pretend world

Decoupling: economics meets itself and loses

The elephant

How are we doing?

What this looks like

The teeming billions

Falling fertility

Falling fertility, rising mortality

The consequences

As above, so below

Shorter

A history of the near future

Overshoot and collapse

A pale horse

An unlikely story

The paperclip maximizers

The paperclip maximizer

A sufficiency of paperclips

Package-local nicknames

Macros (from Zyni)

The network is the computer

How the backtrace was conquered

Backup retention

Unhappy far-off things

An Englishman's camera bag

I remember Apollo

The end of the world

Playing cards with the Devil