The Developer Outreach team is happy to announce that we will be migrating the Tech Blog into Diff. This move will allow us to provide better support and more visibility for the incredible work of the technical community. Diff is the community news and event blog supported by the Movement Communications team. Diff sees about 20,000 visits a month and has 1,200 email subscribers. 

• What will happen to the Tech Blog content?
  • All Tech Blog posts will be accessible on Diff, clearly tagged with “techblog”. Old links will automatically redirect to their new location. New posts with a technical focus will be tagged with “techblog” so they will be easily discoverable.You’ll be able to find all techblog posts – old and new – on the landing page at https://diff.wikimedia.org/techblog 
• When is this happening?
  • The migration should be complete in April 2026.
• How do I submit a blog post with a technical focus?
  • For now, please hold your post until we complete the migration.
  • After the migration is done: The process remains the same. For WMF staff, talk to your manager about your interest in writing a blog post so they are not surprised when you ask them to approve it once it is written. For folks outside WMF, if you are part of a team or other larger organization, be sure they are aware and approve. Then, see the Diff submission process and select the category “Technology” and the tag “techblog” when writing your draft. After you submit, the Developer Outreach team will review your draft. When it’s ready to go, we will schedule your post to be published.

We’re excited for the Tech Blog to evolve and thank the Movement Communications team for helping us make this possible!

In the United States, we are losing our fondness for cash. As in many other countries, cards and other types of electronic payments now dominate everyday commerce. To some, this is a loss. Cash represented a certain freedom from intermediation, a comforting simplicity that you just don't get from Visa. It's funny to consider, then, how cash is in fact quite amenable to automation. Even Benjamin Franklin's face on a piece of paper can feel like a mere proxy for a database transaction. How different is cash itself from "e-cash", when it starts and ends its lifecycle through automation?

Increasing automation of cash reflects the changing nature of banking: decades ago, a consumer might have interacted with banking primarily through a "passbook" savings account, where transactions were so infrequent that the bank recorded them directly in the patron's copy of the passbook. Over the years, increasing travel and nationwide communications led to the ubiquitous use of inter-bank money transfers, mostly in the form of the check. The accounts that checks typically drew on—checking accounts—were made for convenience and ease of access. You might deposit your entire paycheck into an account—it might even be sent there automatically—and then when you needed a little walking around money, you would withdraw cash by the assistance of a teller. By the time I was a banked consumer, even the teller was mostly gone. Today, we get our cash from machines so that it can be deposited into other machines.

Cash handling is fraught with peril. Bills are fairly small and easy to hide, and yet quite valuable. Automation in the banking world first focused on solving this problem, of reliable and secure cash handling within the bank branch. The primary measure against theft by insiders was that the theft would be discovered, as a result of the careful bookkeeping that typifies banks. But, well, that bookkeeping was surprisingly labor-intensive in even the bank of the 1950s.

Histories of the ATM usually focus on just that: the ATM. It's an interesting story, but one that I haven't been particularly inclined to cover due to the lack of a compelling angle. Let's try IBM. IBM is such an important, famous player in business automation that it forms something of a synecdoche for the larger industry. Even so, in the world of bank cash handling, IBM's efforts ultimately failed... a surprising outcome, given their dominance in the machines that actually did the accounting.

In this article, we'll examine the history of ATMs—by IBM. IBM was just one of the players in the ATM industry and, by its maturity, not even one of the more important ones. But the company has a legacy of banking products that put the ATM in a more interesting context, and despite lackluster adoption of later IBM models, their efforts were still influential enough that later ATMs inherited some of IBM's signature design concepts. I mean that more literally than you might think. But first, we have to understand where ATMs came from. We'll start with branch banking.

When you open a bank account, you typically do so at a "branch," one of many physical locations that a national bank maintains. Let us imagine that you are opening an account at your local branch of a major bank sometime around 1930; whether before or after that year's bank run is up to you. Regardless of the turbulent economic times, the branch became responsible for tracking the balance of your account. When you deposit money, a teller writes up a slip. When you come back and withdraw money, a different teller writes up a different slip. At the end of each business day, all of these slips (which basically constitute a journal in accounting terminology) have to be rounded up by the back office and posted to the ledger for your account, which was naturally kept as a card in a big binder.

A perfectly practicable 1930s technology, but you can already see the downsides. Imagine that you appear at a different branch to withdraw money from your account. Fortunately, this was not very common at the time, and you would be more likely to use other means of moving money in most scenarios. Still, the bank tries to accommodate. The branch at which you have appeared can dispense cash, write a slip, and then send it to the correct branch for posting... but they also need to post it to their own ledger that tracks transactions for foreign accounts, since they need to be able to reconcile where their cash went. And that ignores the whole issue of who you are, whether or not you even have an account at another branch, and whether or not you have enough money to cover the withdrawal. Those are problems that, mercifully, could mostly be sorted out with a phone call to your home branch.

Bank branches, being branches, do not exist in isolation. The bank also has a headquarters, which tracks the finances of its various branches—both to know the bank's overall financial posture (critical considering how banks fail), and to provide controls against insider theft. Yes, that means that each of the branch banks had to produce various reports and ledger copies and then send them by courier to the bank headquarters, where an army of clerks in yet another back office did yet another round of arithmetic to produce the bank's overall ledgers.

As the United States entered World War II, an expanding economy, rapid industrial buildup, and a huge increase in national mobility (brought on by things like the railroads and highways) caused all of these tasks to occur on larger and larger scales. Major banks expanded into a tiered system, in which branches reported their transactions to "regional centers" for reconciliation and further reporting up to headquarters. The largest banks turned to unit record equipment or "business machines," arguably the first form of business computing: punched card machines that did not evaluate programs, but sorted and summed.

Simple punched card equipment gave way to more advanced automation, innovations like the "posting machine." These did exactly what they promised: given a stack of punched cards encoding transactions, they produced a ledger with accurately computed sums. Specialized posting machines were made for industries ranging from hospitality (posting room service and dining charges to room folios) to every part of finance, and might be built custom to the business process of a large customer.

If tellers punched transactions into cards, the bank could come much closer to automation by shipping the cards around for processing at each office. But then, if transactions are logged in a machine readable format, and then processed by machines, do we really need to courier them to rooms full of clerks?

Well, yes, because that was the state of technology in the 1930s. But it would not stay that way for long.

In 1950, Bank of America approached SRI about the feasibility of an automated check processing system. Use of checks was rapidly increasing, as were total account holders, and the resulting increase in inter-branch transactions was clearly overextending BoA's workforce—to such an extent that some branches were curtailing their business hours to make more time for daily closing. By 1950, computer technology had advanced to such a state that it was obviously possible to automate this activity, but it still represented one of the most ambitious efforts in business computing to date.

BoA wanted a system that would not only automate the posting of transactions prepared by tellers, but actually automate the handling of the checks themselves. SRI and, later, their chosen manufacturing partner General Electric ran a multi-year R&D campaign on automated check handling that ultimately led to the design of the checks that we use today: preprinted slips with account holder information, and account number, already in place. And, most importantly, certain key fields (like account number and check number) represented in a newly developed machine-readable format called "MICR" for magnetic ink character recognition. This format remains in use today, to the extent that checks remain in use, although as a practical matter MICR has given way to the more familiar OCR (aided greatly by the constrained and standardized MICR character set).

The machine that came out of this initiative was called ERMA, the Electronic Recording Machine, Accounting. I will no doubt one day devote a full article to ERMA, as it holds a key position in the history of business computing while also managing to not have much of a progeny due to General Electric's failure to become a serious contender in the computer industry. ERMA did not lead to a whole line of large-scale "ERM" business systems as GE had hoped, but it did firmly establish the role of the computer in accounting, automate parts of the bookkeeping through almost the entirety of what would become the nation's largest bank, and inspire generations of products from other computer manufacturers.

The first ERMA system went into use in 1959. While IBM was the leader in unit record equipment and very familiar to the banking industry, it took a few years for Big Blue to bring their own version. Still, IBM had their own legacy to build on, including complex electromechanical machines that performed some of the tasks that ERMA was taking over. Since the 1930s, IBM had produced a line of check processing or "proofing" machines. These didn't exactly "automate" check handling, but they did allow a single operator to handle a lot of documents.

The IBM 801, 802, and 803 line of check proofers used what were fundamentally unit record techniques—keypunch, sorting bins, mechanical totalizers—to present checks one at a time in front of the operator, who read information like the amount, account number, and check number off of the paper slip and entered it on a keypad. The machine then whisked the check away, printing the keyed data (and reference numbers for auditing) on the back of the check, stamped an endorsement, added the check's amounts to the branch's daily totals (including subtotals by document type), and deposited the check in an appropriate sorter bin to be couriered to the drawer's bank. While all this happened, the machines also printed the keyed check information and totals onto paper tapes.

By the early 1960s, with ERMA on the scene, IBM started to catch up. Subsequent check processing systems gained support for MICR, eliminating much (sometimes all!) of the operator's keying. Since the check proofing machines could also handle deposit slips, a branch that generated MICR-marked deposit slips could eliminate most of the human touchpoints involved in routine banking. A typical branch bank setup might involve an IBM 1210 document reader/sorter machine connected by serial channel to an IBM 1401 computer. This system behaved much like the older check proofers, reading documents, logging them, and calculating totals. But it was now all under computer control, with the flexibility and complexity that entails.

One of these setups could process almost a thousand checks a minute with a little help from an operator, and adoption of electronic technology at other stages made clerks lives easier. For example, IBM's mid-1960s equipment introduced solid-state memory. The IBM 1260 was used for adding machine-readable MICR data to documents that didn't already have it. Through an innovation that we would now call a trivial buffer, the 1260's operator could key in the numbers from the next document while the printer was still working on the previous.

Along with improvements in branch bank equipment came a new line of "high-speed" systems. In a previous career, I worked at a Federal Reserve bank, where "high-speed" was used as the name of a department in the basement vault. There, huge machines processed currency to pick out bad bills. This use of "high-speed" seems to date to an IBM collaboration with the Federal Reserve to build machines for central clearinghouses, handling checks by the tens of thousands. By the time I found myself in central banking, the use of "high-speed" machinery for checks was a thing of the past—"digital substitute" documents or image-based clearing having completely replaced physical handling of paper checks. Still, the "high-speed" staff labored on in their ballistic glass cages, tending to the green paper slips that the institution still dispenses by the millions.

One of the interesting things about the ATM is when, exactly, it pops up in the history of computers. We are, right now, in the 1960s. The credit card is in its nascent stages, MasterCard's predecessor pops up in 1966 to compete with Bank of America's own partially ERMA-powered charge card offering. With computer systems maintaining account sums, and document processing machines communicating with bookkeeping computers in real-time, it would seem that we are on the very cusp of online transaction authorization, which must be the fundamental key to the ATM. ATMs hand out cash, and one thing we all know about cash is that, once you give yours to someone else you are very unlikely to get it back. ATMs, therefore, must not dispense cash unless they can confirm that the account holder is "good for it." Otherwise the obvious fraud opportunity would easily wipe out the benefits.

So, what do you do? It seems obvious, right? You connect the ATM to the bookkeeping computer so it can check account balances before dispensing cash. Simple enough.

But that's not actually how the ATM evolved, not at all. There are plenty of reasons. Computers were very expensive so banks centralized functions and not all branches had one. Long-distance computer communication links were very expensive as well, and still, in general, an unproven technology. Besides, the computer systems used by banks were fundamentally batch-mode machines, and it was difficult to see how you would shove an ATM's random interruptions into the programming model.

Instead, the first ATMs were token-based. Much like an NYC commuter of the era could convert cash into a subway token, the first ATMs were machines that converted tokens into cash. You had to have a token—and to get one, you appeared at a teller during business hours, who essentially dispensed the token as if it were a routine cash withdrawal.

It seems a little wacky to modern sensibilities, but keep in mind that this was the era of the traveler's check. A lot of consumers didn't want to carry a lot of cash around with them, but they did want to be able to get cash after hours. By seeing a teller to get a few ATM tokens (usually worth $10 or £10 and sometimes available only in that denomination), you had the ability to retrieve cash, but only carried a bank document that was thought (due to features like revocability and the presence of ATMs under bank surveillance) to be relatively secure against theft. Since the tokens were later "cleared" against accounts much like checks, losing them wasn't necessarily a big deal, as something analogous to a "stop payment" was usually possible.

Unlike subway tokens, these were not coin-shaped. The most common scheme was a paper card, often the same dimensions as a modern credit card, but with punched holes that encoded the denomination and account holder information. The punched holes were also viewed as an anti-counterfeiting measure, probably not one that would hold up today, but still a roadblock to fraudsters who would have a hard time locating a keypunch and a valid account number. Manufacturers also explored some other intriguing opportunities, like the very first production cash dispenser, 1967's Barclaycash machine. This proto-ATM used punched paper tokens that were also printed in part with a Carbon-14 ink. Carbon-14 is unstable and emits beta radiation, which the ATM detected with a simple electrostatic sensor. For some reason difficult to divine the radioactive ATM card did not catch on.

For roughly the first decade of the "cash machine," they were offline devices that issued cash based on validating a token. The actual decision making, on the worthiness of a bank customer to withdraw cash, was still deferred to the teller who issued the tokens. Whether or not you would even consider this an ATM is debatable, although historical accounts generally do. They are certainly of a different breed than the modern online ATM, but they also set some of the patterns we still follow. Consider, for example, the ATMs within my lifespan that accepted deposits in an envelope. These ATMs did nothing with the envelopes other than accumulate them into a bin to go to a central processing center later on—the same way that early token-based ATMs introduced deposit boxes.

In this theory of ATM evolution, the missing link that made 1960s­–1970s ATMs so primitive was the lack of computer systems that were amenable to real-time data processing using networked peripherals. The '60s and '70s were a remarkable era in computer history, though, seeing the introduction of IBM's System/360 and System/370 line. These machines were more powerful, more flexible, and more interoperable than any before them. I think it's fair to say that, despite earlier dabbling, it was the 360/370 that truly ushered in the era of business computing. Banks didn't miss out.

One of the innovations of the System/360 was an improved and standardized architecture for the connection of peripherals to the machine. While earlier IBM models had supported all kinds of external devices, there was a lot of custom integration to make that happen. With the System/360, this took the form of "Bisync," which I might grandly call a far ancestor of USB. Bisync allowed a 360 computer to communicate with multiple peripherals connected to a common multi-drop bus, even using different logical communications protocols. While the first Bisync peripherals were "remote job entry" terminals for interacting with the machine via punched cards and teletype, IBM and other manufacturers found more and more applications in the following years.

IBM had already built document processing machines that interacted with their computers. In 1971, IBM joined the credit card fray with the 2730, a "transaction" terminal that we would now recognize as a credit card reader. It used a Bisync connection to a System/360-class machine to authorize a credit transaction in real time. The very next year, IBM took the logical next step: the IBM 2984 Cash Issuing Terminal. Like many other early ATMs, the 2984 had its debut in the UK as Lloyds Bank's "Cashpoint."

The 2984 similarly used Bisync communications with a System/360. While not the very first implementation of the concept, the 2984 was an important step in ATM security and the progenitor of an important line of cryptographic algorithms. To withdraw cash, a user inserted a magnetic card that contained an account number, and then keyed in a PIN. The 2984 sent this information, over the Bisync connection, to the computer, which then responded with a command such as "dispense cash." In some cases, the computer was immediately on the other side of the wall, but it was already apparent that banks would install ATMs in remote locations controlled via leased telephone lines—and those telephone lines were not well-secured. A motivated attacker (and with cash involved, it's easy to be motivated!) could probably "tap" the ATM's network connection and issue it spurious "dispense cash" commands. To prevent this problem, and assuage the concerns of bankers who were nervous about dispensing cash so far from the branch's many controls, IBM decided to encrypt the network connection.

The concept of an encrypted network connection was not at all new; encrypted communications were widely used in the military during the second World War and the concept was well-known in the computer industry. As IBM designed the 2984, in the late '60s, encrypted computer links were nonetheless very rare. There were not yet generally accepted standards, and cryptography as an academic discipline was immature.

IBM, to secure the 2984's network connection, turned to an algorithm recently developed by an IBM researcher named Horst Feistel. Feistel, for silly reasons, had named his family of experimental block ciphers LUCIFER. For the 2984, a modified version of one of the LUCIFER implementations called DSD-1¹. Through a Bureau of Standards design competition and the twists and turns of industry politics, DSD-1 later reemerged (with just slight changes) as the Data Encryption Standard, or DES. We owe the humble ATM honors for its key role in computer cryptography.

The 2984 was a huge step forward. Unlike the token-based machines of the 1960s, it was pretty much the same as the ATMs we use today. To use a 2984, you inserted your ATM card and entered a PIN. You could then choose to check your balance, and then enter how much cash you wanted. The machine checked your balance in real time and, if it was high enough, debited your account immediately before coughing up money.

The 2984 was not as successful as you might expect. The Lloyd's Bank rollout was big, but very few were installed by other banks. Collective memory of the 2984 is vague enough that I cannot give a definitive reason for its limited success, but I think it likely comes down to a common tale about IBM: price and flexibility. The 2984 was essentially a semi-custom peripheral, designed for Lloyd's Bank and the specific System/360 environment already in place there. Adoption for other banks was quite costly. Besides, despite the ATM's lead in the UK, the US industry had quickly caught up. By the time the 2984 would be considered by other banks, there were several different ATMs available in the US from other manufacturers (some of them the same names you see on ATMs today). The 2984 is probably the first "modern" ATM, but since IBM spent 4-5 years developing it, it was not as far ahead of the curve on launch day as you might expect. Just a year or two later, a now-forgotten company called Docutel was dominating the US market, leaving IBM little room to fit in.

Because most other ATMs were offered by companies that didn't control the entire software stack, they were more flexible, designed to work with simpler host support. There is something of an inverse vertical integration penalty here: when introducing a new product, close integration with an existing product family makes it difficult to sell! Still, it's interesting that the 2984 used pretty much the same basic architecture as the many ATMs that followed. It's worth reflecting on the 2984's relationship with its host, a close dependency that generally holds true for modern ATMs as well.

The 2984 connected to its host via a Bisync channel (possibly over various carrier or modem systems to accommodate remote ATMs), a communications facility originally provided for remote job entry, the conceptual ancestor of IBM's later block-oriented terminals. That means that the host computer expected the peripheral to provide some input for a job and then wait to be sent the results. Remote job entry devices, and block terminals later, can be confusing when compared to more familiar Unix-family terminals. In some ways, they were quite sophisticated, with the host computer able to send configuration information like validation rules for input. In other ways, they were very primitive, capable of no real logic other than receiving computer output (which was dumped to cards, TTY, or screen) and then sending computer input (from much the same devices). So, the ATM behaved the same way.

In simple terms, the ATM's small display (called a VDU or Video Display Unit in typical IBM terminology) showed whatever the computer sent as the body of a "display" command. It dispensed whatever cash the computer indicated with a "dispense cash" command. Any user input, such as reading a card or entry of a PIN number, was sent directly to the computer. The host was responsible for all of the actual logic, and the ATM was a dumb terminal, just doing exactly what the computer said. You can think of the Cash Issuing Terminal as, well, just that: a mainframe terminal with a weird physical interface.

Most modern ATMs follow this same model, although the actual protocol has become more sophisticated and involves a great deal more XML. You can be reassured that when the ATM takes a frustratingly long time to advance to the next screen, it is at least waiting to receive the contents of that screen from a host computer that is some distance away or, even worse, in The Cloud.

Incidentally, you might wonder about the software that ran on the host computer. I believe that the IBM 2984 was designed for use with CICS, the Customer Information Control System. CICS will one day get its own article, but it launched in 1966, built specifically for the Michigan Bell to manage customer and billing data. Over the following years, CICS was extensively expanded for use in the utility and later finance industries. I don't think it's inaccurate to call CICS the first "enterprise customer relationship management system," the first voyage in an adventure that took us through Siebel before grounding on the rocks of Salesforce. Today we wouldn't think of a CRM as the system of record for depository finance institutions like banks, but CICS itself was very finance-oriented from the start (telephone companies sometimes felt like accounting firms that ran phones on the side) and took naturally to gathering transactions and posting them against customer accounts. Since CICS was designed as an online system to serve telephone and in-person customer service reps (in fact making CICS a very notable early real-time computing system), it was also a good fit for handling ATM requests throughout the day.

Despite the 2984's lackluster success, IBM moved on. I don't think IBM was particularly surprised by the outcome, the 2984 was always a "request quotation" (e.g. custom) product. IBM probably regarded it as a prototype or pilot with their friendly customer Lloyds Bank. More than actual deployment, the 2984's achievement was paving the way for the IBM 3614 Consumer Transaction Facility.

In 1970, IBM had replaced the System/360 line with the System/370. The 370 is directly based on the 360 and uses the same instruction set, but it came with numerous improvements. Among them was a new approach to peripheral connectivity that developed into the IBM Systems Network Architecture, or SNA, basically IBM's entry into the computer networking wars of the 1970s and 1980s. While SNA would ultimately cede to IP (with, naturally, an interregnum of SNA-over-IP), it gave IBM the foundations for networked systems that are almost modern in their look and feel.

I say almost because SNA was still very much a mainframe-oriented design. An example SNA network might look like this: An S/370 computer running CICS (or one of several other IBM software packages with SNA support) is connected via channel (the high-speed peripheral bus on mainframe computers, analogous to PCI) to an IBM 3705 Communications Controller running the Network Control Program (analogous to a network interface controller). The 3705 had one or more "scanners" installed, which supported simple low-speed serial lines or fast, high-level protocols like SDLC (synchronous data link control) used by SNA. The 3705 fills a role sometimes called a "front-end processor," doing the grunt work of polling (scanning) communications lines and implementing the SDLC protocol so that the "actual computer" was relieved of these menial tasks.

At the other end of one of the SDLC links might be an IBM 3770 Data Communications System, which was superficially a large terminal that, depending on options ordered, could include a teletypewriter, card reader and punch, diskette drives, and a high speed printer. Yes, the 3770 is basically a grown-up remote job entry terminal, and the SNA/SDLC stack was a direct evolution from the Bisync stack used by the 2984. The 3770 had a bit more to offer, though: in order to handle its multiple devices, like the printer and card punch, it acted as a sort of network switch—the host computer identified the 3770's devices as separate endpoints, and the 3770 interleaved their respective traffic. It could also perform that interleaving function for additional peripherals connected to it by serial lines, which depending on customer requirements often included additional card punches and readers for data entry, or line printers for things like warehouse picking slips.

In 1973, IBM gave banks the SNA treatment with the 3600 Finance Communication System ². A beautifully orange brochure tells us:

The IBM 3600 Finance Communication System is a family of products designed to provide the Finance Industry with remote on-line teller station operation.

System/370 computers represented an enormous investment, generally around a million dollars and more often above that point than below. They were also large and required both infrastructure and staff to support them. Banks were already not inclined to install an S/370 in each branch, so it became a common pattern to place a "full-size" computer like an S/370 in a central processing center to support remote peripherals (over leased telephone line) in branches. The 3600 was a turn-key product line for exactly this use.

An S/370 computer with a 3704 or 3705 running the NCP would connect (usually over a leased line) to a 3601 System, which IBM describes as a "programmable communications controller" although they do not seem to have elevated that phrase to a product name. The 3601 is basically a minicomputer of its own, with up to 20KB of user-available memory and diskette drive. A 3601 includes, as standard, a 9600 bps SDLC modem for connection to the host, and a 9600 bps "loop" interface for a local multidrop serial bus. For larger installations, you could expand a 3601 with additional local loop interfaces or 4800 or 9600 bps modems to extend the local loop interface to a remote location via telephone line.

In total, a 3601 could interface up to five peripheral loops with the host computer over a single interleaved SDLC link. But what would you put on those peripheral loops? Well, the 3604 Keyboard Display Unit was the mainstay, with a vacuum fluorescent display and choice of "numeric" (accounting, similar to a desk calculator) or "data entry" (alphabetic) keyboard. A bank would put one of these 3604s in front of each teller, where they could inquire into customer accounts and enter transactions. In the meantime, 3610 printers provided general-purpose document printing capability, including back-office journals (logging all transactions) or filling in pre-printed forms such as receipts and bank checks. Since the 3610 was often used as a journal printer, it was available with a take-up roller that stored the printed output under a locked cover. In fact, basically every part of the 3600 system was available with a key switch or locking cover, a charming reminder of the state of computer security at the time.

The 3612 is a similar printer, but with the addition of a dedicated passbook feature. Remember passbook savings accounts, where the bank writes every transaction in a little booklet that the customer keeps? They were still around, although declining in use, in the 1970s. The 3612 had a slot on the front where an appropriately formatted passbook could be inserted, and like a check validator or slip printer, it printed the latest transaction onto the next empty line. Finally, the 3618 was a "medium-speed" printer, meaning 155 lines per minute. A branch bank would probably have one, in the back office, used for printing daily closing reports and other longer "administrative" output.

A branch bank could carry out all of its routine business through the 3600 system, including cash withdrawals. In fact, since a customer withdrawing cash would end up talking to a teller who simply keyed the transaction into a 3604, it seems like a little more automation could make an ATM part of the system.

Enter the 3614 Consumer Transaction Facility, the first IBM ATM available as a regular catalog item. The 3614 is actually fairly obscure, and doesn't seem to have sold in large numbers. Some sources suggest that it was basically the same as the 2984, but with a general facelift and adaptations to connect to a 3601 Finance Communication Controller instead of directly to a front-end processor. Some features which were optional on the 2984, like a deposit slot, were apparently standard on 3614. I'm not even quite sure when the 3614 was introduced, but based on manual copyright dates they must have been around by 1977.

One of the reasons the 3614 is obscure is that its replacement, the IBM 3624 Consumer Transaction Facility, hit the market in 1978—probably very shortly after the 3614. The 3624 was functionally very similar to the 3614, but with maintainability improvements like convenient portable cartridges for storing cash. It also brought a completely redesigned front panel that is more similar to modern ATMs. I should talk about the front panels—the IBM ATMs won a few design awards over their years, and they were really very handsome machines. The backlit logo panel and function-specific keys of the 3624 look more pleasant to use than most modern ATMs, although they would, of course, render translation difficult.

The 3614/3624 series established a number of conventions that are still in use today. For example, they added an envelope deposit system in which the machine accepted an envelope (with cash or checks) and printed a transaction identifier on the outside of the envelope for lookup at the processing center. This relieved the user of writing up a deposit slip when using the ATM. It was also capable of not only reading but, optionally, writing to the magnetic strips on ATM cards. To the modern reader that sounds strange, but we have to discuss one of the most enduring properties of the 3614/3624: their handling of PIN numbers.

I believe the 2984 did something fairly similar, but the details are now obscure (and seem to get mixed up with its use of LUCIFER/DSD-1/DES for communications). The 3614/3624, though, so firmly established a particular approach to PIN numbers that it is now known as the 3624 algorithm. Here's how it works: the ATM reads the card number (called Primary Account Number or PAN) off of the ATM card, reads a key from memory, and then applies a convoluted cryptographic algorithm to calculate an "intermediate PIN" from it. The "intermediate PIN" is then summed with a "PIN offset" stored on the card itself, modulo 10, to produce the PIN that the user is actually expected to enter. This means that your "true" PIN is a static value calculated from your card number and a key, but as a matter of convenience, you can "set" a PIN of your choice by using an ATM that is equipped to rewrite the PIN offset on your card. This same system, with some tweaks and a lot of terminological drift, is still in use today. You will sometimes hear IBM's intermediate PIN called the "natural PIN," the one you get with an offset of 0, which is a use of language that I find charming.

Another interesting feature of the 3624 was a receipt printer—I'm not sure if it was the first ATM to offer a receipt, but it was definitely an early one. The exact mechanics of the 3624 receipt printer are amusing and the result of some happenstance at IBM. Besides its mainframes and their peripherals, IBM in the 1970s was increasingly invested in "midrange computers" or "midcomputers" that would fill in a space between the mainframe and minicomputer—and, most importantly, make IBM more competitive with the smaller businesses that could not afford IBM's mainframe systems and were starting to turn to competitors like DEC as a result. These would eventually blossom into the extremely successful AS/400 and System i, but not easily, and the first few models all suffered from decidedly soft sales.

For these smaller computers, IBM reasoned that they needed to offer peripherals like card punches and readers that were also smaller. Apparently following that line of thought to a misguided extent, IBM also designed a smaller punch card: the 96-column three-row card, which was nearly square. The only computer ever to support these cards was the very first of the midrange line, the 1969 System/3. One wonders if the System/3's limited success led to excess stock of 96-column card equipment, or perhaps they just wanted to reuse tooling. In any case, the oddball System/3 card had a second life as the "Transaction Statement Printer" on the 3614 and 3624. The ATM could print four lines of text, 34 characters each, onto the middle of the card. The machines didn't actually punch them, and the printed text ended up over the original punch fields. You could, if you wanted, actually order a 3624 with two printers: one that presented the slip to the customer, and another that retained it internally for bank auditing. A curious detail that would so soon be replaced by thermal receipt printers.

Unlike IBM's ATMs before it, and, as we will see, unlike those after it as well, the 3624 was a hit. While IBM never enjoyed the dominance in ATMs that they did in computers, and companies like NCR and Diebold had substantial market share, the 3624 was widely installed in the late 1970s and would probably be recognized by anyone who was withdrawing cash in that era. The machine had technical leadership as well: NCR built their successful ATM line in part by duplicating aspects of the 3624 design, allowing interoperability with IBM backend systems. Ultimately, as so often happens, it may have been IBM's success that became its undoing.

In 1983, IBM completely refreshed their branch banking solution with the 4700 Finance Communication System. While architecturally similar, the 4700 was a big upgrade. For one, the CRT had landed: the 4700 peripherals replaced several-line VFDs with full-size CRTs typical of other computer terminals, and conventional computer keyboards to boot. Most radically, though, the 4700 line introduced distributed communications to IBM's banking offerings. The 4701 Communications Controller was optionally available with a hard disk, and could be programmed in COBOL. Disk-equipped 4701s could operate offline, without a connection to the host, or in a hybrid mode in which they performed some transactions locally and only contacted the host system when necessary. Local records kept by the 4701 could be automatically sent to the host computer on a scheduled basis for reconciliation.

Along with the 4700 series came a new ATM: the IBM 473x Personal Banking Machines. And with that, IBM's glory days in ATMs came crashing to the ground. The 473x series was such a flop that it is hard to even figure out the model numbers, the 4732 is most often referenced but others clearly existed, including the 4730, 4731, 4736, 4737, and 4738. These various models were introduced from 1983 to 1988, making up almost a decade of IBM's efforts and very few sales. The 4732 had a generally upgraded interface, including a CRT, but a similar feature set—unsurprising, given that the 3624 had already introduced most of the features ATMs have today. It also didn't sell. I haven't been able to find any numbers, but the trade press referred to the 4732 with terms like "debacle," so they couldn't have been great.

There were a few faults in the 4732's stars. First, IBM had made the decision to handle the 4700 Finance Communication System as a complete rework of the 3600. The 4700 controllers could support some 3600 peripherals, but 4700 peripherals could not be used with 3600 controllers. Since 3600 systems were widely installed in banks, the compatibility choice created a situation where many of the 4732's prospective buyers would end up having to replace a significant amount of their other equipment, and then likely make software changes, in order to support the new machine. That might not have been so bad on its own had IBM's competitors not provided another way out.

NCR made their fame in ATMs in part by equipping their contemporary models with 3624 software emulation, making them a drop-in modernization option for existing 3600 systems. Other ATM manufacturers had pursued a path of interoperability, with multiprotocol ATMs that supported multiple hosts, and standalone ATM host products that could interoperate with multiple backend accounting systems. For customers, buying an NCR or Diebold product that would work with whatever they already used was a more appealing option than buying the entire IBM suite in one go. It also matched the development cycle of ATMs better: as a consumer-facing device, ATMs became part of the brand image of the bank, and were likely to see replacement more often than back-office devices like teller terminals. NCR offered something like a regular refresh, while IBM was still in a mode of generational releases that would completely replace the bank's computer systems.

The 4732 and its 473x compatriots became the last real IBM ATMs. After a hiatus of roughly a decade, IBM reentered the ATM market by forming a joint venture with Diebold called InterBold. The basic terms were that Diebold would sell its ATMs in the US, and IBM would sell them overseas, where IBM had generally been the more successful of the two brands. The IBM 478x series ATMs, which you might encounter in the UK for example, are the same as the Diebold 1000 series in the US. InterBold was quite successful, becoming the dominant ATM manufacturer in the US, and in 1998 Diebold bought out IBM's share.

IBM had won the ATM market, and then lost it. Along the way, they left us with so much texture: DES's origins in the ATM, the 3624 PIN format, the dumb terminal or thin client model... even InterBold, IBM's protracted exit, gave us quite a legacy: now you know the reason that so many later ATMs ran OS/2. IBM, a once great company, provided Diebold with their once great operating system. Unlike IBM, Diebold made it successful.

The way I see it, few parts of American life are as quintessentially American as buying gas. We love our cars, we love our oil, and an industry about as old as automobiles themselves has developed a highly consistent, fully automated, and fairly user friendly system for filling the former with the latter.

I grew up in Oregon. While these rules have since been relaxed, many know Oregon for its long identity as one of two states where you cannot pump your own gas (the other being New Jersey). Instead, an attendant, employee of the gas station, operates the equipment. Like Portland's lingering indoor gas station, Oregon's favor for "full-service" is a holdover. It makes sense, of course, that all gas stations used to be full-service.

The front part of a gas station, where the pumps are and where you pull up your car, is called the Forecourt. The practicalities of selling gasoline, namely that it is a liquid sold by volume, make the forecourt more complex than you might realize. It's a set of devices that many of us interact with on a regular basis, but we rarely think about the sheer number of moving parts and long-running need for digital communications. Hey, that latter part sounds interesting, doesn't it?

Electric vehicles are catching on in the US. My personal taste in vehicles tends towards "old" and "cheap," but EVs have been on the market for long enough that they now come in that variety. Since my daily driver is an EV, I don't pay my dues at the Circle K nearly as often as I used to. One of the odd little details of EVs is the complexity hidden in the charging system or "EVSE," which requires digital communications with the vehicle for protection reasons. As consumers across the country install EVSE in their garages, we're all getting more familiar with these devices and their price tags. We might forget that, well, handling a fluid takes a lot of equipment as well... we just don't think about it, having shifted the whole problem to a large industry of loosely supervised hazardous chemical handling facilities.

Well, I don't mean to turn this into yet another discussion of the significant environmental hazard posed by leaking underground storage tanks. Instead, we're going to talk about forecourt technology. Let's start, then, with a rough, sketchy history of the forecourt.

The earliest volumetric fuel dispensers used an elevated glass tank where fuel was staged and measured before gravity drained it through the hose into the vehicle tank. Operation of these pumps was very manual, with an attendant filling the calibrated cylinder with the desired amount of gas, emptying it into the vehicle, and then collecting an appropriate sum of money. As an upside, the customer could be quite confident of the amount of fuel they purchased, since they could see it temporarily stored in the cylinder.

As cars proliferated in the 1910s, a company called Gilbarco developed a fuel dispenser that actually measured the quantity of fuel as it was being pumped from storage tank to vehicle... with no intermediary step in a glass cylinder required. The original Gilbarco design involved a metal turbine in a small glass sphere; the passing fuel spun the turbine which drove a mechanical counter. In truth, the design of modern fuel dispensers hasn't changed that much, although the modern volumetric turbines are made more accurate with a positive displacement design similar to a Roots blower.

Even with the new equipment, fuel was sold in much the same way: an attendant operated the pump, read the meter, and collected payment. There was, admittedly, an increased hazard of inattentive or malicious gas stations overcharging. Volumetric dispensers thus lead to dispensers that automatically calculated the price (now generally a legal requirement) and the practice of a regulatory authority like the state or tribal government testing fuel dispensers for calibration. Well, if consumers were expected to trust the gas station, perhaps the gas station ought to trust the consumer... and these same improvements to fuel dispensers made it more practical for the motorist to simply pump their own gas.

At the genesis of self-serve gasoline, most stations operated on a postpayment model. You pulled up, pumped gas, and then went inside to the attendant to pay whatever you owed. Of course, a few unscrupulous people would omit that last step. A simple countermeasure spread in busy cities: the pumps were normally kept powered off. Before dispensing gasoline, you would have to speak with the attendant. Depending on how trustworthy they estimated you to be, they might just turn on power to the pump or they might require you to deposit some cash with them in advance. This came to be known as "prepayment," and is now so universal in th US that the "prepay only" stickers on fuel dispensers seem a bit anachronistic ¹.

It's simple enough to imagine how this scheme worked, electronically. There is separate power wiring to the pumps for each dispenser (and these stations usually only had two dispensers anyway), and that wiring runs to the counter where the attendant can directly switch power. Most gas stations do use submersible pumps in the tank rather than in the actual dispenser, but older designs still had one pump per dispenser and were less likely to use submersible pumps anyway.

Soon, things became more complex. Modern vehicles have big gas tanks, and gas has become fairly expensive. What happens when a person deposits, say, $20 of "earnest cash" to get a pump turned on, and then pumps $25 worth of gas? Hopefully they have the extra $5, but the attendant doesn't know that. Besides, gas stations grew larger and it wasn't always feasible for the attendant to see the dispenser counters out the window. You wouldn't want to encourage people to just lie about the amount of gas they'd dispensed.

Gas stations gained remote control: using digital communications, fuel dispensers reported the value of their accumulators to a controller at the counter. The attendant would use the same controller to enable dispenser, potentially setting a limit at which the dispenser would automatically shut off. If you deposit $20, they enable the pump with a limit of $20. If you pay by card, they will likely authorize the card for a fixed amount (this used to routinely be $40 but has gone up for reasons you can imagine), enable the dispenser with no limit or a high limit, and then capture the actual amount after you finished dispensing ².

And that's how gas stations worked for quite a few decades. Most gas stations that you use today still have this exact same system in operation, but it may have become buried under additional layers of automation. There are two things that have caused combinatorial complexity in modern forecourt control: first, any time you automate something, there is a natural desire to automate more things. With a digital communications system between the counter and the forecourt, you can do more than just enable the dispensers! You might want to monitor the levels in the tanks, update the price on the big sign, and sell car wash vouchers with a discount for a related fuel purchase. All of these capabilities, and many more, have been layered on to forecourt control systems through everything from serial bus accessories to REST API third party integrations.

Speaking of leaking underground storage tanks, you likely even have a regulatory obligation to monitor tank levels and ensure they balance against bulk fuel deliveries and dispenser totals. This detects leakage, but it also detects theft, still a surprisingly common problem for gas stations. Your corporate office, or your bulk fuel provider, may monitor these parameters remotely to schedule deliveries and make sure that theft isn't happening with the cooperation of the station manager. Oh, and prices, those may be set centrally as well.

The second big change is nearly universal "CRIND." This is an awkward industry acronym for everyone's favorite convenience feature, Card Reader IN Dispenser. CRIND fuel dispensers let payment card customers complete the whole authorize, dispense, and capture process right at the dispenser, without coming inside at all. CRIND is so common today that it's almost completely displaced even its immediate ancestor, "fuel island" outdoor payment terminals (OPTs) that provide a central kiosk where customers make payments for multiple dispensers. This used to be a pretty common setup in California where self-service caught on early but, based on my recent travels, has mostly evaporated there.

So you can see that we have a complicated and open-ended set of requirements for communication and automation in the fuel court: enabling and monitoring pumps, collecting card payments, and monitoring and controlling numerous accessories. Most states also require gas stations to have an intercom system so that customers can request help from the attendant inside. Third-party loyalty systems were briefly popular although, mercifully, the more annoying of them have mostly died out... although only because irritating advertising-and-loyalty technology has been better integrated into the dispensers themselves.

Further complicating things, gas station forecourts are the epitome of legacy integration. Fuel dispensers are expensive, concrete slabs are expensive, and gas stations run on thin margins. While there aren't very many manufacturers of fuel dispensers, or multi-product dispensers as they're typically called today, the industry of accessories, control systems, and replacement parts is vast. Most gas stations have accumulated several different generations of control systems and in-dispenser accessories like tree rings. New features like CRIND, chip payment, touchless payment, and "Gas Station TV" have each motivated another round of new communications protocols.

And that's how we get to our modern world, where the brochure for a typical gas station forecourt controller lists 25+ different communications protocols—and assures that you can use "any mix."

Variability between gas stations increases when you consider the differing levels of automation available. It used to be common for gas stations to use standalone pump controllers that didn't integrate with much else—when you prepaid, for example, the cashier would manually enter the pump number and prepayment limit on a separate device from the cash register.

Here in New Mexico, quite a few stations used to use the Triangle MicroSystems MPC family, a wedge-shaped box with an industrial-chic membrane keypad in grey and bright red. Operation of the MPC is pretty simple, basically pressing a pump number and then entering a dollar limit. Of course, the full set of features runs much deeper, including financial reporting and fleet fueling contracts.

This is another important dimension of the gas station control industry: fleet fueling. It used to be that gas stations were divided into two categories, consumer stations that took cash payment and "cardlock" stations that used an electronic payment system. Since cardlock stations originally relied on proprietary, closed payment agreements, they didn't sell to consumers and had different control requirements (often involving an outside payment terminal). As consumers widely adopted card payments, the lines between the two markets blurred. Modern cardlock fueling networks, like CFN and Wex, are largely just another set of payment processors. Most major gas stations participate in most major cardlock networks, just the same as they participate in most major ATM networks for lower-cost processing of debit cards.

Of course, more payment networks call for more integrations. The complexity of the modern payment situation has generally outgrown standalone controllers, and they seem to be fading away. Instead, the typical gas station today has forecourt control completely integrated into their POS system. Forecourt integration is such an important requirement that gas station convenience stores, mostly handling normal grocery-type transactions, nevertheless rely almost exclusively on dedicated gas station POS solutions. In other words, next time you buy a can of Monster and a bag of chips, the cashier most likely rings you up and takes payment through a POS solution offered by the dispenser manufacturer (like Gilbarco Passport Retail) or one of dozens of vendors that caters specifically to gas stations (including compelling names like Petrosoft). Control of fuel dispensers is just too weird of a detail to integrate into other POS platforms... or so it was thought, although things clearly get odd as Gilbarco has to implement basic kitchen video system integration for the modern truck stop.

So how does this all work technically? That's the real topic of fascination, right? Well, it's a mess and hard to describe succinctly. There are so many different options, and particularly legacy retrofit options, that one gas station will be very different from the next.

In the days of "mechanical pumps," simple designs with mechanical counters, control wiring was simple: the dispenser (really a mechanical device called a pulser) was expected to provide "one pulse per penny" on a counting circuit for dollars dispensed, which incremented a synchronized counter on the controller. For control the other way, the controller just closed relays to open "fast" or "slow" valves on the dispenser. The controller might also get a signal when a handle lever is activated, to alert the attendant that someone is trying to use a dispenser, but that was about it.

Later on, particularly as multi-product dispensers with two hoses and four rates (due to diesel and three grades) became common, wiring all the different pulse and valve circuits became frustrating. Besides, pumps with digital counters no longer needed mechanical adjustment when prices changed, allowing for completely centralized price calculation. To simplify wiring while enabling new features, fuel dispenser manufacturers introduced simple current-loop serial buses. These are usually implemented as a single loop that passes through each dispenser, carrying small packets with addressed commands or reports, usually at a pretty low speed. The dispensers designed for use with these systems are much more standalone than the older mechanical dispensers, and perform price accumulation internally, so they only needed to report periodic totals during fueling and at the end of the transaction.

An upside of these more standalone dispensers is that they made CRIND easier to implement: the payment terminal in the dispenser could locally enable the pump, including setting limits, by a direct interface to the pump controller. Still, the CRIND needed some way to actually authorize and report transactions. Solution: another current loop. Most CRIND installations involved a second, similar, but usually higher-speed serial bus that specifically handled payment processing. The CRIND terminals in such a system usually communicated with a back-office payment server using a very simple protocol, sending card information in the clear. That back-office server might be in the back of the convenience store, but it could also be remote.

As gas stations introduced CRIND, plastic card sales became a key part of the business. Card volume is much greater than cash volume at most stations, and it's known that customers will often leave rather than go inside if there is a problem with CRIND payment. So gas stations prioritized reliability of payments. To this day, if you look at the roof of many gas stations, you'll find a small parabolic antenna pointed aimlessly skywards. By the end of the 1990s, many chain gas stations used satellite networks for payment processing, either routinely (cheaper than a leased telephone line!) or as a contingency. Cisco's VSAT terminal modules for edge routers, combined with a boutique industry of Mbps-class data networks on leased transponders, made satellite a fairly inexpensive and easy-to-obtain option for handling small payment processing messages.

This arrangement of one current loop for dispenser control and one current loop for payment terminals lasted for long enough that it became a de facto wiring standard for the gas station forecourt. New construction gas stations provided conduits from the convenience store to the pumps, and those conduits were usually spec'd for an A/C power circuit (controlled, per code, by an emergency stop button) and two low-voltage data circuits. The low-voltage data circuits were particularly important because the electrical code and fire code impose specific rules on electrical systems used in proximity to flammable fluids—what's called a "hazardous environment" in the language of safety codes. Dispenser manufacturers sold specialized field interconnection enclosures that isolated the data circuits to the required safety standard, lowering the complexity of the installation in the dispensers themselves ³.

The next event to challenge forecourt infrastructure was the introduction of EMV chip and tap-to-pay payment cards. Many Americans will remember how fuel dispensers routinely had tap-to-pay terminals physically installed for years, even a decade, before they actually started working. Modernizing dispensers usually meant installing a new CRIND system with EMV support, but upgrades to the underlying network to support them took much longer. The problem was exactly the simplicity of the CRIND current loop design: EMV standards required that all data be encrypted (you couldn't just send card numbers to the backend in the clear as older systems did), and required larger and more numerous messages between the payment network, the terminal, and the card itself. Even if supporting EMV transactions on the serial bus was possible, most manufacturers chose not to, opting for the vastly simpler design of direct IP connectivity to each CRIND terminal.

But how do you put IP over a simple two-wire serial bus? Well, there are a lot of options, and the fuel dispenser industry chose basically all of them. There were proprietary solutions, but more common were IP networking technologies adapted to the forecourt application. Consider DSL: for a good decade, many forecourt interconnection boxes and fuel dispenser controllers supported good old fashioned DSL over the payment loop (not to be confused with DSL as in Diesel, an abbreviation also used in the fuel industry).

Bandwidth requirements increased yet further, though, with the introduction of Maria Menounos. "Forecourt media" advertising systems can deliver full video to each dispenser, a golden opportunity to pitch credit cards and monetize something called "Cheddar." While there was a long era of satellite transponders delivering analog video to chains for in-store marketing (I will one day write about WalMart TV), the "GSTV" phenomenon is newer and completely internet-based. For HD video you need a little better than the 5Mbps performance that industrial DSL systems were delivering. Enter HomePlug.

Despite HomePlug's limited market success, it has been widely adopted in gas station forecourts. The advantage of HomePlug is clear: it dispenses with the control wiring loops entirely, providing IP communications with dispensers over the electrical supply wiring. It usually presents an almost zero-wiring upgrade, just adding HomePlug boards on both ends, so even in stations with good forecourt serial loops, dispenser upgrades often end in a switch to HomePlug.

The most interesting thing about these networks is just how modular it all still is: somewhere in your local gas station, there is a forecourt controller. Depending on the age of the system, that might be a bespoke embedded system with plug-in modules, or it might be a generic N100 Mini PC with a few serial ports and mostly IP connectivity. There is likely a forecourt interconnection box that holds not just the wiring terminals but also adapter boards that convert between various serial protocols, IP carriers, and control signals. The point of sale backend server might interact with the forecourt controller via IP, but older systems used RS-232... and systems in between might use the same logical protocol as they did with RS-232, but encapsulated in TCP. The installation manuals for all of these products include pages of wiring diagrams for each different scenario.

Next time you stop at a gas station and find the CRIND not working, think about all of that: whatever technician comes out to fix it will have their work cut out for them, just to figure out which way that gas station is set up.

The front of the American grocery store contains a strange, liminal space: the transitional area between parking lot and checkstand, along the front exterior and interior of the building, that fills with oddball commodities. Ice is a fixture at nearly every store, filtered water at most, firewood at some. This retail purgatory, both too early and too late in the shopping journey for impulse purchases, is mostly good only for items people know they will need as they check out. One of the standard residents of this space has always struck me as peculiar: dry ice.

Carbon dioxide ice is said to have been invented, or we might better say discovered, in the 1830s. For whatever reason, it took just about a hundred years for the substance to be commercialized. Thomas B. Slate was a son of Oregon, somehow ended up in Boston, and then realized that the solid form of CO2 was both fairly easy to produce and useful as a form of refrigeration. With an eye towards marketing, he coined the name Dry Ice—and founded the DryIce Corporation of America. The year was 1925, and word quickly spread. In a widely syndicated 1930 article, "Use of Carbon Dioxide as Ice Said to be Developing Rapidly," the Alamogordo Daily News and others reported that "the development of... 'concentrated essence of frigidity' for use as a refrigerant in transportation of perishable products, is already taxing the manufacturing facilities of the Nation... So rapidly has the use of this new form of refrigeration come into acceptance that there is not sufficient carbon dioxide gas available."

The rush to dry ice seems strange today, but we must consider the refrigeration technology of the time. Refrigerated transportation first emerged in the US during the middle of the 19th century. Train boxcars, packed thoroughly with ice, carried meat and fruit from midwestern agriculture to major cities. This type of refrigerated transportation greatly expanded the availability of perishables, and the ability to ship fruits and vegetables between growing regions made it possible, for the first time, to get some fresh fruit out of season. Still, it was an expensive proposition: railroads built extensive infrastructure to support the movement of trains loaded down with hundreds of tons of ice. The itself had to be quarried from frozen lakes, some of them purpose-built, a whole secondary seasonal transportation economy.

Mechanical refrigeration, using some kind of phase change process as we are familiar with today, came about a few decades later and found regular use on steamships by 1900. Still, this refrigeration equipment was big and awkward; steam power was a practical requirement. As the Second World War broke out, tens of thousands of refrigerated railcars and nearly 20,000 refrigerated trucks were in service—the vast majority still cooled by ice, not mechanical refrigeration.

You can see, then, the advantages of a "dryer" and lighter form of ice. The sheer weight of the ice significantly reduced the capacity of refrigerated transports. "One pound of carbon dioxide ice at 110 degrees below zero is declared to be equivalent to 16 pounds of water ice," the papers explained, for the purposes of transportation. The use of dry ice could reduce long-haul shipping costs for fruit and vegetables by 50%, the Department of Commerce estimated, and dry ice even opened the door to shipping fresh produce from the West Coast to the East—without having to "re-ice" the train multiple times along the way. Indeed, improvements in refrigeration would remake the American agricultural landscape. Central California was being irrigated so that produce could grow, and refrigeration would bring that produce to market.

1916 saw the American Production Company drilling on the dusty plains of northeastern New Mexico, a few miles south of the town of Bueyeros. On the banks of an anonymous wash, in the shadow of Mesa Quitaras, they hoped to strike oil. Instead, at about 2,000 feet, they struck something else: carbon dioxide. The well blew wide open, and spewed CO2 into the air for about a year, the production estimated at 25,000,000 cubic feet of gas per day under natural pressure. For American Production, this was an unhappy accident. They could identify no market for CO2, and a year later, they brought the well under control, only to plug and abandon it permanently.

Though the "No. 1 Bueyeros" well was a commercial failure at the time, it was not wasted effort. American Production had set the future for northeastern New Mexico. There was oil, if you looked in the right place. American Production found its own productive wells, and soon had neighbors. Whiting Brothers, once operator of charismatic service stations throughout the Southwest and famously along Route 66, had drilled their own wells by 1928. American Production became part of British Petroleum. Breitburn Production of Texas has now consolidated much of the rest of the field, and more than two million cubic feet of natural gas come from northeastern New Mexico each month.

If you looked elsewhere, there was gas—not natural gas, but CO2. Most wells in the region produced CO2 as a byproduct, and the less fortunate attempts yielded nothing but CO2. The clear, non-flammable gas was mostly a nuisance in the 1910s and 1920s. By the 1930s, though, promotion by the DryIce Corporation of America (in no small part through the Bureau of Commerce) had worked. CO2 started to be seen as a valuable commodity.

The production of dry ice is deceptively simple. Given my general knowledge about producing and handling cryogenic gases, I was surprised to read of commercial-scale production with small plants in the 1930s. There is, it turns out, not that much to it. One of the chief advantages of CO2 as an industrial gas is its low critical temperature and pressure. If you take yourself back to high school chemistry, and picture a phase diagram, we can think about liquifying the CO2 gas coming out of a well. The triple point of carbon dioxide, where increasing pressure and temperature will make it a liquid, is at around -60 Celsius and 5 atmospheres. The critical point, beyond which CO2 becomes a supercritical gas-fluid hybrid, is only at 30 degrees Celsius and 72 atmospheres. In terms more familiar to us Americans, that's about 88 degrees F and 1,000 PSI.

In other words, CO2 gas becomes a liquid at temperatures and pressures that were readily achievable, even with the early stages of chemical engineering in the 1930s. With steam-powered chillers and compressors, it wasn't difficult to produce liquid CO2 in bulk. But CO2 makes the next step even more convenient: liquid CO2, released into open air, boils very rapidly. As it bubbles away, the phase change absorbs energy, leaving the remaining liquid CO2 even colder. Some of it freezes into ice, almost like evaporating seawater to extract the salt, evaporating liquid CO2 leaves a snow-like mass of flaky, loose CO2 ice. Scoop that snow up, pack it into forms, and use steam power or weight to compress it, and you have a block of the product we call dry ice.

The Bueyeros Field, as it was initially known, caught the interest of CO2 entrepreneurs in 1931. A company called Timmons Carbonic, or perhaps Southern Dry Ice Company (I suspect these to be two names for the same outfit), produced a well about a mile east, up on the mesa.

Over the next few years, the Estancia Valley Carbon Dioxide Development Company drilled a series of wells to be operated by Witt Ice and Gas. These were located in the Estancia field, further southwest and closer to Albuquerque. Witt built New Mexico's first production dry ice plant, which operated from 1932 to 1942 off of a pipeline from several nearby wells. Low pressure and difficult drilling conditions in the Estancia field limited the plant's output, so by the time it shut down Witt had already built a replacement. This facility, known as the Bueyeros plant, produced 17 tons of dry ice per day starting in 1940. It is located just a couple of miles from the original American Production well, north of Mesa Quitaras.

About 2,000' below the surface at Bueyeros lies the Tubb Sandstone, a loose aggregation of rock stuck below the impermeable Cimarron Anhydrite. Carbon dioxide can form underground through several processes, including the breakdown of organic materials under great heat and pressure (a process that creates petroleum oil as well) and chemical reactions between different minerals, especially when volcanic activity causes rapid mixing with plenty of heat. There are enough mechanisms of formation, either known or postulated, that it's hard to say where exactly the CO2 came from. Whatever its source, the gas flowed upwards underground into the sandstone, where it became trapped under the airtight layer of Anhydrite. It's still there today, at least most of it, and what stands out in particular about northeastern New Mexico's CO2 is its purity. Most wells in the Bueyeros field produce 99% pure CO2, suitable for immediate use.

Near Solano, perhaps 20 miles southwest of Bueyeros by air, the Carbonic Chemical Co built the state's largest dry ice plant. Starting operation in 1942, the plant seems to have initially gone by the name "Dioxice," immortalized as a stop on the nearby Union Pacific branch. Dioxice is an occasional synonym for Dry Ice, perhaps intended to avoid the DryIce Corporation's trademark, although few bothered. The Carbonic Chemical Plant relied on an 18 mile pipeline to bring gas from the Bueyeros field. Uniquely, this new plant used a "high pressure process." By feeding the plant only with wells producing high pressure (hundreds of PSI, as much as 500 PSI of natural pressure at some wells), the pipeline was made more efficient and reliable. Further, the already high pressure of the gas appreciably raised the temperature at which it would liquefy.

The Carbonic Chemical plant's ammonia chillers only had to cool the CO2 to -15 degrees F, liquifying it before spraying it into "snow chambers" that filled with white carbon dioxide ice. A hydraulic press, built directly into the snow chamber, applied a couple of hundred tons of force to create a solid block of dry ice weighing some 180 pounds. After a few saw cuts, the blocks were wrapped in paper and loaded onto insulated train cars for delivery to customers throughout the west—and even some in Chicago.

The main applications of CO2, a 1959 New Mexico Bureau of Mines report explains, were dry ice for shipping. Secondarily, liquid CO2 was shipped in tanks for use in carbonating beverages. Witt Ice and Gas in particular built a good business out of distributing liquid CO2 for beverage and industrial use, and for a time was a joint venture with Chicago-based nationwide gas distributor Cardox. Bueyeros's gas producers found different customers over time, so it is hard to summarize their impact, but we know some salient examples. Most beverage carbonation in mid-century Denver, and perhaps all in Albuquerque, used Bueyeros gas. Dry ice from Bueyeros was used to pack train cars passing through from California, and accompanied them all the way to the major cities of the East Coast.

By the 1950s, much of the product went to a more modern pursuit. Experimental work pursued by the military and the precursors to the Department of Energy often required precise control of low temperatures, and both solid and liquid CO2 were suitable for the purpose. In the late 1950s, Carbonic Chemical listed Los Alamos Scientific Laboratory, Sandia Laboratories, and White Sands Missile Range as their primary customers.

Bueyeros lies in Harding County, New Mexico. Harding County is home to two incorporated cities (Roy and Mosquero), a couple of railroad stops, a few highways, and hardly 650 people. It is the least populous county of New Mexico, but it's almost the size of Delaware. Harding County has never exactly been a metropolis, but it did used to be a more vital place. In the 1930s, as the CO2 industry built out, there were almost 4,500 residents. Since then, the population has declined about 20% from each census to the next.

CO2 production went into a similar decline. After the war, significant improvements in refrigeration technology made mechanical refrigeration inevitable, even for road transportation. Besides, the growing chemical industry had designed many industrial processes that produced CO2 as a byproduct. CO2 for purposes like carbonation and gas blanketing was often available locally at lower prices than shipped-in well CO2, leading to a general decline in the CO2 industry.

Growing understanding of New Mexico geology and a broader reorganizing of the stratigraphic nomenclature lead the Bueyeros Field to become part of the Bravo Dome. Bravo Dome CO2 production in the 1950s and 1960s was likely supported mostly by military and weapons activity, as by the end of the 1960s the situation once again looked much like it did in the 1910s: the Bravo Dome had a tremendous amount of gas to offer, but there were few applications. The rate of extraction was limited by the size of the market. Most of the dry ice plants closed, contributing, no doubt, to the depopulation of Harding County.

The whole idea of drilling for CO2 is now rather amusing. Our modern problems are so much different: we have too much CO2, and we're producing even more without even intending to. It has at times seemed like the industry of the future will be putting CO2 down into the ground, not taking it out. What happened out in Harding County was almost the opening of Pandora's box. A hundred years ago, before there was a dry ice industry in the US, newspaper articles already speculated as to the possibility of global warming by CO2. At the time, it was often presented as a positive outcome: all the CO2 released by burning coal would warm the environment and thus reduce the need for that coal, possibly even a self-balancing problem. It's even more ironic that CO2 was extracted mostly to make things colder, given the longer-term consequences. Given all that, you would be forgiven for assuming that drilling for CO2 was a thing of the past.

The CO2 extraction industry has always been linked to the oil industry, and oil has always been boom and bust. In 1982, there were 16 CO2 wells operating in the Bravo Dome field. At the end of 1985, just three years later, there were 258. Despite the almost total collapse of demand for CO2 refrigeration, demand for liquid CO2 was up by far. It turns out that American Production hadn't screwed up in 1917, at least not if they had known a little more about petroleum engineering.

In 1972, the Scurry Area Canyon Reef Operators Committee of West Texas started an experiment, attempting industrial application of a technique first proposed in the 1950s. Through a network of non-productive oil wells in the Permian Basin, they injected liquid CO2 deep underground. The rapidly evaporating liquid raised the pressure in the overall oil formation, and even lubricated and somewhat fractured the rock, all of which increased the flow rate at nearby oil wells. A decade later, the concept was proven, and CO2 Enhanced Oil Recovery (EOR) swept across the Permian Basin.

Today, it is estimated that about 62% of the global industrial production of CO2 is injected into the ground somewhere in North America to stimulate oil production. The original SACROC system is still running, now up to 414 injection wells. There are thousands more. Every day, over two billion cubic feet of CO2 are forced into the ground, pushing back up 245,000 barrels of additional oil.

British Petroleum's acquisition of American Production proved fortuitous. BP became one of the country's largest producers of CO2, extracted from the ground around Bueyeros and transported by pipeline directly to the Permian Basin for injection. In 2000, BP sold their Bravo Dome operations to Occidental Petroleum ¹. Now going by Oxy, the petroleum giant has adopted a slogan of "Zero In". That's zero as in carbon emissions.

I would not have expected to describe Occidental Petroleum as "woke," but in our contemporary politics they stand out. Oxy mentions "Diversity, Inclusion, and Belonging" on the front page of their website, which was once attractive to investors but now seems more attractive to our nation's increasingly vindictive federal government. Still, Oxy is sticking to a corporate strategy that involves acknowledging climate change as real, which I suppose counts as refreshing. From a 2025 annual report:

Oxy is building an integrated portfolio of low-carbon projects, products, technologies and companies that complement our existing businesses; leveraging our competitive advantages in CO2 EOR, reservoir management, drilling, essential chemicals and major infrastructure projects; and are designed to sustain long term shareholder value as we work to implement our Net-Zero Strategy.

Yes, Oxy has made achieving net-zero carbon a major part of their brand, and yes, this model of reducing carbon emissions relies heavily on CO2 EOR: the extraction of CO2 from the ground.

In a faltering effort to address carbon emissions, the United States has leaned heavily on the promise of Carbon Capture and Storage (CCS) technologies. The idea is to take CO2 out of the environment (potentially by separating it from the air but, more practically, by capturing it in places where it is already concentrated by industrial processes) and to put it somewhere else. Yes, this has shades of the Australian television sketch about the ship whose front fell off, but the key to "sequestration" is time. If we can put enough carbon somewhere that it will say for enough time, we can reduce the "active" greenhouse gas content of our environment. The main way we have found of doing this is injecting it deep underground. How convenient, then, that the oil industry is already looking for CO2 for EOR.

CCS has struggled in many ways, chief among them that the majority of planned CCS projects have never been built. As with most of our modern carbon reduction economy, even the CCS that has been built is, well, a little bit questionable. There is something of a Faustian bargain with fossil fuels. As we speak, about 45 megatons of CO2 are captured from industrial processes each year for CCS. Of that 45 Mt, 9 Mt are injected into dedicated CO2 sequestration projects. The rest, 80%, is purchased by the oil industry for use in EOR.

This form of CCS, in which the captured CO2 is applied to an industrial process that leads to the production of more CO2, has taken to the name CCUS. That's Carbon Capture, Utilization, and Storage. Since the majority of the CO2 injected for EOR never comes back up, it is a form of sequestration. Although the additional oil produced will generally be burned, producing CO2, the process can be said to be inefficient in terms of CO2. In other words, the CO2 produced by burning oil from EOR is less in volume than the CO2 injected to stimulate recovery of that oil.

Mathematically, CCUS, the use of CO2 to produce oil, leads to a net reduction in released CO2. Philosophically, though, it is deeply unsatisfying. This is made all the worse by the fact that CCUS has benefited from significant government support. Outright subsidies for CCS are uncommon, although they do exist. What are quite common are grants and subsidized financing for the capital costs of CCS facilities. Nearly all CCS in the US has been built with some degree of government funding, totaling at least four billion dollars, and regulatory requirements for CCS to offset new fossil fuel plants may create a de facto electrical ratepayer subsidy for CCS. Most of that financial support, intended for our low-carbon future, goes to the oil producers.

The Permian Basin is well-positioned for CCS EOR because it produces mostly natural gas. Natural gas in its raw form, "well gas," almost always includes CO2. Natural gas processing plants separate the combustible gases from noncombustible ones, producing natural gas that has a higher energy content and burns more cleanly—but, in the process, venting large quantities of CO2 into the atmosphere. Oxy is equipping its Permian Basin natural gas plants with a capture system that collects the CO2 and compresses it for use in EOR.

The problem is that CO2 consumption for EOR has, as always, outpaced production. There aren't enough carbon capture systems to supply the Permian Basin fields, so "sequestered" CO2 is mixed with "new" CO2. Bravo Dome CO2 production has slowly declined since the 1990s, due mostly to declining oil prices. Even so, northeastern New Mexico is still full of Oxy wells bringing up CO2 by the millions of cubic feet. 218 miles of pipeline deliver Bueyeros CO2 into West Texas, and 120 miles of pipeline the other way land it in the oil fields of Wyoming. There is very nearly one producing CO2 well per person in Harding County.

Considering the totality of the system, it appears that government grants, financing incentives, and tax credits for CCS are subsidizing not only natural gas production but the extraction of CO2 itself. Whether this is progress on climate change or a complete farce depends a mathematical analysis. CO2 goes in, from several different sources; CO2 goes out, to several different dispositions. Do we remove more from the atmosphere than we end up putting back? There isn't an obvious answer.

The oil industry maintains that CCS is one of the most practical means of reducing carbon emissions, with more CO2 injected than produced and a resulting reduction in the "net CO2 impact" of the product natural gas.

As for more independent researchers, well, a paper finding that CCS EOR "cannot contribute to reductions" isn't the worst news. A 2020 literature review of reports on CCS EOR projects found that they routinely fail to account for significant secondary carbon emissions and that, due to a mix of the construction and operational realities of CCS EOR facilities and the economics of oil consumption, CCS EOR has so far produced a modest net increase in greenhouse gas emissions.

They're still out there today, drilling for carbon dioxide. The reports from the petroleum institute today say that the Permian Basin might need even more shipped in. New Mexico is an oil state; Texas gets the reputation but New Mexico has the numbers. Per-capita oil production here is significantly higher than Texas and second only to North Dakota. New Mexico now produces more oil than Old Mexico, if you will, the country to our south.

Per capita, New Mexico ranks 12th for CO2 emissions, responsible for about 1% of the nation's total. Well, I can do a bit better: for CO2 intentionally extracted from the ground, New Mexico is #3, behind only Colorado and Mississippi for total production. We produce something around 17% of the nation's supply of extracted CO2, and we even use most of it locally. I guess that's something you could put a good spin on.

Previously on Computers Are Bad, we discussed the early history of air traffic control in the United States. The technical demands of air traffic control are well known in computer history circles because of the prominence of SAGE, but what's less well known is that SAGE itself was not an air traffic control system at all. SAGE was an air defense system, designed for the military with a specific task of ground-controlled interception (GCI). There is natural overlap between air defense and air traffic control: for example, both applications require correlating aircraft identities with radar targets. This commonality lead the Federal Aviation Agency (precursor to today's FAA) to launch a joint project with the Air Force to adapt SAGE for civilian ATC.

There are also significant differences. In general, SAGE did not provide any safety functions. It did not monitor altitude reservations for uniqueness, it did not detect loss of separation, and it did not integrate instrument procedure or terminal information. SAGE would need to gain these features to meet FAA requirements, particularly given the mid-century focus on mid-air collisions (a growing problem, with increasing air traffic, that SAGE did nothing to address).

The result was a 1959 initiative called SATIN, for SAGE Air Traffic Integration. Around the same time, the Air Force had been working on a broader enhancement program for SAGE known as the Super Combat Center (SCC). The SCC program was several different ideas grouped together: a newer transistorized computer to host SAGE, improved communications capabilities, and the relocation of Air Defense Direction Centers from conspicuous and vulnerable "SAGE Blockhouses" to hardened underground command centers, specified as an impressive 200 PSI blast overpressure resistance (for comparison, the hardened telecommunication facilities of the Cold War were mostly specified for 6 or 10 PSI).

At the program's apex, construction of the SCCs seemed so inevitable that the Air Force suspended the original SAGE project under the expectation that SCC would immediately obsolete it. For example, my own Albuquerque was one of the last Air Defense Sectors scheduled for installation of a SAGE computer. That installation was canceled; while a hardened underground center had never been in the cards for Albuquerque, the decision was made to otherwise build Albuquerque to the newer SCC design, including the transistorized computer. By the same card, the FAA's interest in a civilian ATC capability, and thus the SATIN project, came to be grouped together with the SCC program as just another component of SAGE's next phase of development.

SAGE had originally been engineered by MIT's Lincoln Laboratory, then the national center of expertise in all things radar. By the late 1950s a large portion of the Lincoln Laboratory staff were working on air defense systems and specifically SAGE. Those projects had become so large that MIT opted to split them off into a new organization, which through some obscure means came to be called the MITRE Corporation. MITRE was to be a general military R&D and consulting contractor, but in its early years it was essentially the SAGE company.

The FAA contracted MITRE to deliver the SATIN project, and MITRE subcontracted software to the Systems Development Corporation, originally part of RAND and among the ancestors of today's L3Harris. For the hardware, MITRE had long used IBM, who designed and built the original AN/FSQ-7 SAGE computer and its putative transistorized replacement, the AN/FSQ-32. MITRE began a series of engineering studies, and then an evaluation program on prototype SATIN technology.

There is a somewhat tenuous claim that you will oft see repeated, that the AN/FSQ-7 is the largest computer ever built. It did occupy the vast majority of the floorspace of the four-story buildings built around it. The power consumption was around 3 MW, and the heat load required an air conditioning system at the very frontier of HVAC engineering (you can imagine that nearly all of that 3 MW had to be blown out of the building on a continuing basis). One of the major goals of the AN/FSQ-32 was reduced size and power consumption, with the lower heat load in particular being a critical requirement for installation deep underground. Of course, the "deep underground" part more than wiped out any savings from the improved technology.

From Air Defense to Air Traffic Control

By the late 1950s, enormous spending for the rapid built-out of defense systems including SAGE and the air defense radar system (then the Permanent System) had fatigued the national budget and Congress. The winds of the Cold War had once again changed. In 1959, MITRE had begun operation of a prototype civilian SAGE capability called CHARM, the CAA High Altitude Remote Monitor (CAA had become the FAA during the course of the CHARM effort). CHARM used MIT's Whirlwind computer to process high-altitude radar data from the Boston ARTCC (Air Route Traffic Control Center), which it displayed to operators while continuously evaluating aircraft movements for possible conflicts. CHARM was designed for interoperability with SAGE, the ultimate goal being the addition of the CHARM software package to existing SAGE computers. None of that would ever happen; by the time the ball dropped for the year 1960 the Super Combat Center program had been almost completely canceled. SATIN, and the whole idea of civilian air traffic control with SAGE, became blast damage.

In 1961, the Beacon Report concluded that there was an immediate need for a centralized, automated air traffic control system. Mid-air collisions had become a significant political issue, subject of congressional hearings and GAO reports. The FAA seemed to be failing to rise to the task of safe civilian ATC, a perilous situation for such a new agency... and after the cancellation of the SCCs, the FAA's entire plan for computerized ATC was gone.

During the late 1950s and 1960s, the FAA adopted computer systems in a piecemeal fashion. Many enroute control centers (ARTCCs), and even some terminal facilities, had some type of computer system installed. These were often custom software running on commodity computers, limited to tasks like recording flight plans and making them available to controllers at other terminals. Correlation of radar targets with flight plans was generally manual, as were safety functions like conflict detection.

These systems were limited in scale—the biggest problem being that some ARTCCs remained completely manual even in the late 1960s. On the upside, they demonstrated much of the technology required, and provided a test bed for implementation. Many of the individual technical components of ATC were under development, particularly within IBM and Raytheon, but there was no coordinated nationwide program. This situation resulted in part from a very intentional decision by the FAA to grant more decision making power to its regional offices, a concept that was successful in some areas but in retrospect disastrous in others. In 1967, the Department of Transportation was formed as a new cabinet-level executive department. The FAA, then the Federal Aviation Agency, was reorganized into DOT and renamed the Federal Aviation Administration. The new Administration had a clear imperative from both the President and Congress: figure out air traffic control.

In the late 1960s, the FAA coined a new term: the National Airspace System ¹, a fully standardized, nationwide system of procedures and systems that would safely coordinate air traffic into the indefinite future. Automation of the NAS began with NAS Enroute Stage A, which would automate the ARTCCs that handled high-altitude aircraft on their way between terminals. The remit was more or less "just like SAGE but with the SATIN features," and when it came to contracting, the FAA decided to cut the middlemen and go directly to the hardware manufacturer: IBM.

The IBM 9020

It was 1967 by the time NAS Enroute Stage A was underway, nearly 20 years since SAGE development had begun. IBM would thus benefit from considerable advancements in computer technology in general. Chief among them was the 1965 introduction of the System/360. S/360 was a milestone in the development of the computer: a family of solid-state, microcoded computers with a common architecture for software and peripheral interconnection. S/360's chief designer, Gene Amdahl, was a genius of computer architecture who developed a particular interest in parallel and multiprocessing systems. Soon after the S/360 project, he left IBM to start the Amdahl Corporation, briefly one of IBM's chief competitors. During his short 1960s tenure at IBM, though, Amdahl contributed IBM's concept of the "multisystem."

A multisystem consisted of multiple independent computers that operated together as a single system. There is quite a bit of conceptual similarity between the multisystem and modern concepts like multiprocessing and distributed computing, but remember that this was the 1960s, and engineers were probing out the possibilities of computer-to-computer communication for the first time. Some of the ideas of S/360 multisystems read as strikingly modern and prescient of techniques used today (like atomic resource locking for peripherals and shared memory), while others are more clearly of their time (the general fact that S/360 multisystems tended to assign their CPUs exclusively to a specific task).

One of the great animating tensions of 1960s computer history is the ever-moving front between batch processing systems and realtime computing systems. IBM had its heritage manufacturing unit record data processing machines, in which a physical stack of punched cards was the unit of work, and input and output ultimately occurred between humans on two sides of a service window. IBM computers were designed around the same model: a "job" was entered into the machine, stored until it reached the end of the queue, processed, and then the output was stored for later retrieval. One could argue that all computers still work this way, it's just process scheduling, but IBM had originally envisioned job queuing times measured in hours rather than milliseconds.

The batch model of computing was fighting a battle on multiple fronts: rising popularity of time-sharing systems meant servicing multiple terminals simultaneously and, ideally, completing simple jobs interactively while the user waited. Remote terminals allowed clerks to enter and retrieve data right where business transactions were taking place, and customers standing at ticket counters expected prompt service. Perhaps most difficult of all, fast-moving airplanes and even faster-moving missiles required sub-second decisions by computers in defense applications.

IBM approached the FAA's NAS Enroute Stage A contract as one that required a real-time system (to meet the short timelines necessary in air traffic control) and a multisystem (to meet the FAA's exceptionally high uptime and performance requirements). They also intended to build the NAS automation on an existing, commodity architecture to the greatest extent possible. The result was the IBM 9020.

The 9020 is a fascinating system, exemplary of so many of the challenges and excitement of the birth of the modern computer. On the one hand, a 9020 is a sophisticated, fault-tolerant, high-performance computer system with impressive diagnostic capabilities and remarkably dynamic resource allocation. On the other hand, a 9020 is just six to seven S/360 computers married to each other with a vibe that is more duct tape and bailing wire than aerospace aluminum and titanium.

The first full-scale 9020 was installed in Jacksonville, Florida, late in 1967. Along with prototype systems at the FAA's experimental center and at Raytheon (due to the 9020's close interaction with Raytheon-built radar systems), the early 9020 computers served as development and test platforms for a complex and completely new software system written mostly in JOVIAL. JOVIAL isn't a particularly well-remembered programming language, based on ALGOL with modifications to better suit real-time computer systems. The Air Force was investing extensively in real-time computing capabilities for air defense and JOVIAL was, for practical purposes, an Air Force language.

It's not completely clear to me why IBM selected JOVIAL for enroute stage A, but we can make an informed guess. There were very few high-level programming languages that were suitable for real-time use at all in the 1960s, and JOVIAL had been created by Systems Development Corporation (the original SAGE software vendor) and widely used for both avionics and air defense. The SCC project, if it had been completed, would likely have involved rewriting large parts of SAGE in JOVIAL. For that reason, JOVIAL had been used for some of the FAA's earlier ATC projects including SATIN. At the end of the day, JOVIAL was probably an irritating (due to its external origin) but obvious choice for IBM.

More interesting than the programming language is the architecture of the 9020. It is, fortunately, well described in various papers and a special issue of IBM Systems Journal. I will simplify IBM's description of the architecture to be more legible to a modern reader who hasn't worked for IBM for a decade.

Picture this: seven IBM S/360 computers, of various models, are connected to a common address and memory bus used for interaction with storage. These computers are referred to as Compute Elements and I/O Control Elements, forming two pools of machines dedicated to two different sets of tasks. Also on that bus are something like 10 Storage Elements, specialized machines that function like memory controllers with additional features for locking, prioritization, and diagnostics. These Storage Elements provide either 131 kB or about 1 MB of memory each; due to various limitations the maximum possible memory capacity of a 9020 is about 3.4 MB, not all of which is usable at any given time due to redundancy.

At least three Compute Elements, and up to four, serve as the general-purpose part of the system where the main application software is executed. Three I/O Control Elements existed mostly as "smart" controllers for peripherals connected to their channels, the IBM parlance for what we might now call an expansion bus.

The 9020 received input from a huge number of sources (radar digitizers, teletypes at airlines and flight service stations, controller workstations, other ARTCCs). Similarly, it sent output to most of these endpoints as well. All of these communications channels, with perhaps the exception of the direct 9020-to-9020 links between ARTCCs, were very slow even by the standards of the time. The I/O Control Elements each used two of their high-speed channels for interconnection with display controllers (discussed later) and tape drives in the ARTCC, while the third high-speed channel connected to a multiplexing system called the Peripheral Adapter Module that connected the computer to dozens of peripherals in the ARTCC and leased telephone lines to radar stations, offices, and other ATC sites.

Any given I/O Control Element had a full-time job of passing data between peripherals and storage elements, with steps to validate and preprocess data. In addition to ATC-specific I/O devices, the Control Elements also used their Peripheral Adapter Modules to communicate with the System Console. The System Console is one of the most unique properties of the 9020, and one of the achievements of which IBM seems most proud.

Multisystem installations of S/360s were not necessarily new, but the 9020 was one of the first attempts to present a cluster of S/360s as a single unified machine. The System Console manifested that goal. It was, on first glance, not that different from the operator's consoles found on each of the individual S/360 machines. It was much more than that, though: it was the operator's console for all seven of them. During normal 9020 operation, a single operator at the system console could supervise all components of the system through alarms and monitors, interact with any element of the system via a teletypewriter terminal, and even manually interact with the shared storage bus for troubleshooting and setup. The significance of the System Console's central control was such that the individual S/360 machines, when operating as part of the Multisystem, disabled their local operator's consoles entirely.

One of the practical purposes of the System Console was to manage partitioning of the system. A typical 9020 had three compute elements and three I/O control elements, an especially large system could have a fourth compute element for added capacity. The system was sized to produce 50% redundancy during peak traffic. In other words, a 9020 could run the full normal ATC workload on just two of the compute elements and two of the I/O control elements. The remaining elements could be left in a "standby" state in which the multisystem would automatically bring them online if one of the in-service elements failed, and this redundancy mechanism was critical to meeting the FAA's reliability requirement. You could also use the out-of-service elements for other workloads, though.

For example, you could remove one of the S/360s from the multisystem and then operate it manually or run "offline" software. An S/360 operating this way is described as "S/360 compatibility mode" in IBM documentation, since it reduces the individual compute element to a normal standalone computer. IBM developed an extensive library of diagnostic tools that could be run on elements in standby mode, many of which were only slight modifications of standard S/360 tools. You could also use the offline machines in more interesting ways, by bringing up a complete ATC software chain running on a smaller number of elements. For training new controllers, for example, one compute element and one I/O control element could be removed from the multisystem and used to form a separate partition of the machine that operated on recorded training data. This partition could have its own assigned peripherals and storage area and largely operate as if it were a complete second 9020.

Multisystem Architecture

You probably have some questions about how IBM achieved these multisystem capabilities, given the immature state of operating systems design at the time. The 9020 used an operating system derived from OS/360 MVT, an advanced form of OS/360 with a multitasking capability that was state-of-the-art in the mid-1960s but nonetheless very limited and with many practical problems. Fortunately, IBM was not exactly building a general-purpose machine, but a dedicated system with one function. This allowed the software to be relatively simple.

The core of the 9020 software system is called the control program, which is similar to what we would call a scheduler today. During routine operation of the 9020, any of the individual computers might begin execution of the control program at any time—typically either because the computer's previous task was complete (along the lines of cooperative multitasking) or because an interrupt had been received (along the lines of preemptive multitasking). To meet performance and timing requirements, especially with the large number of peripherals involved, the 9020 extensively used interrupts which could either be generated and handled within a specific machine or sent across the entire multisystem bus.

The control program's main function is to choose the next task to execute. Since it can be started on any machine at any time, it must be reentrant. The fact that all of the machines have shared memory simplifies the control program's task, since it has direct access to all of the running programs. Shared memory also added the complexity that the control program has to implement locking and conflict detection to ensure that it doesn't start the same task on multiple machines at once, or start multiple tasks that will require interaction with the same peripheral.

You might wonder about how, exactly, the shared memory was implemented. The storage elements were not complete computers, but did implement features to prevent conflicts between simultaneous access by two machines, for example. By necessity, all of the memory management used for the multisystem is quite simple. Access conflicts were resolved by choosing one machine and making the other wait until the next bus cycle. Each machine had a "private" storage area, called the preferential storage area. A register on each element contained an offset added to all memory addresses that ensured the preferential storage areas did not overlap. Beyond that, all memory had to be acquired by calling system subroutines provided by the control program, so that the control program could manage memory regions. Several different types of memory allocations were available for different purposes, ranging from arbitrary blocks for internal use by programs to shared buffer areas that multiple machines could use to queue data for an I/O Control Element to send elsewhere.

At any time during execution of normal programs, an interrupt could be generated indicating a problem with the system (IBM gives the examples of a detection of high temperature or loss of A/C power in one of the compute elements). Whenever the control program began execution, it would potentially detect other error conditions using its more advanced understanding of the state of tasks. For example, the control program might detect that a program has exited abnormally, or that allocation of memory has failed, or an I/O operation has timed out without completing. All of these situations constitute operational errors, and result in the Control Program ceding execution to the Operational Error Analysis Program or OEAP.

The OEAP is where error-handling logic lives, but also a surprising portion of the overall control of the multisystem. The OEAP begins by performing self-diagnosis. Whatever started the OEAP, whether the control program or a hardware interrupt, is expected to leave some minimal data on the nature of the failure in a register. The OEAP reads that register and then follows an automated data-collection procedure that could involve reading other registers on the local machine, requesting registers from other machines, and requesting memory contents from storage elements. Based on the diagnosis, the OEAP has different options: some errors are often transient (like communications problems), so the OEAP might do nothing and simply allow the control program to start the task again.

On the other hand, some errors could indicate a serious problem with a component of the system, like a storage element that is no longer responding to read and write operations in its address range. In those more critical cases, the OEAP will rewrite configuration registers on the various elements of the system and then reset them—and on initialization, the configuration registers will cause them to assume new states in terms of membership in the multisystem. In this way, the OEAP is capable of recovering from "solid" hardware failures by simply reconfiguring the system to no longer use the failed hardware. Most of the time, that involves changing the failed element's configuration from "online" to "offline," and choosing an element in "online standby" and changing its configuration to "online." During the next execution of the control program, it will start tasks on the newly "online" element, and the newly "offline" element may as well have never existed.

The details are, of course, a little more complex. In the case of a failed storage element, for example, there's a problem of memory addresses. The 9020 multisystem doesn't have virtual memory in the modern sense, addresses are more or less absolute (ignoring some logical addressing available for specific types of memory allocations). That means that if a storage element fails, any machines which have been using memory addresses within that element will need to have a set of registers for memory address offsets reconfigured and then execution reset. Basically, by changing offsets, the OEAP can "remap" the memory in use by a compute or I/O control element to a different storage element. Redundancy is also built in to the software design to make these operations less critical. For example, some important parts of memory are stored in duplicate with an offset between the two copies large enough to ensure that they will never fall on the same physical storage element.

So far we have only really talked about the "operational error" part, though, and not the "analysis." In the proud tradition of IBM, the 9020 was designed from the ground up for diagnosis. A considerable part of IBM's discussion of the architecture of the Control Program, for example, is devoted to its "timing analysis" feature. That capability allows the Control Program to commit to tape a record of when each task began execution, on which element, and how long it took. The output is a set of start and duration times, with task metadata, remarkably similar to what we would now call a "span" in distributed tracing. Engineers used these records to analyze the performance of the system and more accurately determine load limits such as the number of in-air flights that could be simultaneously tracked. Of course, details of the time analysis system remind us that computers of this era were very slow: the resolution on task-start timestamps was only 1/2 second, although durations were recorded at the relatively exact 1/60th of a second.

That was just the control program, though, and the system's limited ability to write timing analysis data (which, even on the slow computers, tended to be produced faster than the tape drives could write it and so had to fit within a buffer memory area for practical purposes) meant that it was only enabled as needed. The OEAP provided long-term analysis of the performance of the entire machine. Whenever the OEAP was invoked, even if it determined that a problem was transient or "soft" and took no action, it would write statistical records of the nature of the error and the involved elements. When the OEAP detected an unusually large number of soft errors from the same physical equipment, it would automatically reconfigure the system to remove that equipment from service and then generate an alarm.

Alerts generated by the OEAP were recorded by a printer connected to the System Console, and indicated by lights on the System Console. A few controls on the System Console allowed the operator manual intervention when needed, for example to force a reconfiguration.

One of the interesting aspects of the OEAP is where it runs. The 9020 multisystem is truly a distributed one in that there is no "leader." The control program, as we have discussed, simply starts on whichever machine is looking for work. In practice, it may sometimes run simultaneously on multiple machines, which is acceptable as it implements precautions to prevent stepping on its own toes.

This model is a little more complex for the OEAP, because of the fact that it deals specifically with failures. Consider a specific failure scenario: loss of power. IBM equipped each of the functional components of the 9020 with a battery backup, but they only rate the battery backup for 5.5 seconds of operation. That isn't long enough for a generator to reliably pick up the load, so this isn't a UPS as we would use today. It's more of a dying gasp system: the computer can "know" that it has lost power and continue to operate long enough stabilize the state of the system for faster resumption.

When a compute element or I/O control element loses power, an interrupt is generated within that single machine that starts the OEAP. The OEAP performs a series of actions, which include generating an interrupt across the entire system to trigger reconfiguration (it is possible, even likely given the physical installations, that the power loss is isolated to the single machine) and resetting task states in the control program so that the machine's tasks can be restarted elsewhere. The OEAP also informs the system console and writes out records of what has happened. Ideally, this all completes in 5.5 seconds while battery power remains reliable.

In the real world, there could be problems that lead to slow OEAP execution, or the batteries could fail to make it for long enough, or for that matter the compute element could encounter some kind of fundamentally different problem. The fact that the OEAP is executing on a machine means that something has gone wrong, and so until the OEAP completes analysis, the machine that it is running on should be considered suspect. The 9020 resolves this contradiction through teamwork: beginning of OEAP execution on any machine in the total system generates an interrupt that starts the OEAP on other machines in a "time-down" mode. The "time-down" OEAPs wait for a random time interval and then check the shared memory to see if the original OEAP has marked its execution as completed. If not, the first OEAP to complete its time-down timer will take over OEAP execution and attempt to complete diagnostics from afar. That process can, potentially, repeat multiple times: in some scenario where two of the three compute elements have failed, the remaining third element will eventually give up on waiting for the first two and run the OEAP itself. In theory, someone will eventually diagnose every problem. IBM asserts that system recovery should always complete within 30 seconds.

Let's work a couple of practical examples, to edify our understanding of the Control Program and OEAP. Say that a program running on a Compute Element sets up a write operation for an I/O Control Element, which formats and sends the data to a Peripheral Adapter Module which attempts to send it to an offsite peripheral (say an air traffic control tower teleprinter) but fails. A timer that tracks the I/O operation will eventually fail, triggering the OEAP on the I/O control element running the task. The OEAP reads out the error register on its new home, discovers that it is an I/O problem related to a PAM, and then speaks over the channel to request the value of state registers from the PAM. These registers contain flags for various possible states of peripheral connections, and from these the OEAP can determine that sending a message has failed because there was no response. These types of errors are often transient, due to telephone network trouble or bad luck, so the OEAP increments counters for future reference, looks up the application task that tried to send the message and changes its state to incomplete, clears registers on the PAM and I/O control element, and then hands execution back to the Control Program. The Control Program will most likely attempt to do the exact same thing over again, but in the case of a transient error, it'll probably work this time.

Consider a more severe case, where the Control Program starts a task on a Compute Element that simply never finishes. A timer runs down to detect this condition, and an interrupt at the end of the timer starts the Control Program, which checks the state and discovers the still-unfinished task. Throwing its hands in the air, the Control Program sets some flags in the error register and hands execution to the OEAP. The OEAP starts on the same machine, but also interrupts other machines to start the OEAP in time-down mode in case the machine is too broken to complete error handling. It then reads the error register and examines other registers and storage contents. Determining that some indeterminate problem has occurred with the Compute Element, the OEAP triggers what IBM confusingly calls a "logout" but we might today call a "core dump" (ironically an old term that was more appropriate in this era). The "logout" entails copying the contents of all of the registers and counters to the preferential storage area and then directing, via channel, one of the tape drives to write it all to a tape kept ready for this purpose—the syslog of its day. Once that's complete, the OEAP will reset the Compute Element and hand back to the Control Program to try again... unless counters indicate that this same thing has happened recently. In that case, the OEAP will update the configuration register on the running machine to change its status to offline, and choose a machine in online-standby. It will write to that machine's register, changing its status to online. A final interrupt causes the Control Program to start on both machines, taking them into their new states.

Lengthy FAA procedure manuals described what would happen next. These are unfortunately difficult to obtain, but from IBM documentation we know that basic information on errors was printed for the system operator. The system operator would likely then use the system console to place the suspicious element in "test" mode, which completely isolates it to behave more or less like a normal S/360. At that point, the operator could use one of the tape drives attached to the problem machine to load IBM's diagnostic library and perform offline troubleshooting. The way the tape drives are hooked up to specific machines is important; in fact, since the OEAP is fairly large, it is only stored in one copy on one Storage Element. The 9020 requires that one of the tape drives always have a "system tape" ready with the OEAP itself, and low-level logic in the elements allows the OEAP to be read from the ready-to-go system tape in case the storage element that contains it fails to respond.

A final interesting note about the OEAP is a clever optimization called "problem program mode." During analysis and handling of an error, the OEAP can decide that the critical phase of error handling has ended and the situation is no longer time sensitive. For example, the OEAP might decide that no action is required except for updating statistics, which can comfortably happen with a slight delay. These lower-priority remaining tasks can be added to memory as "normal" application tasks, to be run by the Control Program like any other task after error handling is complete. Think of it as a deferral mechanism, to avoid the OEAP locking up a machine for any longer than necessary.

For the sake of clarity, I'll note again an interesting fact by quoting IBM directly: "OEAP has sole responsibility for maintaining the system configuration." The configuration model of the 9020 system is a little unintuitive to me. Each machine has its own configuration register that tells it what its task is and whether it is online or offline (or one of several states in between like online-standby). The OEAP reconfigures the system by running on any one machine and writing the configuration registers of both the machine it's running on, and all of the other machines via the shared bus. Most reconfigurations happen because the OEAP has detected a problem and is working around it, but if the operator manually reconfigures the system (for example to facilitate testing or training), they also do so by triggering an interrupt that leads the Control Program to start the OEAP. The System Console has buttons for this, along with toggles to set up a sort of "main configuration register" that determines how the OEAP will try to set up the system.

The Air Traffic Control Application

This has become a fairly long article by my norms, and I haven't even really talked about air traffic control that much. Well, here it comes: the application that actually ran on the 9020, which seems to have had no particular name, besides perhaps Central Computing Complex (although this seems to have been adopted mostly to differentiate it from the Display Complex, discussed soon).

First, let's talk about the hardware landscape of the ARTCC and the 9020's role. An ARTCC handles a number of sectors, say around 30. Under the 9020 system, each of these sectors has three controllers associated with it, called the R, D, and A controllers. The R controller is responsible for monitoring and interpreting the radar, the D controller for managing flight plans and flight strips, and the A controller is something of a generalist who assists the other two. The three people sit at something like a long desk, made up of the R, D, and A consoles side by side.

The R console is the most recognizable to modern eyes, as its centerpiece is a 22" CRT plan-view radar display. The plan-view display (PVD) of the 9020 system is significantly more sophisticated than the SAGE PVD on which it is modeled. Most notably, the 9020 PVD is capable of displaying text and icons. No longer does a controller use a light gun to select a target for a teleprinter to identify; the "data blocks" giving basic information on a flight were actually shown on the PVD next to the radar contact. A trackball and a set of buttons even allowed the controller to select targets to query for more information or update flight data. This was quite a feat of technology even in 1970, and in fact one that the 9020 was not capable of. Well, it was actually capable of it, but not initially.

The original NAS stage A architecture separated the air traffic control data function and radar display function into two completely separate systems. The former was contracted to IBM, the latter to Raytheon, due to their significant experience building similar systems for the military. Early IBM 9020 installations sat alongside a Raytheon 730 Display Channel, a very specialized system that was nearly as large as the 9020. The Display Channel's role was to receive radar contact data and flight information in digital form from the 9020, and convert it into drawing instructions sent over a high-speed serial connection to each individual PVD. A single Display Channel was responsible for up to 60 PVDs. Further complicating things, sector workstations were reconfigurable to handle changing workloads. The same sector might be displayed on multiple PVDs, and where sectors met, PVDs often overlapped so the same contact would be visible to controllers for both sectors. The Display Channel had a fairly complex task to get the right radar contacts and data blocks to the right displays, and in the right places.

Later on, the FAA opted to contract IBM to build a slightly more sophisticated version of the Display Channel that supported additional PVDs and provided better uptime. To meet that contract, IBM used another 9020. Some ARTCCs thus had two complete 9020 systems, called the Central Computer Complex (CCC) and the Display Channel Complex (DCC).

The PVD is the most conspicuous part of the controller console, but there's a lot of other equipment there, and the rest of it is directly connected to the 9020 (CCC). At the R controller's position, a set of "hotkeys" allow for quickly entering flight data (like new altitudes) and a computer readout device (CRD), a CRT that displays 25x20 text for general output. For example, when a controller selects a target on the PVD to query for details, that query is sent to the 9020 CCC which shows the result on the R controller's CRD above the PVD.

At the D controller's position, right next door, a large rack of slots for flight strips (small paper strips used to logically organize flight clearances, still in use today in some contexts) surrounds the D controller's CRD. The D controller also has a Computer Entry Device, or CED, a specialized keyboard that allows the D controller to retrieve and update flight plans and clearances based on requests from pilots or changes in the airspace situation. To their right, a modified teleprinter is dedicated to producing the flight strips that they arrange in front of them. Flight strips are automatically printed when an aircraft enters the sector, or when the controller enters changes. The A controller's position to the right of the flight strip printer is largely the same as the D controller's position, with another CRD and CED that operate independently from the D controller's—valuable during peak traffic.

While controller consoles are the most visible peripherals of the system, they're far from the only ones. Each 9020 system had an extensive set of teletypewriter circuits. Some of these were local; for example, the ATC supervisor had a dedicated TTY where they could not only interact with flight data (to assist a sector controller for example) but also interact with the status of the NAS automation itself (for example to query the status of a malfunctioning radar site and then remove it from use for PVDs).

Since the 9020 was also the locus of flight planning, TTYs were provided in air traffic control towers, terminal radar facilities, and even the dispatch offices of airlines. These allowed flight plans to be entered into the 9020 before the aircraft was handed off to enroute control. Flight service stations functioned more or less as the dispatch offices for general aviation, so they were similarly equipped with TTYs for flight plan management. In many areas, military controllers at air defense sectors were also provided with TTYs for convenient access to flight plans. Not least of all, each 9020 had high-speed leased lines to its neighboring 9020s. Flights passing from one ARTCC to the next had their flight strip "digitally passed" by transmission from one 9020 to the next.

A set of high-speed line printers connected to the 9020 printed diagnostic data as well as summary and audit reports on air traffic. Similar audit data, including a detailed record of clearances, was written to tape drives for future reference.

To organize the whole operation, IBM divided the software architecture of the system into the "supervisor state" and the "problem state." These are reasonably analogous to kernel and user space today, and "problem" is meant as in "the problem the computer solves" rather than "a problem has occurred." The Control Program and OEAP run in the supervisor state, everything else runs after the Control Program has set up a machine in the Problem State and started a given program.

IBM organized the application software into five modules, which they called the five Programs. These are Input Processing, Flight Processing, Radar Processing, Output Processing, and Liaison Management. Most of these are fairly self-explanatory, but the list reveals the remarkably asynchronous design of the system. Consider an example, we'll say a general aviation flight taking off from an airport inside of one of the ARTCC's sectors.

The pilot first contacts a Flight Service Station, which uses their TTY to enter a flight plan into the 9020. Next, the pilot interacts with the control tower, which in the process of giving a takeoff clearance uses their TTY to inform the 9020 that the flight plan is active. They may also update the flight plan with the aircraft's planned movements shortly after takeoff, if they have changed due to operating conditions. The Input Processing program handles all of these TTY inputs, parsing them into records stored on a Storage Element. In case any errors occur, like an invalid entry, those are also written to the Storage Element, where the Output Processing program picks them up and sends an appropriate message to the originating TTY. IBM notes that there were, as originally designed, about 100 types of input messages parsed by the input processing program.

As the aircraft takes off, it is detected by a radar site (such as a Permanent System radar or Air Route Surveillance Radar) which digitally encodes the radar contact (a Raytheon system) for transmission to the 9020. The Radar Processing program receives these messages, converts radial radar coordinates to the XY plane used by the system, correlates contacts with similar XY positions from multiple radar sites into a single logical contact, and computes each contact's apparent heading and speed to extrapolate future positions. Complicating things, the 9020 went into service during the development of secondary surveillance radar, also known as the transponder system ². On appropriately equipped aircraft, the transponder provides altitude. The Radar Processing system makes an altitude determination on each aircraft, a slightly more complicated task than you might expect as, at the time, only some radar systems and some transponders provided altitude information. The Radar Processing program thus had to track if it had altitude information at all and, if so, where from. In the mean time, the Radar Processing program tracked the state of the radar sites and reported any apparent trouble (such as loss of data or abnormal data) to the supervisor.

The Flight Processing program periodically evaluates all targets from the Radar Processing program against all filed flight plans, correlating radar targets with filed flight plans, calculating navigational deviations, and predicting future paths. Among other outputs, the Flight Processing program generated up-to-date flight strips for each aircraft and predicted their arrival times at each flight plan fix for controller's planning purposes. The Flight Processing program hosted a set of rules used for safety protections, such as separation distances. This capability was fairly minimal during the 9020's original development, but was enhanced over time.

The Output Processing program had two key roles. First, it handled data that was specifically queued for it because of a reactive need to send data to a given output. For example, if someone made a data entry error or a controller queried for a specific aircraft's flight plan, the Input Processing program placed the resulting data in memory, where the Output Processing program would "find it" to format and send to the correct device. The Output Processing program also continuously prepared common outputs like flight data blocks and radar station status messages that were formatted once to a common memory buffer to be sent to many devices in bulk. For example, a new flight strip for an aircraft would be formatted and stored once, and then sent in sequence to every controller position with a relation to that aircraft.

Legacy

The 9020 is just one corner of the evolution of air traffic control during the 1960s and 1970s, a period that also saw the introduction of secondary radar for civilian flights and the first effort to automate the role of flight service stations. These topics quickly spiral out into others: unlike the ARTCCs of the time, the flight service stations dealt extensively with weather and interacted with both FAA and National Weather Service teletype networks and computer systems. An early effort to automate the flight service function involved the use of a teletext system originally developed for agricultural use as a "flight briefing terminal." That wasn't the agricultural teletext system in Kentucky that I discussed, but a different one, in Kansas. Fascinating things everywhere you look!

This article has already become long, though, and so we'll have to save plenty for later. To round things out, let's consider the fate of the 9020. SAGE is known not only for its pioneering role in the computing art, but because of its remarkably long service life, roughly from 1958 to 1984. The 9020 was almost 20 years younger than SAGE, and indeed outlived it, but not by much. In 1982, IBM announced the IBM 3083, a newer implementation of the Enhanced S/370 architecture that was directly descended from S/360 but with greatly improved I/O capabilities. In 1986, the FAA accepted a new 3083-based system called "HOST." Over the following three years, all of the 9020 CCCs were replaced by HOST systems.

The 9020 was not to be forgotten so easily, though. First, the HOST project was mostly limited to hardware modernization or "rehosting." The HOST 3083 computers ran most of the same application code as the original 9020 system, incorporating many enhancements made over the intervening decades.

Second, there is the case of the Display Channel Complex. Once again, because of the complexity of the PVD subsystem the FAA opted to view it as a separate program. While an effort was started to replace the 9020 DCCs alongside the 9020 CCCs, it encountered considerable delays and was ultimately canceled. The 9020 DCCs remained in service controlling PVDs until the ERAM Stage A project replaced the PVD system entirely in the 1990s.

While IBM's efforts to market the 9020 overseas generally failed, a 9020 CCC system (complete with simplex test machine) was sold to the UK Civil Aviation Authority for use in the London Air Traffic Centre. This 9020 remained in service until 1990, and perhaps because of its singularity and unusually long life, it is better remembered as a historic object. There are photos.

Some years ago, I had a frustrating and largely fruitless encounter with the politics of policing. As a member of an oversight commission, I was particularly interested in the regulation of urban surveillance. The Albuquerque Police Department, for reasons good and bad, has often been an early adopter of surveillance technology. APD deployed automated license plate readers, mounted on patrol cars and portable trailers, in 2013. Initially, the department kept a six-month history of license plate data. For six months, police could retrospectively search the database to reconstruct a vehicle, or person's, movements—at least, those movements that happened near select patrol cars and "your speed is" trailers. Lobbying by the American Civil Liberties Union and public pressure on APD and city council lead to a policy change to retain data for only 14 days, a privacy-preserving measure that the ACLU lauded as one of the best ALPR policies in the nation.

Today, ALPR is far more common in Albuquerque. Lowering costs and a continuing appetite for solving social problems with surveillance technology means that some parts of the city have ALPR installed at every signalized intersection—every person's movements cataloged at a resolution of four blocks. The data is retained for a full year. Some of it is offered, as a service, to law enforcement agencies across the country.

One of the most frustrating parts of the mass surveillance debate is the ability of law enforcement agencies and municipal governments to advance wide-scale monitoring programs, weather the controversy, and then ratchet up retention and sharing after public attention fades. For years, expansive ALPR programs spread through most American cities with little objection. In my part of the country, it seemed that the controversy over ALPR had been completely forgotten until one particularly significant ALPR vendor—Flock Safety—started repeatedly stepping in long-festering controversies with such wild abandon that they are clearly either idiots or entirely unconcerned about public perception.

I try not to be too cynical but I am, unfortunately, more inclined to the latter. Companies like Flock know that they are in treacherous territory, morally and legally. They know that their customers are mostly governments or organizations with elected leaders that are subject to popular opinion. They know that helping Texas law enforcement track down abortion seekers in other states is a "bad look." They know all of these things, but they do not particularly care. They don't have to care: decades of incipient corruption, legal and political maneuvering, and the routine inefficacy of municipal politics has created an environment where public opinion doesn't matter.

I can't definitely tell you where public opinion lies on ALPR, although it seems like the average person might be mildly in support. From at least my experience, in my corner of the world, I will tell you this: it doesn't matter. Police departments and the means by which they purchase and field technology are so isolated from the political process that it is extremely difficult to imagine a scenario where voters could affect change. Year by year, city by city, the police become more dug in. Law enforcement agencies across the country have found that the most straightforward way to address privacy concerns around surveillance technology is to keep the department's purchase and deployment of that technology a secret. Most city governments at least passively support this approach. The vendors of surveillance systems facilitate, support, and even demand secrecy through their contract terms.

More recently, Las Vegas and the Bay Area have offered a model even more opaque to public scrutiny: law enforcement surveillance technology is simply purchased by wealthy private donors, almost invariably from the software industry, and then either the systems or their use are donated to the city. If carefully designed, these programs can be completely exempt from public information rules. They can take the form, for example, of a business association that runs its own private surveillance state, involving the public and ostensibly accountable police only when an arrest is made. We are privatizing mass surveillance.

Flock continues to generate enormous press, mostly on the back of persistent investigation by 404 Media. More recently, security researchers have published significant defects in the design of Flock's technology that can make the original video publicly accessible. Just about every time that someone looks into Flock, the company turns out to be less ethical, the users less concerned about compliance, the design of the system itself less competent than charitable viewers had assumed.

I'm trying not to be a doomer for Christmas, but I am sometimes frustrated with Flock coverage because it can miss the entire history of this issue. I think that a more contextually complete discussion of urban surveillance could be useful. And I am sitting in a coffee shop, in a trendy part of town, looking out the window at a PTZ camera on the side of a traffic light. That camera, I know, is owned and operated by APD's Real Time Crime Center (RTCC). Between the police department itself, city agencies that participate in the RTCC, and businesses that volunteer real-time access to their surveillance, there are thousands of others like it. Courtesy of a transit station, there are at least a dozen within my view.

Whether or not this is a good or bad idea, whether or not it is effective in reducing crime, whether or not it will be leveraged against political opposition; these are tricky questions. I suppose what worries me is that it feels like hardly anyone is asking them any more. It took Flock's remarkable ability to step on rakes and the apparent victory of fascism in national politics for anyone to remember that the construction of ubiquitous surveillance is a project that started many years ago, and that has proceeded largely unhindered ever since.

Domestic Signals Intelligence

Within the tech community, there has historically been much attention to the ability to track people by passively observing Bluetooth traffic. This technique has been widely used, both in commercial and government applications. There are popular "smart city" street lighting systems, for example, that allow every street light to passively collect signatures of the people passing underneath it. To my knowledge, these techniques are not actually very widely used by law enforcement. There are, perhaps, two reasons: one of mechanisms of government, the other of countermeasures.

First, "smart city" data collection systems are usually funded and deployed by municipal works or environmental departments. While police could make arrangements for access to that data, those arrangements would require the kind of inter-agency Memoranda of Understanding that tend to lead to far more public scrutiny than police acquisitions of surveillance products. Besides, since they aren't deployed for law enforcement purposes, they're often not useful sources for the areas that law enforcement find most interesting: higher-crime areas that are usually lower-income and, thus, less likely to have working street lights at all—much less "smart" ones. Besides, smartphones have widely adopted randomization of Bluetooth and WiFi identifiers, and protocol revisions have reduced the number of identifiers transmitted in plaintext. Passive signals intelligence just doesn't work as well as it used to, at least at the capability level of a municipality.

Talking about Bluetooth and WiFi on phones does raise the question of the "phone" part, the cellular interface, which is targeted by the family of devices often known as "Stingrays" after the trademark of a particular manufacturer. Fortunately, improvements in the security design of cellular protocols is making these less effective over time. Unfortunately, the technology continues to advance, sometimes undoing the improvements of newer GSM revisions. Federal and local law enforcement continue to purchase and use these devices, largely in secret, benefiting from a particular model of federal ownership/local use that makes it especially difficult to get a police department to even confirm or deny that they have ever deployed them. "Stingrays" or IMSI surveillance is mostly a shadow world of rumors and carefully worded non-denials. Despite technical measures against them, they are clearly still in use, and thus clearly still useful.

Optics

Out in the rest of the world, visual surveillance is more salient than radio. Early rollouts of police-operated surveillance, dating back to at least the 1970s, generated some controversy over privacy implications. Two things have since happened: first, law enforcement have relied on an increasing number of public-private partnerships and commercial vendors to gain access to surveillance without directly owning it. Second, police video surveillance has largely been normalized, and no longer faces much opposition or even public notice.

One of the interesting changes here is one of visibility: police video surveillance has often edged in and out of public awareness to fit the politics. In periods of pro-privacy, anti-surveillance sentiment, departments rely more on voluntary arrangements for access to cameras installed by others. In periods of pro-police, anti-crime sentiment, departments install cameras with flashing lights and police badges. Both types tend to persist after the next change in the tide.

The public usually knows little about these systems, a result of intentional opacity by police departments and a general lack of interest by the press. That leads to a lot of confusion. Where I live, we do have an extensive network of police-operated cameras on intersection traffic signal arms. And yet, if you ask the average person to identify a police surveillance camera, they will point to the camera for the traffic signal's video-based lane occupancy sensing every single time. That's not a police camera, it's barely even a camera as the video is rarely retained. All of these people, it seems, have worked themselves into a sort paranoia where they think that every camera is an eye of the police. Well, the police are watching them, from about ten feet over. The "speed dome" PTZ cameras get so much less attention, perhaps because they are usually mounted on the pole further from sightline, or perhaps because they are of a type more common for commercial surveillance systems that we have learned to simply ignore.

Video surveillance is an interesting topic to me, philosophically. I am largely unconcerned with the privacy implications of most video surveillance installations (such as the one on my own house) because, historically, the video was recorded locally and generally reviewed only when there was a specific reason. The simple fact that reviewing large amounts of video is so time consuming meant that the pervasive surveillance potential of video surveillance was, at one time not so long ago, quite limited.

Of course, the age of the computer has somewhat changed that situation. There are two phenomenon of the automation of surveillance that I think should be considered separately: first, machine vision has improved to an extent that computers can automatically process surveillance video to extract events and identities. Second, the appified, everything-social-media attitude of consumer products creates new dynamics in access to surveillance data, and those dynamics are spreading upwards into the commercial segment.

Machine Vision

Historically, much of the attention to video pervasive surveillance has centered around facial recognition. Facial recognition has indeed been applied to video surveillance for years, but I think that the average person vastly overestimates how effective and widely used facial recognition is.

The vast majority of currently installed surveillance cameras do not produce video of sufficient quality for facial recognition. That has less to do with the quality of the video itself (although that is poorer than you think for most real systems) and more to do with the way that surveillance cameras are used and installed. Most cameras are positioned high up with wide coverage of a room; this perspective is ideal for reconstructing a series of events but just about the worst case for facial recognition. For most surveillance cameras, human faces are small and at indirect angles. There is little geometry that you can extract from a face that is about ten pixels wide and subjected to aggressive h264 compression, which is how most surveillance video comes out.

Practical facial recognition systems involve cameras installed specifically for that purpose, roughly at eye level where they will get close-up, straight-on images of people who pass by. Next time you visit a bank branch, look by the exit doors for a conspicuously thick height strip. Height strips by exit doors were invented to allow clerks to give police a more accurate description of a robber, but they have since evolved to serve largely as subterfuge. Somewhere around 5' 6", you will notice a small hole, and behind that hole is a camera. Several manufacturers offer these and they seem very popular in financial services.

Kroger is more to the point: they've just been installing dome cameras right at eye height on their exit doors, for at least a decade.

When discussing surveillance, it's important to remember that the vast majority of real-world video surveillance systems are old, inexpensive, and poorly maintained. Even where cameras are installed specifically for a good view of faces, there probably isn't any facial recognition in use, most of the time. Facial recognition products are expensive and don't currently manifest many benefits unless the organization is large enough to have a security department to work with the resulting data, which requires a degree of operational maturity beyond most surveillance users (e.g. gas stations).

All of that said, there are plenty of real-world facial recognition deployments. Rite Aid, for example, prominently rolled out facial recognition to flag known shoplifters at their stores... a rollout that went so poorly that it lead to a lawsuit and an FTC settlement including the end of the facial recognition program and a five-year moratorium on further attempts. This is not to say that facial recognition on video surveillance isn't legal (although in some states it isn't or at least requires a lot of disclosure), but there is definitely a degree of legal and reputational hazard involved.

The ACLU has periodically conducted call-around surveys on use of facial recognition. The most notable trend is that most large chains now refuse to talk about it. I could be wrong, but my experience with corporate communications behavior leads to me to interpret a refusal to comment along these lines: Home Depot, for example, is a very large company that will have various initiatives underway, and either confirming or denying their use of facial recognition would probably be wrong in some cases and expose them to compliance or legal risk in others. I would bet good money that Home Depot has some facial recognition technology deployed at some locations, but knowing how slowly security technology tends to roll out in that kind of company and how complicated the legal and compliance situation can become, it's probably limited. They are probably acutely aware of the controversy surrounding this type of surveillance and, given the example of Rite Aid, the ways a rollout could go wrong. That means that surveillance will spread slowly.

But it will spread. At this point, I think it is inevitable that facial recognition will become widely used in video surveillance. I just think the point at which "facial recognition is everywhere" remains some years away, due to all the normal reasons: technical limitations, slow-moving bureaucracies, and a somewhat complex and unclear regulatory situation. It will inevitably happen, for the same reasons as well: aggressive sales by facial recognition vendors.

There are some sectors where facial recognition is very common, although I don't get the impression that retail is one of these yet. Casinos, for example—some of the larger Las Vegas casinos have reportedly had facial recognition systems in use for decades, and Nevada law is very permissive of them. Casinos are, of course, pretty much ideal users. Large institutions with a lot of financial risk and large, sophisticated security departments. Few other businesses outside of Target can compete with the size and sophistication of casino security departments. There's a whole lot of money flying around, and they can spend some of it on expensive per-camera licensing without much leadership objection.

License Plate Reading

Popular attention to facial recognition has mostly fallen away as industry and media focus has shifted to another application of machine vision that is, it turns out, a whole lot easier: license plates. License plates are designed for readability, and most states use retroreflective paints that give you an absolutely beautiful high-contrast image under coaxial (i.e. mounted alongside the camera) infrared illumination. There's not much that is easier to read by machine vision. Automated license plate readers have been available for quite a long time: US CBP had an experimental ALPR installation at a Texas border crossing in 1994. That system was actually deemed a failure and removed, but technology improved and there were permanent installations at larger border crossings by the end of the 1990s.

For a long time, the dominant vendor of ALPR equipment in the US was Motorola. Motorola's product line remains popular for vehicle-mounted systems, but the high price of the cameras, controllers, and software package had a side benefit of limiting the pervasiveness of ALPR. The equipment was just too expensive to put up all over the place.

In Albuquerque, for example, the ALPR program long consisted of Motorola systems mounted on portable "your speed is" trailers. The portable nature of these setups made the expense more worthwhile, and besides, portability has its own utilities: an APD detective once told me, for example, of how they would leave ALPR trailers in front of the houses of people suspected to lead criminal gangs. While there was value in the intelligence collection, the main motivation was intimidation: while you might call the "your speed is" trailers a concealed system, they're not all that subtle, and one supposes that most vehicle-based criminals (the main kind here) are aware that they function as the eyes of the police.

At some point, in response to growing budgets or lowered costs I'm not sure, APD began installing fixed Motorola ALPR systems on the light arms of major intersections. I know of around a dozen installations of this type in Albuquerque, which is the beginning of a widespread capability to monitor public movements but not exactly the dystopian pervasive surveillance of Minority Report.

ALPR works pretty well, but it is not perfect. Cameras need to be installed with fairly narrow optics aimed at the right spot, and infrared illumination makes reading far more reliable. Speaking of Las Vegas, I used to use a certain casino parking garage with an ALPR-based payment system with some regularity. It printed the license plate, as read by the ALPR, on the parking ticket, which is why I know that it was almost comically inept at reading my very legible California plate. It always got about half the characters wrong. I have gotten much better results in my own home experiments with budget equipment, so I figure that system must have been very poorly installed or maintained, but I'm sure there are plenty of others out there just like it.

That's the tricky thing about video surveillance, from the "blue team" side of the house: people don't tend to pay a lot of attention to it until there's been an incident, at which point they find out that the lens has had mud on it for the last three months (a much bigger problem with ALPR cameras that used to be mounted pretty low to the ground for a better look angle). I bring this up because I think that people tend to vastly overestimate the quality of real-world video surveillance, and I like to take every opportunity to remind people that the main failure case of retail video surveillance used to be failure to replace the continuous-loop tape cassette before it was completely demagnetized by repeated recording. Now, in 2025, the continuous-loop tape cassettes are pretty much gone, but maintenance practices haven't improved. Lots of the cameras you see in public only barely work or don't work at all. So it goes.

Flock

In 2017, though, a VC-backed (and specifically YCombinator) company called Flock Safety introduced a bold new idea to ALPR: a Silicon Valley sales model. Flock's system is built to be low-cost, and the sensors are smaller, simpler, and cheaper than Motorola's. I suppose they might be less effective as a result, but a reduced "catch" rate doesn't really detract from mass-surveillance ALPR installations that much. Flock has also greatly expanded their customer base, emphasizing sales to private organizations as well as law enforcement and government. Speaking of Home Depot, for example, Home Depot seems to have installed their own Flock cameras in all of their parking lots. Lowes Home Improvement has done the same.

I wanted to know more definitely how much Flock systems cost, because I suspected they were making significant inroads just through low pricing. It's a little tricky to say definitively because Flock is a "call for quote" kind of company and I think they offer contracts on different price bases. Scouring contracting documents, meeting notes, etc., it seems like a "typical" cost for a Flock camera is around $4,000 with about $3,000 a year in per-camera software licensing fees.

That might seem expensive but it compares well to the five-figure prices I have heard associated with Motorola systems, especially since the Flock offering is more "white glove" with installation and maintenance packaged. Motorola systems are usually purchased through an integrator who adds their own considerable margin.

Flock also designs their cameras to be amenable to solar power, which radically reduces install costs compared to Motorola systems that usually need a utility worker out to splice power from a streetlight. More even than a price reduction, it makes Flock cameras much more available to organizations like HOAs that control territory in a sense but do not have the full bucket truck or utility work order capabilities of a municipal government.

Another recent innovation in ALPR is less traceable to Flock but certainly seems associated with them: flexible funding sources. Police departments have limited budgets with which to acquire new technology, and technology vendors have to compete with other budget priorities like salaries, vehicles, and black-on-black tactical vinyl jobs for those vehicles. ALPR seems especially attractive for public-private partnership mechanisms, so there are a lot of Flock installations that were funded by business associations, HOAs, neighborhood associations, and other "indirect" sources. Some of these systems are owned and operated by the police with only the funding donated, others are owned and operated by the private group that paid for them. This can result in curious deployment decisions: sometimes the lowest-crime neighborhoods are the most replete with ALPR, as they tend to be wealthier and more politically organized communities with the wherewithall to put up the money.

The most important thing to understand about Flock, though, is that it has built on Amazon's concept of "Ring neighbors" to build a sort of nationwide, ALPR-centric Nextdoor. Flock customers can basically check a box that allows other Flock customers to access data from their sensors, and of course Flock strongly encourages users to turn sharing on. While there are some audit and access controls available on Flock data sharing, they seem like pretty minimal efforts that are often ignored.

Flock sharing has generated a lot of press, especially with some dramatic examples like use by a Texas sheriff to locate an abortion patient and use by ICE/CBP to track suspects. These are both examples that raise one of the most alarming aspects of the Flock situation: many states and municipalities have laws in place that limit or at least monitor collaboration of local police with other police agencies and federal law enforcement. Some people find this surprising, but it's important to understand that the United States is a republic of nominally independent governments. Laws, policies, and priorities can vary greatly from jurisdiction to jurisdiction. There is a specific historical thread, related to the tracing of escaped slaves, that has made resource sharing between different law enforcement agencies a known area of moral and legal treachery for a very long time.

And yet, it turns out, most Flock customers seem to have sharing turned on, possibly entirely without their knowledge. There are now multiple well-established cases of local law enforcement agencies violating state laws by having data sharing with ICE/CBP enabled. It is possible for Flock users to turn off or restrict sharing, but I think a lot of them honestly don't know that. Some state Attorneys General have ordered Flock users to disable sharing, some have restricted or banned Flock products entirely, but in general it's a very messy situation. It appears that a lack of care by law enforcement and other Flock customers, facilitate and no doubt encouraged by Flock's motivation towards "network effect" lock-in, has resulted in widespread and brazen violation of privacy laws that is only now making its way to the courts.

In other words, the tech industry happened.

Acoustics

I will not spend much time here discussing wide-area acoustic surveillance like ShotSpotter, in part because I have written a bit about it before. It's a complex issue: a well-designed gunshot detection system would probably be a good thing, but I find SoundThinking (manufacturer of the ShotSpotter system) to be profoundly untrustworthy.

Futures

The changes we are already seeing will continue: ALPR will become more ubiquitous, facial recognition will advance further into the public sphere, and the tech industry will continue to centralize data and facilitate queries by law enforcement. There's a lot of money to me made out of the whole thing, and funding towards law enforcement or public safety purchases are usually politically safe. Pretty much everything is stacked in the direction of more pervasive surveillance in the United States.

Do you find that upsetting? It seems that some people do, and some people do not. I am probably not as opposed to surveillance of public spaces as the most vocal privacy advocates, but I am also convinced that vendor-enabled mass surveillance technology like Flock is subject to enormous abuse and will inevitably undermine constitutional protections. Unfortunately, vocal organizing against mass surveillance has become pretty limited. The ACLU is doing a lot of good work in this area, but I don't see much public organizing.

The best thing you can do is probably to advocate for transparency. The most alarming part of this whole thing, to me, is the way that police departments have brazenly structured purchases of surveillance technology to get around public record and approval requirements. Companies like Flock and SoundThinking encourage this, and write it into their contracts. The end result is that many police departments have installed cameras and microphones in all kinds of places, and will not disclose when, where, how many, or how they are used. We should not allow that kind of secrecy, but preventing it seems to require legislation. The federal situation seems like a loss, so the best pressure point might be to lobby for municipal or state legislation that will require police departments to disclose their surveillance programs. Even better would be a requirement for review and approval of surveillance purchases, but unfortunately that kind of rule often already exists and police departments still structure their purchase arrangements to avoid invoking it.

I suppose the bottom line is this: keep bringing it up. Mass surveillance in the US often feels like a lost cause, but I suppose it's only lost if we give up. It doesn't take that many people showing up at a city council meeting to make something a priority to the councilors; and perhaps the police can only stonewall for so long. It's worth a shot.

One of the difficult things about describing a grift, or at least what became a grift, is judging the sincerity with which the whole thing started. Scams often crystallize around a kernel of truth: genuinely good intentions that start rolling down the hill to profitability and end up crashing through every solid object along the way. I'm not totally sure about Evelyn Wood; she seems to have had all the best in mind but still turned so quickly to hotel conference room seminars that I have trouble lending her the benefit of the doubt.

Still, she was a teacher, and I am inclined to be sympathetic to teachers. Funny, then, that Wood's journey to fame started with another teacher. His curious reading behavior, whether interpreted as intense attention or half-assed inattention, set into motion one of the mid-century's greatest and, perhaps, most embarrassing executive self-help sensations.

In 1929, Evelyn Wood earned a bachelor's in English at the University of Utah. The following two decades are a bit obscure; she took various high-school jobs around Utah leading ultimately to Salt Lake City's Jordan High School. There, as a counselor to girl students, Wood found that many students struggled because of their reading. Assigned books were arduous, handouts discarded. These students struggled to read so severely that it hampered their performance in every area. She launched a remedial reading program of her own design, during which she made her first discovery: as her students learned to read faster, their comprehension improved. Then their grades—in every subject—followed suit. Reading, she learned, was a foundational skill. A person could learn more, do more, achieve more, if only they could read faster.

Wood became fascinated with reading, probably the reason for her return to the University of Utah for a master's degree in speech. Around 1946, she turned her thesis in to Dr. Lowell Lees. Lees was the chair of the Speech and Theater Department, and had a hand in much of the development of Utah theater from the Great Depression until his death in the 1950s. A period photo of Lees depicts him with a breastplate-microphone intercom headset and a look of concentration, hands on the levers of a mechanical variac dimmer rack. He is backstage of either "Show Boat" or "A Midsummer Night's Dream" at the university's summer theater festival. A theater department chair on lights seems odd, yes, but theater was Lees passion.

Perhaps reading was not. When Wood turned her thesis into Lees, he "read, graded, and returned the thesis within a matter of minutes." Wood was amazed that he seemed to just leaf through the pages, but then still had insightful questions to ask. Perhaps I am too cynical. It feels most likely to me that Lees was already familiar with the contents (he was probably Wood's advisor and would have discussed the research plenty of times before) and just didn't bother to read the final document. To Wood, though, something more remarkable had happened. With a series of tests, she convinced herself that Dr. Lees could read over 6,000 words per minute with full comprehension.

A typical American college graduate can read at about 250 words per minute, at least if the material isn't too challenging. Some people, Wood contends, are "10x readers." They read so quickly, and with such good understanding, that they simply outpace the rest of us at every intellectual pursuit. What's more, Wood could make you one of those people. As she tells it, she spent two years, probably in the 1950s, tracking down fifty some examples of other exceptional readers. She published "Reading Skills" in 1958, a book evidently based on some of this research but more focused on remedial skills for grade students than executive achievement.

The introduction of Reading Skills tells us of ten different students. Anna was pretty, but she couldn't read. Joseph hated school, because he couldn't read. Carl's hair is a mess, and his parents neglectful. He also can't read. All of them became proficient readers through Wood's program. But Wood had more in mind than grade students. A year later, with her business-educated husband, she brought her reading program to adults by launching a chain of training centers under the name Evelyn Wood Reading Dynamics.

Books neither bored nor scared me any longer. I could read almost any book within an hour, and more important, I better understood that which I read.

Reading Dynamics became a sensation. Over the following years, Evelyn Wood institutes opened across the country. The speed reading movement received a considerable boost from President John F. Kennedy—he claimed to read at 1,200 words per minute, a skill he learned in part through a correspondence speed reading course. It wasn't one of Evelyn Wood's, but that detail was mostly lost on the public and the success of the Kennedies became linked to Reading Dynamics. He seems to have bought the same course for his brother Ted, and encouraged his staff to take speed reading courses as well. Reading Dynamics didn't miss the marketing opportunity, and indeed the very first dedicated Institute opened in Washington, D.C. and advertised specifically to politicians. Senators and representatives were among her earliest students and her strongest advocates.

Evelyn Wood Reading Dynamics underwent several changes of ownership through the 1960s, but Wood stayed on as developer of the training materials. Soon there were more than 60 institutes, and newspaper ads directed the interested public to "free mini-lessons" held in the meeting rooms of fine hotels across the country. It became a franchise system, with the Woods personally owning the franchise for Utah and Idaho. The company's fortunes have trended up and down with those of speed reading as a concept, but genuine Evelyn Wood speed reading courses are still available today from business training firm Pryor. There has been a bit of a reckoning: far from the 1,000+ WPM rates promised by early Evelyn Wood marketing material, Pryor now advertises "a potential rate of 400-700 words per minute." These numbers align with the upper end of reading speeds observed among the general population. In effect, Pryor no longer claims that speed reading courses will make you a faster reader than more conventional methods of training reading, like just doing a lot of it.

The science has never really been with speed reading. As early as 1959, when Reading Dynamics hit Washington, researchers and educators called Wood's data and methods into question. As with most self-help materials, Wood's writing was heavy on anecdotes and light on quantitative analysis. Certain elements of her method contradicted psychology's growing understanding of human language and perception. At the core of the problem, though, was her promise of comprehension.

It is obvious that a person can "read" a document very quickly, if we relax our definition of "read." This is just as obvious to the developers of speed reading courses. Many advise students to start by skimming, flipping through the whole book or document and taking in the headings and subjects. You can certainly get through a book under an hour that way, but of course, you haven't exactly read it. Think of it as a lossy process: the less time you spend on a document, the less you comprehend and retain its contents. That seems pretty intuitive, doesn't it?

But Wood disagreed, or at least, the company she founded did. It can be a little difficult to untangle Evelyn Wood's original theory from the many generations of Reading Dynamics and competing speed reading systems that followed. Subsequent owners of Reading Dynamics, which included companies like the publisher of Encyclopedia Britannica, made significant revisions to the material. By the 1970s, Wood's role was more as a celebrity spokesperson than a teacher. In any case, Reading Dynamics came to emphasize a key principle that reading faster actually improves comprehension. The most skilled readers, Reading Dynamics taught, don't even read words. They scan a page vertically, not horizontally, taking in an entire line at a time by peripheral vision. There is no need to sound out, read, recognize, or even really see individual words, as the mind actually processes language in large chunks at a time. Reading occurs mostly subconsciously, so in a way all you have to do is see the text and believe that you have read it, and you will retain the content.

In a 2016 review paper on speed reading, a team of psychologists deliver bad news: it just doesn't hold up. Laboratory studies confirm that the eye only has the acuity to distinguish words in a small area, and that reading requires fixating on just about every word individually. That doesn't even matter, though, because other laboratory experiments strongly suggest that the limiting factor on reading speed is not the eyes at all but the mind. Even when clever computer techniques are developed to present text more quickly, comprehension trails off at about the same speed. In fact, when humans read, we regard an even smaller area of our vision than the limits of the fovea would suggest. When looking at a word, we are basically blind to anything further than about seven characters away.

The problems with speed reading are not merely theoretical, though. The researchers considered studies of actual speed readers, people who had either completed speed reading courses or claimed to naturally read at exceptional speeds. Almost no studies can be found that support the claim of faster reading with retained comprehension. Speed readers perform poorly on comprehension tests on new material. People who "speed read" a document generally show similar comprehension to people who have no speed reading training but skimmed the document in the same period of time. When speed readers have performed better, researchers suspect the result comes more from advanced familiarity with the material (a common problem with speed reading courses that use the same texts repeatedly), broader general education (you retain more from non-fiction material if you already knew the information to begin with), and greater experience and confidence in "interpolating" by speculating as to the content of the text that wasn't actually read.

Ultimately, eye tracking experiments tend to confirm the worst. People who speed read don't do all that much actual reading. Skilled speed readers skip much of the text completely, and tend to make things up when asked about things they never fixated on. Most interesting, there seems to be a certain Dunning-Kruger effect at play. People who have speed-read a book on a subject, for example, tend to rate their knowledge of the subject highly and then perform very poorly on questions about it (often scoring similar to chance on multiple choice tests). Speed reading, it turns out, is a placebo. It makes you feel like you have read something, even though you haven't.

And yet we still have speed reading. Wood's efforts were perhaps sincere, but the commercial imperative of the growing Reading Dynamics institutes steered the whole thing away from evidence-based methods and towards ideas with an increasingly tenuous connection to reality. The on-again, off-again success of Reading Dynamics left a lot of room for imitators, or innovators, depending on your perspective. Evelyn Wood's original strain of speed reading has mostly fallen away, replaced by a new set of courses that build on Wood's ideas—the worst of them.

Take, for example, the work of Paul Scheele. Scheele is one of those business conference motivational speaker types, the kind of person who is introduced with a vast and impressive resume but who doesn't seem to have really done anything. With a PhD in "Leadership and Change," he founded Scheele Learning Systems to market a series of innovations. His work is so interconnected with other self-help and new-age grifts that it can be hard to untangle what comes from where, but one of their key programs clearly builds on the Evelyn Wood method: PhotoReading.

The basic concept of PhotoReading is that the mind is able to subconsciously process far more information than the conscious mind. In a marketing sheet, he writes:

Your conscious mind can handle seven pieces of information at a time, while your subconscious mind can handle a staggering 20,000 pieces of information. That's the difference between regular reading and PhotoReading.

So imagine a future in which you pick up a book, flip through the pages, and in a matter of minutes gain a full command of the material contained therein. The key is that you don't actually have to read anything, you just have to see it and your subconscious mind files every word away for later retrieval.

Well, of course, it's not quite that simple. There's a whole technique to it, a technique that you can learn from a self-guided digital course for only $530. Sure, that might seem a little steep, but consider that it's a package that includes not only the course but "The PhotoReading Activator Paraliminal CD." Paraliminal activation or paraliminal hypnosis is another major product from Scheele Learning Systems, although I think it's licensed at least in part from a different organization (Centerpointe Research Institute) founded by different cranks ("transcendental meditation" enthusiasts Bill Harris and Wes Wait). The idea of paraliminal activation is roughly halfway between subliminal inducement videos ¹ and binaural beats ², in that it's both of them mixed together. Incidentally, a lot of subliminal videos are like that anyway, so I'm not sure that Scheele is offering anything you can't get for free. All of these organizations offer rotating carousels of endorsements from famous and successful customers. The fact that these happy customers are almost invariably self-help authors or business conference motivational speakers goes unremarked upon.

Scheele's ultimate claim is that PhotoReading allows you "to 'mentally photograph' the printed page at 25,000 words per minute." 600-800 WPM is an excellent, exceptional reading rate among the normal population. For today's speed reading industry, though, 25,000 WPM is the bar to meet. "Harry Potter and the Deathly Hallows" counts up to about 198,000 words. An experienced PhotoReader, then, ought to be able to complete it in around eight minutes. Well, celebrity speed reader Ann Jones says it took her 47, so no one is perfect. She knocked out "Go Set a Watchman" in 25 and a half, and that on live television. Yes, for the most successful speed readers, people who claim rates in excess of 10,000 WPM, there's almost always some aspect of performance involved... whether that's television appearances or elected office. 25,000 WPM became cemented because it's the rate at which the Guinness World Records clocked celebrity speed reader Howard Berg. Actually, there's a woman who claims to have a Guinness World Record at 80,000 WPM, but it's hard to substantiate as Guinness stopped publishing the speed reading record at all decades ago. I suppose it became too questionable for even them.

The reason I'm so fascinated by speed reading is its close interconnection to the concept of the executive. One of the earliest newspaper ads for Reading Dynamics reads "For Executives, Businessmen, Students, Housewives." The housewives part doesn't quite fit the theme, but I think that might be better understood with the Utah LDS context of the housewife side hustle. Multi-level marketing schemes were becoming a cornerstone of the Salt Lake City business scene during the 1960s, a role they still fill today, and MLM brands like Avon found their success in part by melding the two worlds of the housewife and the business executive. Feminine products sold with masculine hustle, you might venture; some housewives were applying themselves to business with a zeal that would make a railroad baron blush.

For students, the motivation is more obvious. Much of education comes down to reading, and we all remember the feeling of a paper due in two days on a book that you haven't yet opened. For the student, getting good comprehension of a text in a fraction of the time is an incredible offer. So promising was speed reading for education that, in its early days, it found considerable adoption in the educational establishment. Many universities offered speed reading courses, some even made them core curriculum. A particularly prominent speed reading course at Harvard served as the pattern on which many others were taught. Besides a series of demonstration films developed Harvard, devices called "reading regulators" or "reading accelerators" were popular lab equipment for these courses. They automated Evelyn Wood's idea of running a ruler down the page, sliding a metal shield down the page faster and faster to force the student to read at a higher and higher rate. For a few years, speed reading for universities became an entire industry, but it was short lived. Academic speed reading courses faded away as criticisms of Wood's theories became better known and attempts at validating speed reading continued to fail.

"Speed reading," it turns out, did not work out in education. But perhaps that's a matter of framing. If we consider the broader landscape of "things that promise to save you time reading," speed reading is just one in a long line of ideas. It turns out that students have been trying to skip the reading for just about as long as there has been reading—consider Monarch Notes, a line of book summaries and critical commentary already available a hundred years ago. From Monarch to CliffsNotes to Chegg, students have looked to a whole sector of the publishing industry to do the hard work of actually reading books for them. You could say that the purpose of these digests or study guides is to help a student maintain the appearance that they have read a text even though they have not, by imparting only the parts of the text that are most important... important either because they are key to the plot or theme, or because they are likely to appear on exams or be expected in papers.

While students have an obvious need for these types of summaries (scoring well on assignments with less time invested), the appeal to the business executive might seem a little fuzzier. Well, unless we take the cynical view that the ultimate goal of an executive is to look smart, and I'm not sure that you really have to be so cynical to accept that as truth.

Summarizations are obviously "lossy," in that a digest form of a book cannot possibly contain the full information of the original book. Similarly, the weight of scientific evidence, as well as most credible practical experience, tells us that speed reading is a lossy process. There is, as the psychologists put it, no silver bullet in reading. Comprehension takes time; less time means less comprehension; and while you likely can improve your reading speed it will take years of practice.

And yet book summaries are an even larger industry than speed reading, and one that is both older and better adapted to the modern age. There clearly is a market for fast, low-comprehension reading of large texts. The audience is not purely made up of people seeking to create the appearance of work they have not done, although that's clearly a large part of it. Consider the magazine book review: long a staple of magazines, book reviews serve two purposes. They give you an idea of whether or not a book is worth reading, but they also summarize the book, or at least explain the major themes. That gives you some of the content of the book, the major ideas and a few choice details, in just a page of three-column prose. A third of that might be taken up by a wine club ad, to boot.

The case of the magazine book review reminds us that there is a serious, a respectable application for summaries. The perfect example might be the lawyer or doctor, people who are paid explicitly for their expertise and education but who also make heavy use of digests and summaries and desk references. There is a lot of information in the modern world, even in any given field, and no one can keep track of all of it. You might need to speed read, to use the CliffsNotes, just to keep up with the state of the field and find the things that you do need to read in their full length.

And so we have seen the dual facets of the executive demand for speed reading: the businessman, the leader, the executive is the perfect intersection of the professional need to find what to read and the personal need to look like you have done a lot of reading. Executives are expected to know what's out there, but also to seem like they already know all of it. It's a matter of opinion which of these is more prominent, but I think we can all agree that publications like CTO Magazine are aimed at that dual purpose.

Well, these days, publications like CTO Magazine are mostly aimed at drumming up AI hype. That's the other thing about business publishing: it is itself a business, and as beholden to the trends as any other.

The funny thing about speed reading is that it has never been that credible. Evelyn Wood's theories were inconsistent with the research and, frankly, a bit "out there" even as she developed them into a business in the 1960s. Experiments on speed reading, some of them conducted by the same people selling courses, have always shown iffy to clearly negative results. And yet speed reading has, in its good times, enjoyed a level of credibility and popularity that seems out of step with even its promises and certainly with its outcomes.

US Presidents Kennedy, Carter, and Nixon were all speed readers. Carter and Nixon both arranged Evelyn Wood Reading Dynamics courses for their staff, and it seems that Kennedy probably purchased some sort of course for White House staff as well. This was very much perceived as an endorsement from the top, and speed reading became not just a new innovation in education, not just a trend, but practically a requirement for any serious leader. Marketing, and the celebrity adoption that it intentionally engineered, outpaced the results. Evelyn Wood's newspaper ads and reputation got so far out front of the actual pedagogy that today's speed reading industry, spinning ever farther from reason, continues to coast on the same set of presidents.

That's not to say that there has been nothing new in speed reading. In 1984, psychologist Mary C. Potter described a method called "rapid serial visual presentation" or RSVP. The idea of RSVP is to eliminate the whole eye movement part of reading entirely, instead using a computer to present one word at a time, each centered in the same location. In theory, the words can be presented faster and faster until the user is reading more quickly than the visual system allows. Well, that's a theory at least. It's inconsistent with later research suggesting that reading speed is limited by cognition rather than perception, but most of that wasn't yet known at the time. Even so, Potter doesn't seem to have viewed RSVP as a speed reading technique. She described it as a method for cognitive research, one that could enable new experiments and improve old results by controlling for the many variables involved in scanning a page of text.

The idea of RSVP as a speed reading technique seems to have been popularized by software startup Spritz, who launched an RSVP speed reading application in 2014. Spritz seems to have spun it as "text streaming," although I think that might have been a later branding innovation. The claims of Spritz are relatively modest, only 1,000 WPM in most cases and sometimes as low as 600 WPM. These are speeds achievable (even if only narrowly) without technical assistance for exceptionally fast readers. Even so, it doesn't really work out. Research on the RSVP method of speed reading finds that comprehension decreases with increasing speed. Amusingly, some experiments show that RSVP results in decreased comprehension even when run at the same speed the subject reads naturally. Psychologists tend to attribute that effect to the fact that RSVP prevents going back and rereading a sentence that you didn't fully understand—a behavior that seems to be a natural and even required part of good reading, despite the fact that Evelyn Wood and virtually every speed reading theorist since has outlawed it.

The fact that the RSVP concept is fundamentally at odds with blinking is probably the major cause of a reported increase in fatigue, as well, but none of these shortcomings have prevented the massive popularity of RSVP within the tech industry especially. Spritz, the company, has gone basically nowhere, but the concept has graduated from TED talks to a huge inventory of browser extensions, mobile apps, CLI tools, and various and sundry GitHub projects that all make the same claims about increased reading speed. "Speed Reading Makes a Comeback" was the title of an NBC News spot on Iris Reading, more of a traditional Wood-style speed reading training company that has since wholeheartedly embraced the RSVP concept.

If software is part of the speed reading story, and a particularly core part of it today, we will have to take on the elephant in the room: in a certain sense, a very real sense, summarizing text is now the largest single driver of the US economy.

The appeal of summarization to the business executive has never gone away; the underlying technology has just evolved. Since the 2022 launch of ChatGPT, television spots, bus shelter ads, and the collective buzz of the south end of the San Francisco Peninsula have promised first and foremost that AI will relieve us of the obligation of reading. An LLM can read your email, read the news, read a book, or read the comments. Actually, the LLM has already read a lot of these things. On command, it can summarize them to you.

AI advertising seems to imagine a world that is, well, oddly familiar: one in which students, housewives, and, yes, business executives can save hours of each day by using the LLM to, in effect, read at 25,000 WPM. It also seems that the same basic principles apply: the LLM's output loses some of the content of the original material. It might also gain some content, a benefit of all of the other things that the LLM has also been trained on. Still: there's always a certain rounding out, a sanding down of the details.

What strikes me most about LLM summaries is just how long they are. When I have asked Claude to summarize reading notes, it has routinely produced output that is longer than the original notes. This problem can probably be addressed by prompting, although my efforts at appending everything from "be brief" to "for the love of God keep it to one paragraph" have failed to produce a good result. Maybe I'm holding it wrong, maybe I'm an idiot, I possess no qualifications in this area besides decades as a natural language user and an unfinished degree in technical writing. But experience suggests that my coworkers have the same problem. I see AI generated meeting summaries, AI generated issue descriptions, AI generated sales documents. One of their common properties is that they are astoundingly, uselessly verbose.

Of course, modern AI can do so much more than summarize text. "Generative AI" promises not only to summarize, but also to create something new. Perhaps that's why the LLM is so verbose. I, personally, find that I make up for my lackluster interpersonal skills by writing. Perhaps LLMs make up for their similar limitations, their fundamentally text-based, screen-resident nature, by using the one tool that they have. The LLM cannot think or feel, yet it can write. So it writes: a simple question answered with such energy that it merits four distinct bulleted lists, each with an emoji-laden heading and an introductory paragraph. I suppose I can sympathize. We must imagine Grok happy.

I do not mean to criticize AI too harshly, although I think the level of criticism that this entire industry phenomena deserves is high enough that you have to go big.

But the relationship between speed reading and the LLM—between Sam Altman and Evelyn Wood—is vague but vivid. The software industry's imagined future, in which people use LLMs to generate text that other people use LLMs to summarize, genuinely haunts me. AI has created a profound contradiction: it promises the productivity gains of speed reading, the ease of CliffsNotes, but it doesn't just shorten text. It also lengthens it. My joking reference to Camus, shoddy as it is, becomes more meaningful. ChatGPT pushes the written word up the hill, it watches it roll back down again.

I read "The Myth of Sisyphus" for the same reason everyone else did: high school. IB English HL. Yes, I went to one of those schools. If you are not familiar you can look it up and one of the top results, at least for me, is a clearly LLM-generated article that is four or five times longer than it should be based on the factual content. You can have your web browser's LLM feature summarize it back down for you, if you want. The result comes out a lot less useful than the Wikipedia article but it is, as they say, disruptive nonetheless.

If the purpose of reading is solely to acquire information and accumulate thought units, then surely speed and efficiency are the essential criteria. Regressing is obviously a morbid symptom since it is destructive of time and energy, while horizontal reading not only taxes the optic muscles, but requires that the same tome remain clutched by fingers which could be more profitably employed in reaching for yet another volume.

We're all full of opinions on the era of AI. I am perhaps not as pessimistic as you might think: the machine learning innovations of the last few years clearly do have useful applications. Even summarizing text has its time and place. I suppose that what frustrates me most about it all is the lack of ambition. LLMs train on text, take text as input, and generate text as output. A room of Silicon Valley visionaries, presented with this astounding tool, came up with such world-changing applications as "reading emails" and "writing emails." The whole industry is still struggling to move past this trivial, boring, frequently nonproductive use.

As for lip motions, any toddler knows that the mouth is tardier than the eye, and retardation is one of the most dreaded words in an educator's terminology. Furthermore, the speed cult is quick to point out that slow readers are rarely careful ones, and generally speaking, comprehension appears to increase with reading velocity. Speed reading, it would appear, is all profit and no loss and if it can make good its claims at linking efficiency and comprehension, then it is well that its methodology and objectives are incorporated into any reading program.

There is the potential, the AI's industries advocates say, of AI actually expanding human creativity. Machine learning methods of producing "art," whether text or image or audio or video, will lower the barrier of entry to artistic production. Of course, that depends a lot on how you define "artistic production," but at least it's a rare promise of a better future rather than a worse one. It only takes a brief interaction with the modern internet to realize that we do live in an age blessed with text. We are rich in the written word like never before, so wealthy with words that they crowd out the actual information. Search results are mostly AI-generated, but the search engine doesn't want you to look at them anyway, it's provided its own AI-generated treatise. The headings, the paragraphs, the bulleted lists, they run down the page, drip from our screens, they leave our desks filthy with content.

But prior to debating the plausibility of the claims in an Evelyn Wood brochure, it would seem logical to consider the desirability of the goals—goals which appear to have slipped unchallenged into the realm of pedagogical axioms. Are such facile reading practices worthy of unqualified adulation? A careful look at their implications suggests that such seeming saints can in fact be devils.

It's enough to drive you to madness. Why do we use computers to write text that no one will read? Why do we use computers to read text that no one wrote?

Years ago, in college, during a previous AI winter, I sat in my room reading a shitty science fiction novel. Leo, from across the hall, walked in. "What class is that for?" he asked.

"Not for a class," I responded.

"So you're just reading it?"

Taken by themselves, the cardinal virtues of reading efficiency can collectively demean the entire reading process by treating it as a function rather than as an art.

There is nothing new under the sun. We have done this all before: we have fixated on reading as production, production as profitable, and reading thus, ultimately, unimportant. A detail to be optimized away. An expense. ChatGPT didn't start this. It won't end it. That's what I remind myself: we are living through just another step in the evolution of culture.

But then I still worry. What if this is it? Between short-form video and AI, between social media's pivot to stoking fascism and the publishing industry's pivot to reprinting AO3, what if language arts are done?

None of these people care. That's the one thing I can say confidently, or at least say that I truly believe. These people building the cutting edge of natural language, these industry titans who style themselves as the loyalists of our nation and revolutionaries of the arts, they don't give a damn about writing or reading. Text is an asset, an asset to extract, refine, and dispense. They're just trying to make it through the news and their Twitter feeds and a half dozen pop-science books as fast as possible so that they can be, feel, or at least look like they're well-read. They assume that everyone else feels the same way.

How can any teacher extol the pleasures of reading when classroom practice implicitly asserts that books are mines to be stripped and not pastures in which to dwell and delight?

I've been quoting from Leonard R. Mendelsohn, whose paper "Jetting to Utopia: The Speed Reading Phenomenon" ignores the question of whether or not speed reading works and instead considers whether or not it is a good idea. His context was the classroom of the 1970s: speed reading had caught on in education, and Mendelsohn worried. Well-intentioned teachers were training their students to absolutely optimize the mechanics of reading. In the process, Mendelsohn feared, they had forgotten the point.

Reading can provide fodder for the brain by the ready conversion of wood pulp and printer's ink into social poise, persuasiveness, and a financially rewarding livelihood.

Things have changed a great deal since Mendelsohn's day. The wood pulp is gone, so too the printer ink, and so too the financial rewards. Writing is, I suppose, more of an art than ever before, as my chosen industry devotes its full might to destroying my chosen avocation.

Although reading might be branded with the explicit label "fun," it is not long before the apt student reaches the conclusion that speed, concepts, and information are all one knows and all one needs to know.

Mendelsohn's paper ran in the journal "Language Arts." It's about four pages long, about 2,300 words. It took me around ten minutes to read. An accomplished student of Evelyn Wood could read it in just a couple of minutes. With some chiding to stay brief and cut it out with the bulleted lists, Claude summarized it in a few sentences.

For the journal, though, the paper is not quite long enough. Its last page is only half full. The journal editor made up the difference, they found some filler. It's a poem about clouds.

ChatGPT can do so much, but it can't do the work of a poet. It can't match Christa Kessler, age 10, Powhatan School, Boyce, Virginia. She wrote "Clouds" almost fifty years ago, an editor used it to round out the layout of a journal, JSTOR coughed it up along with my article, and now I am thinking about how clouds really are interludes in the middle of a great blue sea.

That's what it's like to read slow. That's what it means to write.

Today's Anonymous submitter sends us some React code. We'll look at the code and then talk about the WTF:

// inside a function for updating checkboxes on a page
if (!e.target.checked) {
  const removeIndex = await checkedlist.findIndex(
    (sel) => sel.Id == selected.Id,
  )
  const removeRowIndex = await RowValue.findIndex(
    (sel) => sel == Index,
  )

// checkedlist and RowValue are both useState instances.... they should never be modified directly
  await checkedlist.splice(removeIndex, 1)
  await RowValue.splice(removeRowIndex, 1)

// so instead of doing above logic in the set state, they dont
  setCheckedlist(checkedlist)
  setRow(RowValue)
} else {
  if (checkedlist.findIndex((sel) => sel.Id == selected.Id) == -1) {
    await checkedlist.push(selected)
  }
// same, instead of just doing a set state call, we do awaits and self updates
  await RowValue.push(Index)
  setCheckedlist(checkedlist)
  setRow(RowValue)
}

Comments were added by our submitter.

This code works. It's the wrong approach for doing things in React: modifying objects controlled by react, instead of using the provided methods, it's doing asynchronous push calls. Without the broader context, it's hard to point out all the other ways to do this, but honestly, that's not the interesting part.

I'll let our submitter explain:

This code is black magic, because if I update it, it breaks everything. Somehow, this is working in perfect tandem with the rest of the horrible page, but if I clean it up, it breaks the checkboxes; they're no longer able to be clicked. Its forcing React somehow to update asynchronously so it can use these updated values correctly, but thats the neat part, they aren't even being used anywhere else, but somehow the re-rendering page only accepts awaits. I've tried refactoring it 5 different ways to no avail

That's what makes truly bad code. Code so bad that you can't even fix it without breaking a thousand other things. Code that you have to carefully, slowly, pick through and gently refactor, discovering all sorts of random side-effects that are hidden. The code so bad that you actually have to live with it, at least for awhile.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Aankhen has a peer who loves writing Python scripts to automate repetitive tasks. We'll call this person Ernest.

Ernest was pretty proud of some helpers he wrote to help him manage his Docker containers. For example, when he wanted to stop and remove all his running Docker containers, he wrote this script:

#!/usr/bin/env python
import subprocess

subprocess.run("docker kill $(docker ps -q)", shell=True)
subprocess.run("docker rm $(docker ps -a -q)", shell=True)

He aliased this script to docker-stop, so that with one command he could… run two.

"Ernest," Aankhen asked, "couldn't this just be a bash script?"

"I don't really know bash," Ernest replied. "If I just do it in bash, if the first command fails, the second command doesn't run."

Aankhen pointed out that you could make bash not do that, but Ernest replied: "Yeah, but I always forget to. This way, it handles errors!"

"It explicitly doesn't handle errors," Aankhen said.

"Exactly! I don't need to know when there are no containers to kill or remove."

"Okay, but why not use the Docker library for Python?"

"What, and make the software more complicated? This has no dependencies!"

Aankhen was left with a sinking feeling: Ernest was either the worst developer he was working with, or one of the best.

[Advertisement] Keep all your packages and Docker containers in one place, scan for vulnerabilities, and control who can access different feeds. ProGet installs in minutes and has a powerful free version with a lot of great features that you can upgrade when ready.Learn more.

Every once in awhile, we get a bit of terrible code, and our submitter also shares, "this isn't called anywhere," which is good, but also bad. Ernesto sends us a function which is called in only one place:

///
/// Shutdown server
///
private void shutdownServer()
{
    shutdownServer();
}

The "one place", obviously, is within itself. This is the Google Search definition of recursion, where each recursive call is just the original call, over and over again.

This is part of a C# service, and this method shuts down the server, presumably by triggering a stack overflow. Unless C# has added tail calls, anyway.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Our anonymous submitter relates a tale of simplification gone bad. As this nightmare unfolds, imagine the scenario of a new developer coming aboard at this company. Imagine being the one who has to explain this setup to said newcomer.

Imagine being the newcomer who inherits it.

David's job should have been an easy one. His company's sales data was stored in a database, and every day the reporting system would query a SQL view to get the numbers for the daily key performance indicators (KPIs). Until the company's CTO, who was proudly self-taught, decided that SQL views are hard to maintain, and the system should get the data from one of those new-fangled APIs instead.

But how does one call an API? The reporting system didn't have that option, so the logical choice was Azure Data Factory to call the API, then output the data to a file that the reporting system could read. The only issue was that nobody on the team spoke Azure Data Factory, or for that matter SQL. But no problem, one of David's colleagues assured, they could do all the work in the best and most multifunctional language ever: C#.

But you can't just write C# in a data factory directly, that would be silly. What you can do is have the data factory pipeline call an Azure function, which calls a DLL that contains the bytecode from C#. Oh, and a scheduler outside of the data factory to run the pipeline. To read multiple tables, the pipeline calls a separate function for each table. Each function would be based on a separate source project in C#, with 3 classes each for the HTTP header, content, and response; and a separate factory class for each of the actual classes.

After all, each table had a different set of columns, so you can't just re-use classes for that.

There was one little issue: the reporting system required an XML file, whereas the API would export data in JSON. It would be silly to expect a data factory, of all things, to convert this. So the CTO's solution was to have another C# program (in a DLL called by a function from a pipeline from an external scheduler) that reads the JSON document saved by the earlier program, uses foreach to go over each element, then saves the result as XML. A distinct program for each table, of course, requiring distinct classes for header, content, response, and factories thereof.

Now here's the genius part: to the C# class representing the output data, David's colleague decided to attach one different object for each input table required. The data class would use reflection to iterate over the attached objects, and for each object, use a big switch block to decide which source file to read. This allows the data class to perform joins and calculations before saving to XML.

To make testing easier, each calculation would be a separate function call. For example, calculating a customer's age was a function taking struct CustomerWithBirthDate as input, use a foreach loop to copy all the data except replacing one field, and return a CustomerWithAge struct to pass to the next function. The code performed a bit slowly, but that was an issue for a later year.

So basically, the scheduler calls the data factory, which calls a set of Azure functions, which call a C# function, which calls a set of factory classes to call the API and write the data to a text file. Then, the second scheduler calls a data factory, which calls Azure functions, which call C#, which calls reflection to check attachment classes, which read the text files, then call a series of functions for each join or calculation, then call another set of factory classes to write the data to an XML file, then call the reporting system to update.

Easy as pie, right? So where David's job could have been maintaining a couple hundred lines of SQL views, he instead inherited some 50,000 lines of heavily-duplicated C# code, where adding a new table to the process would easily take a month.

Or as the song goes, Somebody Told Me the User Provider should use an Adaptor to Proxy the Query Factory Builder ...

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

First up with the money quote, Peter G. remarks "Hi first_name euro euro euro, look how professional our marketing services are! "

"It takes real talent to mispell error" jokes Mike S. They must have done it on purpose.

I long wondered where the TikTok profits came from, and now I know. It's Daniel D. "I had issues with some incorrectly documented TikTok Commercial Content API endpoints. So I reached out to the support. I was delighted to know that it worked and my reference number was . PS: 7 days later I still have not been contacted by anyone from TikTok. You can see their support is also . "

Fortune favors the prepared, and Michael R. is very fortunate. "I know us Germans are known for planning ahead so enjoy the training on Friday, February 2nd 2029. "

Someone other than dragoncoder047 might have shared this earlier, but this time dragoncoder047 definitely did. "Digital Extremes (the developers of Warframe) were making many announcements of problems with the new update that rolled out today [February 11]. They didn’t mention this one!"

We mostly don't pick on bad SQL queries here, because mostly the query optimizer is going to fix whatever is wrong, and the sad reality is that databases are hard to change once they're running; especially legacy databases. But sometimes the code is just so hamster-bowling-backwards that it's worth looking into.

Jim J has been working on a codebase for about 18 months. It's a big, sprawling, messy project, and it has code like this:

AND CASE WHEN @c_usergroup = 50 AND NOT EXISTS(SELECT 1 FROM l_appl_client lac WHERE lac.f_application = fa.f_application AND lac.c_linktype = 840 AND lac.stat = 0 AND CASE WHEN ISNULL(lac.f_client,0) <> @f_client_user AND ISNULL(lac.f_c_f_client,0) <> @f_client_user THEN 0 ELSE 1 END = 1 ) THEN 0 ELSE 1 END = 1 -- 07.09.2022

We'll come back to what it's doing, but let's start with a little backstory.

This code is part of a two-tier application: all the logic lives in SQL Server stored procedures, and the UI is a PowerBuilder application. It's been under development for a long time, and in that time has accrued about a million lines of code between the front end and back end, and has never had more than 5 developers working on it at any given time. The backlog of feature requests is nearly as long as the backlog of bugs.

You may notice the little date comment in the code above. That's because until Jim joined the company, they used Visual Source Safe for version control. Visual Source Safe went out of support in 2005, and let's be honest: even when it was in support it barely worked as a source control system. And that's just the Power Builder side- the database side just didn't use source control. The source of truth was the database itself. When going from development to test to prod, you'd manually export object definitions and run the scripts in the target environment. Manually. Yes, even in production. And yes, environments did drift and assumptions made in the scripts would frequently break things.

You may also notice the fields above use a lot of Hungarian notation. Hungarian, in the best case, makes it harder to read and reason about your code. In this case, it's honestly fully obfuscatory. c_ stands for a codetable, f_ for entities. l_ is for a many-to-many linking table. z_ is for temporary tables. So is x_. And t_. Except not all of those "temporary" tables are truly temporary, a lesson Jim learned when trying to clean up some "junk" tables which were not actually junk.

I'll let Jim add some more detail around these prefixes:

an "application" may have a link to a "client", so there is an f_client field; but also it references an "agent" (which is also in the f_client table, surpise!) - this is how you get an f_c_f_client field. I have no clue why the prefix is f_c_ - but I also found c_c_c_channel and fc4_contact columns. The latter was a shorthand for f_c_f_c_f_c_f_contact, I guess.

"f_c_f_c_f_c_f_c" is also the sound I'd make if I saw this in a codebase I was responsible for. It certainly makes me want to change the c_c_c_channel.

With all this context, let's turn it back over to Jim to explain the code above:

And now, with all this background in mind, let's have a look at the logic in this condition. On the deepest level we check that both f_client and f_c_f_client are NOT equal to @f_client_user, and if this is the case, we return 0 which is NOT equal to 1 so it's effectively a negation of the condition. Then we check that records matching this condition do NOT EXIST, and when this is true - also return 0 negating the condition once more.

Honestly, the logic couldn't be clearer, when you put it that way. I jest, I've read that twelve times and I still don't understand what this is for or why it's here. I just want to know who we can prosecute for this disaster. The whole thing is a quadruple negative and frankly, I can't handle that kind of negativity.

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

Today's snippet from Rich D is short and sweet, and admittedly, not the most TFs of WTFs out there. But it made me chuckle, and sometimes that's all we need. This Java snippet shows us how to delete a file:

if (Files.exists(filePath)) {
    Files.deleteIfExists(filePath);
}

If the file exists, then if it exists, delete it.

This commit was clearly submitted by the Department of Redundancy Department. One might be tempted to hypothesize that there's some race condition or something that they're trying to route around, but if they are, this isn't the way to do it, per the docs: "Consequently this method may not be atomic with respect to other file system operations." But also, I fail to see how this would do that anyway.

The only thing we can say for certain about using deleteIfExists instead of delete is that deleteIfExists will never throw a NoSuchFileException.

Agatha has inherited some Windows Forms code. This particular batch of such code falls into that delightful category of code that's wrong in multiple ways, multiple times. The task here is to disable a few panels worth of controls, based on a condition. Or, since this is in Spanish, "bloquear controles". Let's see how they did it.

private void BloquearControles()
{
	bool bolBloquear = SomeConditionTM; // SomeConditionTM = a bunch of stuff. Replaced for clarity.

	// Some code. Removed for clarity.
	
	// private System.Windows.Forms.Panel pnlPrincipal;
	foreach (Control C in this.pnlPrincipal.Controls)
	{
		if (C.GetType() == typeof(System.Windows.Forms.TextBox))
		{
			C.Enabled = bolBloquear;
		}
		if (C.GetType() == typeof(System.Windows.Forms.ComboBox))
		{
			C.Enabled = bolBloquear;
		}
		if (C.GetType() == typeof(System.Windows.Forms.CheckBox))
		{
			C.Enabled = bolBloquear;
		}
		if (C.GetType() == typeof(System.Windows.Forms.DateTimePicker))
		{
			C.Enabled = bolBloquear;
		}
		if (C.GetType() == typeof(System.Windows.Forms.NumericUpDown))
		{
			C.Enabled = bolBloquear;
		}
	}
	
	// private System.Windows.Forms.GroupBox grpProveedor;
	foreach (Control C1 in this.grpProveedor.Controls)
	{
		if (C1.GetType() == typeof(System.Windows.Forms.TextBox))
		{
			C1.Enabled = bolBloquear;
		}
		if (C1.GetType() == typeof(System.Windows.Forms.ComboBox))
		{
			C1.Enabled = bolBloquear;
		}
		if (C1.GetType() == typeof(System.Windows.Forms.CheckBox))
		{
			C1.Enabled = bolBloquear;
		}
		if (C1.GetType() == typeof(System.Windows.Forms.DateTimePicker))
		{
			C1.Enabled = bolBloquear;
		}
		if (C1.GetType() == typeof(System.Windows.Forms.NumericUpDown))
		{
			C1.Enabled = bolBloquear;
		}
	}

	// private System.Windows.Forms.GroupBox grpDescuentoGeneral;
	foreach (Control C2 in this.grpDescuentoGeneral.Controls)
	{
		if (C2.GetType() == typeof(System.Windows.Forms.TextBox))
		{
			C2.Enabled = bolBloquear;
		}
		if (C2.GetType() == typeof(System.Windows.Forms.ComboBox))
		{
			C2.Enabled = bolBloquear;
		}
		if (C2.GetType() == typeof(System.Windows.Forms.CheckBox))
		{
			C2.Enabled = bolBloquear;
		}
		if (C2.GetType() == typeof(System.Windows.Forms.DateTimePicker))
		{
			C2.Enabled = bolBloquear;
		}
		if (C2.GetType() == typeof(System.Windows.Forms.NumericUpDown))
		{
			C2.Enabled = bolBloquear;
		}
	}

	// Some more code. Removed for clarity.
}

This manages two group boxes and a panel. It checks a condition, then iterates across every control beneath it, and sets their enabled property on the control. In order to do this, it checks the type of the control for some reason.

Now, a few things: every control inherits from the base Control class, which has an Enabled property, so we're not doing this check to make sure the property exists. And every built-in container control automatically passes its enabled/disabled state to its child controls. So there's a four line version of this function where we just set the enabled property on each container.

This leaves us with two possible explanations. The first, and most likely, is that the developer responsible just didn't understand how these controls worked, and how inheritance worked, and wrote this abomination as an expression of that ignorance. This is extremely plausible, extremely likely, and honestly, our best case scenario.

Because our worse case scenario is that this code's job isn't to disable all of the controls. The reason they're doing type checking is that there are some controls used in these containers that don't match the types listed. The purpose of this code, then, is to disable some of the controls, leaving others enabled. Doing this by type would be a terrible way to manage that, and is endlessly confusing. Worse, I can't imagine how this behavior is interpreted by the end users; the enabling/disabling of controls following no intuitive pattern, just filtered based on the kind of control in use.

The good news is that Agatha can point us towards the first option. She adds:

They decided to not only disable the child controls one by one but to check their type and only disable those five types, some of which aren't event present in the containers. And to make sure this was WTF-worthy the didn't even bother to use else-if so every type is checked for every child control

She also adds:

At this point I'm not going to bother commenting on the use of GetType() == typeof() instead of is to do the type checking.

Bad news, Agatha: you did bother commenting. And even if you didn't, don't worry, someone would have.

Python is (in)famous for its "batteries included" approach to a standard library, but it's not that notable that it has plenty of standard data structures, like dicts. Nor is in surprising that dicts have all sorts of useful methods, like pop, which removes a key from the dict and returns its value.

Because you're here, reading this site, you'll also be unsurprised that this doesn't stop developers from re-implementing that built-in function, badly. Karen sends us this:

def parse_message(message):
    def pop(key):
        if key in data:
            result = data[key]
            del data[key]
            return result
        return ''

    data = json.loads(message)
    some_value = pop("some_key")
    # <snip>...multiple uses of pop()...</snip>

Here, they create an inner method, and they exploit variable hoisting. While pop appears in the code before data is declared, all variable declarations are "hoisted" to the top. When pop references data, it's getting that from the enclosing scope. Which while this isn't a global variable, it's still letting a variable cross between two scopes, which is always messy.

Also, this pop returns a default value, which is also something the built-in method can do. It's just the built-in version requires you to explicitly pass the value, e.g.: some_value = data.pop("some_key", "")

Karen briefly wondered if this was a result of the Python 2 to 3 conversion, but no, pop has been part of dict for a long time. I wondered if this was just an exercise in code golf, writing a shorthand function, but even then- you could just wrap the built-in pop with your shorthand version (not that I'd recommend such a thing). No, I think the developer responsible simply didn't know the function was there, and just reimplemented a built-in method badly, as so often happens.

Pike pike pike pike Pike pike pike.

Lincoln KC repeated "I never knew Bank of America Bank of America Bank of America was among the major partners of Bank of America."

"Extra tokens, or just a stutter?" asks Joel "An errant alt-tab caused a needless google search, but thankfully Gemini's AI summary got straight-to-the-point(less) info. It is nice to see the world's supply of Oxford commas all in once place. "

Alessandro M. isn't the first one to call us out on our WTFs. "It’s adorable how the site proudly supports GitHub OAuth right up until the moment you actually try to use it. It’s like a door with a ‘Welcome’ sign that opens onto a brick wall." Meep meep.

Float follies found Daniel W. doubly-precise. "Had to go check on something in M365 Admin Center, and when I was on the OneDrive tab, I noticed Microsoft was calculating back past the bit. We're in quantum space at this point."

Weinliebhaber Michael R. sagt "Our German linguists here will spot the WTF immediately where my local wine shop has not. Weiẞer != WEIBER. Those words mean really different things." Is that 20 euro per kilo, or per the piece?

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

Industrial machines are generally accompanied by "Human Machine Interfaces", HMIs. This is industrial slang for a little computerized box you use to control the industrial machine. All the key logic and core functionality and especially the safety functionality is handled at a deeper computer layer in the system. The HMI is just buttons users can push to interact with the machine.

Purchasers of those pieces of industrial equipment often want to customize that user interface. They want to guide users away from functions they don't need, or make their specific workflow clear, or even just brand the UI. This means that the vendor needs to publish an API for their HMI.

Which brings us to Wendy. She works for a manufacturing company which wants to customize the HMI on a piece of industrial equipment in a factory. That means Wendy has been reading the docs and poking at the open-sourced portions of the code, and these raise more questions than they answer.

For example, the HMI's API provides its own set of collection types, in C#. We can wonder why they'd do such a thing, which is certainly a WTF in itself, but this representative line raises even more questions than that:

Int32 Count { get; set; }

What happens if you use the public set operation on the count of items in a collection? I don't know. Wendy doesn't either, as she writes:

I'm really tempted to set the count but I fear the consequences.

All I can hear in my head when I think about "setting the Count" is: "One! One null reference exception! Two! TWO null reference exceptions! HA HA HA HA!"

By http://muppet.wikia.com/wiki/Count_von_Count

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

I've had the misfortune of working in places which did source-control via comments. Like one place which required that, with each section of code changed, you needed to add a comment with your name, the ticket number, and the reason the change was made. You know, the kind of thing you can just get from your source control service.

In their defense, that policy was invented for mainframe developers and then extended to everyone else, and their source control system was in Visual Source Safe. VSS was a) terrible, and b) a perennial destroyer of history, so maybe they weren't entirely wrong and VSS was the real WTF. I still hated it.

In any case, Alice's team uses more modern source control than that, which is why she's able to explain to us the story of this function:

public function calculateMassGrossPay(array $employees, Payroll $payroll): array
{
    // it shouldn't enter here, but if it does by any change, do nth
    return [];
}

Once upon a time, this function actually contained logic, a big pile of fairly complicated logic. Eventually, a different method was created which streamlined the functionality, but had a different signature and logic. All the callers were updated to use that method instead- by commenting out the line which called this one. This function had a comment added to the top: // it shouldn't enter here.

Then, the body of this function got commented out, and the return was turned into an empty array. The comment was expanded to what you see above. Then, eventually, the commented-out callers were all deleted. Years after that, the commented out body of this function was also deleted, leaving behind the skeleton you see here.

This function is not referenced anywhere else, not even in a comment. It's truly impossible for code to "enter here".

Alice writes: "Version control by commented out code does not work very well."

Indeed, it does not.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

Henrik H's employer thought they could save money by hiring offshore, and save even more money by hiring offshore junior developers, and save even more money by basically not supervising them at all.

Henrik sends us just one representative line:

if (System.DateTime.Now.AddDays(-365) <= f.ReleaseDate) // 365 days means one year 

I appreciate the comment, that certainly "helps" explain the magic number. There's of course, just one little problem: It's wrong. I mean, ~75% of the time, it works every time, but it happily disregards leap years. Which may or may not be a problem in this case, but if they got so far as learning about the AddDays method, they were inches from using AddYears.

The utility closet Ellis had inherited and lived with for 17 years had been a cesspool of hazards to life and limb, a collection of tangible WTFs that had everyone asking an uncaring god, "What were they thinking?"

Every contractor who'd ever had to perform any amount of work in there had come away appalled. Many had even called over their buddies to come and see the stunning mess for themselves:

• All of the electrical components, dating from the 1980s, were scarily underpowered for what they were supposed to be powering.
• To get to the circuit breaker box—which was unlabeled, of course—one had to contort themselves around a water heater almost as tall as Ellis herself.
• As the house had no basement, the utility closet was on the first floor in an open house plan. A serious failure with said water heater would've sent 40 gallons (150 liters) of scalding-hot tsunami surging through the living room and kitchen.
• The furnace's return air vent had been screwed into crumbling drywall, and only prayers held it in place. Should it have fallen off, it would never have been replaceable. And Ellis' cat would've darted right in there for the adventure of a lifetime.
• To replace the furnace filter, Ellis had to put on work gloves, unscrew a sharp sheet-metal panel from the side of the furnace, pull the old filter out from behind a brick (the only thing holding it in place), manipulate the filter around a mess of water and natural gas pipes to get it out, thread the new filter in the same way, and then secure it in place with the brick before screwing the panel back on. Ellis always pretended to be an art thief in a museum, slipping priceless paintings around security-system lasers.
• Between the water tank, furnace, water conditioning unit, fiber optical network terminal, and router, there was barely room to breathe, much less enough air to power ignition for the gas appliances. Some genius had solved this by cutting random holes in several walls to admit air from outside. One of these holes was at floor-level. Once, Ellis opened the closet door to find a huge puddle on the floor, making her fear her hot water heater was leaking. As it turned out, a power-washing service had come over earlier that day. When they'd power-washed the exterior of her home, some of that water shot straight through one of those holes she hadn't known about, giving her utility closet a bonus bath.
• If air intake was a problem, venting the appliances' exhaust was an even worse issue. The sheet-metal vents had calcified and rusted over time. If left unaddressed, holes could've formed that would've leaked carbon monoxide into Ellis' house.

Considering all the above, plus the fact that the furnace and air conditioner were coming up on 20 years of service, Ellis couldn't put off corrective action any longer. Last week, over a span of 3 days, contractors came in to exorcise the demons:

• Upgrading electricals that hadn't already been dealt with.
• Replacing the hot water tank with a wall-mounted tankless heater.
• Replacing the furnace and AC with a heat pump and backup furnace, controlled by a new thermostat.
• Creating new pipes for intake and venting (no more reliance on indoor air for ignition).
• Replacing the furnace return air vent with a sturdier one.
• Putting a special hinged door on the side of the furnace, allowing the filter to be replaced in a matter of seconds (RIP furnace brick).

With that much work to be done, there were bound to be hiccups. For instance, when the Internet router was moved, an outage occurred: for no good reason, the optical network terminal refused to talk to Ellis' Wifi router after powering back up. A technician came out a couple days later, reset the Internet router, and everything was fine again.

All in all, it was an amazing and welcome transformation. As each new update came online, Ellis was gratefully satisfied. It seemed as though the demons were finally gone.

Unbeknownst to them all, there was one last vengeful spirit to quell, one final WTF that it was hell-bent on doling out.

It was late Friday afternoon. Despite the installers' best efforts, the new thermostat still wasn't communicating with the new heat pump. Given the timing, they couldn't contact the company rep to troubleshoot. However, the thermostat was properly communicating with the furnace. And so, Ellis was left with the furnace for the weekend. She was told not to mess with the thermostat at all except to adjust the set point as desired. They would follow back up with her on Monday.

For Ellis, that was perfectly fine. With the historically cold winter they'd been enduring in her neck of the woods, heat was all she cared about. She asked whom to contact in case of any issues, and was told to call the main number. With all that squared away, she looked forward to a couple of quiet, stress-free days before diving back into HVAC troubleshooting.

Everything was fine, until it wasn't. Around 11AM on Saturday, Ellis noticed that the thermostat displayed the word "Heating" while the furnace wasn't actually running. Maybe it was about to turn on? 15 minutes went by, then half an hour. Nothing had changed except for the temperature in her house steadily decreasing.

Panic set in at the thought of losing heat in her home indefinitely. That fell on top of a psyche that was already stressed out and emotionally exhausted from the last several days' effort. Struggling for calm, Ellis first tried to call that main number line for help as directed. She noticed right away that it wasn't a real person on the other end asking for her personal information, but an AI agent. The agent informed her that the on-call technician had no availablity that weekend. It would pencil her in for a service appointment on Monday. How did that sound?

"Not good enough!" Ellis cried. "I wanna speak to a representative!"

"I understand!" replied the blithe chatbot. "Hold on, let me transfer you!"

For a moment, Ellis was buoyed with hope. She'd gotten past the automated system. Soon, she'd be talking with a live person who might even be able to walk her through troubleshooting over the phone.

The new agent answered. Ellis began pouring her heart out—then stopped dead when she realized it was another AI agent, this time with a male voice instead of a female one. This one proceeded through nearly the same spiel as the first. It also scheduled her for a Monday service appointment even though the other chatbot had already claimed to have done so.

This was the first time an AI had ever pulled such a trick on Ellis. It was not a good time for it. Ellis hung up and called the only other person she could think to contact: her sales rep. When he didn't answer, she left a voicemail and texts: no heat all weekend was unacceptable. She would really appreciate a call back.

While playing the horrible waiting game, Ellis tried to think about what she could do to fix this. They had told her not to mess with the thermostat. Well, from what she could see, the thermostat was sending a signal to the furnace that the furnace wasn't responding to for whatever reason. It was time to look at the docs. Fortunately, the new furnace's manual was resting right on top of it. She spread it open on her kitchen table.

OK, Ellis thought, this newfangled furnace has an LED display which displays status codes. Her old furnace had lacked such a thing. Lemme find that.

Inside her newly remodeled utility closet, she located the blinking display, knelt, and spied the code: 1dL. Looking that up in the doc's troubleshooting section, she found ... Normal Operation. No action.

The furnace was OK, then? Now what?

Aside from documentation, another thing Ellis knew pretty well was tech support. She decided to break out the ol' turn-it-off-and-on-again. She shut off power to both the furnace and thermostat, waited a few minutes, then switched everything back on, crossing her fingers.

No change. The indoor temperature kept dropping.

Her phone rang: the sales rep. He connected her with the on-call technician for that weekend, who fortunately was able to arrive at her house within the hour.

One tiny thermostat adjustment later, and Ellis was enjoying a warm house once more.

What had happened?

This is where an understanding of heat pumps comes into play. In this configuration, the heat pump is used for cooling and for heating, unless the outside temperature gets very cold. At that point, the furnace kicks in, which is more efficient. (Technology Connections has some cool videos about this if you're curious.)

Everything had been running fine for Ellis while the temperatures had remained below freezing. The problem came when, for the first time in approximately 12 years, the temperature rose above 40F (4C). At that point, the new thermostat decided, without telling Ellis, I'm gonna tell the HEAT PUMP to heat the joint!

... which couldn't do anything just then.

Workaround: the on-call technician switched the thermostat to an emergency heat mode that used the furnace no matter what.

Ellis had been told not to goof around with the thermostat. Even if she had, as a heat pump neophyte, she wouldn't have known to go looking for such a setting. She might've dug it up in a manual. Someone could've walked her through it over the phone. Oh, well. There is heat again, which is all that matters.

They will attempt to bring the heat pump online soon. We shall see if the story ends here, or if this becomes The WTF That Wouldn't Die.

P.S. When Ellis explained the AI answering service's deceptive behavior, she was told that the agent had been universally complained about ever since they switched to it. Fed up, they told Ellis they're getting rid of it. She feels pretty chuffed about more people seeing the light concerning garbage AI that creates far more problems than it solves.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

...sent us five wtfs. And so on anon.

Item the first, an anon is "definitely not qualified" for this job. "These years of experience requirements are getting ridiculous."

Item the second unearthed by a farmanon has a loco logo. "After reading about the high quality spam emails which are indistinguishable from the company's emails, I got one from the spammer just starting his first day."

In thrid place, anon has only good things to say: "I'm liking their newsletter recommendations so far."

"A choice so noice, they gave it twoice," quipped somebody.

And foinally, a tdwtfer asks "I've seen this mixmastered calendering on several web sites. Is there an OSS package that is doing this? Or is it a Wordpress plugin?" I have a sneaking suspicion I posted this before. Call me on it.

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

Barbara Booth, CNBC:

Civil liberties’ advocates warn that concentrating large volumes of identity data among a small number of verification vendors can create attractive targets for hackers and government demands. Earlier this year, Discord disclosed a data breach that exposed ID images belonging to approximately 70,000 users through a compromised third-party service, highlighting the security risks associated with storing sensitive identity information.

[…]

According to Tandy, as more states adopt age-verification mandates and companies race to comply, the infrastructure behind those systems is likely to become a permanent fixture of online life. Taken together, industry leaders say the rapid spread of age-verification laws may push platforms toward systems that verify age once and reuse that credential across services.

The hurried implementation of age verification sounds fairly terrible, counterproductive, illegal in the U.S., and discriminatory, but we should not pretend that we are only now being subject to risky and overbearing surveillance on the web. The ecosystem powering behavioural ad targeting — including data brokers, the biggest of which have reported staggering data breaches for a decade — has all but ensured our behaviour on popular websites and in mobile apps is already tracked and tied to some proxy for our identity.

That is not an excuse for the poor implementation of age verification, nor justification for its existence. If anything, it is a condemnation of the current state of the web that this barely moves the needle on privacy. If I had to choose whether to compromise for commerce or for the children, it would be the latter, but the correct answer is, likely, neither.

⌥ Permalink

Casey Newton, Platformer:

On Friday I learned to my surprise that I had become an editor for Grammarly. The subscription-based writing assistant has introduced a feature named “expert review” that, in the company’s words, “is designed to take your writing to the next level — with insights from leading professionals, authors, and subject-matter experts.”

Read a little further, though, and you’ll learn that these “insights” are not actually “from” leading professionals, or any human person at all. Rather, they are AI-generated text, which may or may not reflect whichever “leading professional” Grammarly slapped their names on.

Miles Klee, Wired:

As advertised on a support page, Grammarly users can solicit tips from virtual versions of living writers and scholars such as Stephen King and Neil deGrasse Tyson (neither of whom responded to a request for comment) as well as the deceased, like the editor William Zinsser and astronomer Carl Sagan. Presumably, these different AI agents are trained on the oeuvres of the people they are meant to imitate, though the legality of this content-harvesting remains murky at best, and the subject of many, many copyright lawsuits.

I do not think a disclaimer explaining it does “not indicate any affiliation with Grammarly or endorsement by those individuals or entities” will sufficiently distance the company from its claim of providing “insights from leading professionals, authors, and subject-matter experts” attributed to the names of people who did not agree to participate in this. Apparently, it is incumbent upon them to opt out by emailing expertoptout@superhuman.com. Most people will obviously not do this — because why would anyone realize they need to opt out? — but especially those who are dead yet are still being called upon for their expertise. Let Carl Sagan rest.

⌥ Permalink

Charlie Sorrel, of iFixit:

Apple’s MacBooks haven’t always been monolithic, barely repairable slabs of aluminum, glass, and glue. They used to be almost delightful in their repairable features, from their batteries to their Wi-Fi cards. Powerbooks, iBooks, and especially early MacBooks showed what happens when Apple applies its design skills directly to repairability and maintenance, instead of to thinness above all. Today we’re going to take a look at the best repairability features that Apple has ditched.

These four complaints range from the somewhat quaint — swappable Wi-Fi cards — to the stuff I actually miss, which is everything else. RAM and disk upgrades are a gimme since the cost-per-gigabyte (generally) declines over time, and I would love easily swappable batteries. But right now, nearly four years into owning this MacBook Pro, I would also really like to be able to swap in a new keyboard in the future. Not only are the keycaps unintentionally becoming polished, some oft-used keys feel a little mushy. Not much, and barely enough to notice, but I imagine their clickiness will not improve over time.

One quibble, emphasis mine:

[…] I have an old 2012 MacBook Air running Linux. I swapped the HDD for an SSD, maxed out the RAM, and dropped in a new battery, and I see no reason it wouldn’t easily keep rolling for another 10 years.

Unlikely. The 2012 MacBook Air only came with an SSD; a standard hard disk was not an option.

⌥ Permalink

Juli Clover, MacRumors:

Apple renamed the prior Reduce Highlighting Effects Accessibility setting to “Reduce Bright Effects,” and explained what it does.

Apple says the feature “minimizes highlighting and flashing when interacting with onscreen elements, such as buttons or the keyboard.

In my testing, this does exactly what you would expect. In places like toolbar buttons — or the buttons in the area of what is left of a toolbar, anyhow — the passcode entry screen, and Control Centre, the glowing tap effects are minimized or removed.

I do not find those effects particularly distracting, and I think turning them off saps some of the life out of the Liquid Glass design language, but I can see why some would be bothered by them. It is not the case that iOS 26 would be better if none of these appearance controls were present, only that they should not be necessary.

⌥ Permalink

There are three agreed-upon policies which, in the airy language of a government press release, seem reasonable enough to apply to all social platforms, yet are only relevant to TikTok. The first is exceedingly vague:

TikTok will implement enhanced protection for Canadians’ personal information, including new security gateways and privacy-enhancing technologies to control access to Canadian user data in order to reduce the risk of unauthorized or prohibited access.

There are no details about what the “new security gateways and privacy-enhancing technologies” are, nor why the sole goal is preventing “prohibited access” rather than “exploitative access”.

The second — complying with the recommendations of the Privacy Commissioner — was already underway, and the third is an “independent third-party monitor”, which seems fine.

⌥ Permalink

Do you want an all-in-one solution to block ads, trackers, and annoyances across all your Apple devices?

Then download Magic Lasso Adblock — the ad blocker designed for you.

With Magic Lasso Adblock you can effortlessly block ads on your iPhone, iPad, Mac, and Apple TV.

Magic Lasso is a single, native app that includes everything you need:

• Safari Ad Blocking — Browse 2.0× faster in Safari by blocking all ads, with no annoying distractions or pop ups

• YouTube Ad Blocking — Block all YouTube ads in Safari, including all video ads, banner ads, search ads, plus many more

• App Ad Blocking — Block ads and trackers across the news, social media, and game apps on your device, including other browsers such as Chrome and Firefox

• Apple TV Ad Blocking — Watch your favourite TV shows with less interruptions and protect your privacy from in-app ad tracking with Magic Lasso on your Apple TV

Best of all, with Magic Lasso Adblock, all ad blocking is done directly on your device, using a fast, efficient Swift-based architecture that follows our strict zero data collection policy.

With over 5,000 five star reviews, it’s simply the best ad blocker for your iPhone, iPad, Mac, and Apple TV.

And unlike some other ad blockers, Magic Lasso Adblock respects your privacy, doesn’t accept payment from advertisers, and is 100% supported by its community of users.

So, ensure your browsing history, app usage, and viewing habits stay private with Magic Lasso Adblock.

Join over 400,000 users and download Magic Lasso Adblock today.

⌥ Permalink

Nilay Patel and Liz Lopatto discussed “prediction markets” on the Verge’s “Decoder” podcast; here is Patel’s summary:

Insider trading is supposed to be illegal, and so is operating an unregulated sports book. So you’re now starting to see Kalshi and Polymarket getting hit from both sides of this broader regulatory debate, and 2026 is shaping up to be the year that all of this really comes to a head. To what end? It’s hard to say, especially as these companies cozy up to the Trump administration.

But it’s also becoming increasingly untenable for prediction markets to sit in the middle of the tension between gambling on the news and trying to self-regulate such that they don’t encourage insider trading.

A little under a month after Gallup announced it would stop polling for presidential approval, the Associated Press said it would begin integrating Kalshi bets into its election coverage. As Patel and Lopatto say, however, election betting is among the least problematic news gambling.

⌥ Permalink

Charlie Warzel, the Atlantic (gift link):

Prediction markets claim to harness the wisdom of crowds to provide reliable public data: Because people are putting real money behind their opinions, they are expressing what they actually believe is most likely to happen, which, according to the reasoning of these platforms, means that events will unfold accordingly. Many news organizations, and Substack, now have partnerships with prediction markets — the subtext being that they provide some kind of news-gathering function. Some users who distrust mainstream media turn to the markets in place of traditional journalism.

But in reality, prediction markets produce the opposite of accurate, unbiased information. They encourage anyone with an informational edge to use their knowledge for personal financial gain. In this way, prediction markets are the perfect technology for a low-trust society, simultaneously exploiting and reifying an environment in which believing the motives behind any person or action becomes harder.

I had no idea so-called “prediction markets” like Kalshi and Polymarket were promoting themselves as forecasters of real information, let alone that anyone believed them. I always assumed “prediction markets” was a euphemism.

A spokesperson for Kalshi told Warzel that betting on current events is a way to “create accurate, unbiased forecasts”, and that is something we can verify. If this were true, bettors should have been able to forecast, for example, the popular vote split of the 2024 U.S. presidential election. Polls had Harris and Trump neck and neck, but on election day, 75.8% of Kalshi bettors believed Harris would prevail. There is not much granularity to Kalshi’s charts, but the forecast on Polymarket was favourable to Harris at 5:00 PM on November 5 — election day — and it flips to a Trump lead at the next available data point, 5:00 AM the following day, and well after it was obvious Trump won the popular vote.

This is just a way to gamble on current events, which is tragic and pathetic. We do not need to pretend these sites are anything more substantial than that.

⌥ Permalink

Sameer Samat, Google’s president of Android Ecosystem:

Today we are announcing substantial updates that evolve our business model and build on our long history of openness globally.  We’re doing that in three ways: more billing options, a program for registered app stores, and lower fees and new programs for developers.

Epic Games CEO Tim Sweeney on X:

Google is opening up Android all the way with robust support for competing stores, competing payments, and a better deal for all developers. So, we’ve settled all of our disputes worldwide. THANKS GOOGLE!

Simon Sharwood, the Register:

Epic Games approved of the changes.

“These changes will evolve Android into a true open platform with competition among stores,” the company stated. “Globally, developers will have choices in how they make payments using Google Play’s payment system and competing payment systems, with reduced fees and the ability to point users outside apps to make purchases.

Epic also said “Google will take steps to support the future open metaverse,” a probable reference to the deal that will see games made with the Unity engine made available within Fortnite.

Neither Sweeney nor Epic Games can express anything less than elation with this outcome in no small part because they signed away their ability to do that. It still amazes me that concession ended up in the final agreement. It seems like the kind of thing that Google’s very expensive lawyers would pitch as leverage with Epic Games’ not-quite-as-expensive lawyers. In an interview with Dean Takahashi, of GamesBeat, it seems like Epic was eager to settle with terms that apply worldwide:

Asked why Sweeney decided to settle rather than litigate in every court in the world, he said, “This is just a really important thing that people should understand. The Epic versus Google court decision in the United States only has effect in the United States. It does nothing about the rest of the world. And the United States is about 30% of Google Play revenue and about 5% of Google Play users.”

He said it was never going to be a complete worldwide solution, and the court, throughout the proceedings, very clearly, said that the court wanted to establish competition among stores and competition among payments without setting prices in the market.

Curiously, not long before this settlement, Google announced it would begin requiring Android developers to be verified for their software to be installable, even by side-loading. I am curious if the combination of these changes meaningfully impacts users’ security or privacy. At a glance, the changes that settled this lawsuit seem like a welcome set of improvements that, sure, was assuredly not an altruistic fight by Epic Games, and will probably result in Sweeney getting even richer.

Regardless, it is notable for these sweeping changes to be brought to Android phones worldwide in the coming years, while Apple’s App Store is a patchwork of region-specific policies difficult for developers to navigate. It is too bad there is not really competition between these stores. Most people who buy smartphones choose the platform as a whole and accept whatever software experience they are provided. They do not need to bother themselves with the business terms of each store. With the improvements to third-party stores on Android, it sets up the possibility for greater competition within that platform. Apple should do the same.

⌥ Permalink

In hardware terms alone, Apple has been delivering an incredible run of Macs arguably since 2020, and easily since 2021. There are quibbles, sure — the display notch still bugs some people, the keyboard material wears poorly, and repairability has declined — but these are, overall, pretty sweet machines. The Macs announced this week seem like they will continue that hot streak.

I happen to be in the market for a new Mac, perhaps this year, and I should be spoiled for choice. I kind of am — the Mac Mini and Mac Studio are both alluring. But I am sadly attached to the room offered by my beloved 27-inch iMac, and Apple’s new lineup of displays is a sore point.

Stephen Hackett, 512 Pixels:

Yes, those are two different products, but they both feature 27-inch, 5K displays in the same enclosure as the previous Studio Display.

Starting at $1599, the new Studio Display is a slight upgrade to the 2022 model.

[…]

The much more interesting of the pair is the $3299 Studio Display XDR.

Those prices are, respectively, $2,100 and $4,500 in Canada. I am not a stranger to spending a lot of money on a screen — I bought a Thunderbolt Display at $1,000 — but that is a lot of money for even the basest of base models, especially since I have no idea whether the sketchy firmware issues have been resolved.

It is not that these displays are bad — far from it — but it is extraordinary that we are ten years removed from 27-inch Retina iMacs that started at just $200 more than the Studio Display is today. Only recently are we seeing more choice in 27-inch 5K displays at considerably lower prices, though without Apple’s very nice stand and quality of materials. At least the XDR has a seemingly new panel.

Three of the seven models in the Mac lineup require an external display. Apple has two choices: one really advanced one that costs as much as a generously-specced Mac Studio, and another that feels like it is stumbling along.

Anyway, here I go again looking for a sick deal I will not find on a Pro Display XDR. Those things really hold their value. Pity.

⌥ Permalink

Sean Hollister, the Verge:

But Google has finally muzzled Tim Sweeney. It’s right there in a binding term sheet for his settlement with Google.

On March 3rd, he not only signed away Epic’s rights to sue and disparage the company, he signed away his right to advocate for any further changes to Google’s app store polices. He can’t criticize Google’s app store practices. In fact, he has to praise them.

The terms (PDF) helpfully clarify that Epic is still allowed to “advocat[e] changes to the policies or practices of […] other companies, including Apple”. This does not mean future criticism of Apple’s business practices — or past criticism, for that matter — is unwarranted or invalid, but it now carries the blunted quality of someone who is not allowed to make the same complaints about Google.

⌥ Permalink

Google’s Threat Intelligence Group:

Google Threat Intelligence Group (GTIG) has identified a new and powerful exploit kit targeting Apple iPhone models running iOS version 13.0 (released in September 2019) up to version 17.2.1 (released in December 2023). The exploit kit, named “Coruna” by its developers, contained five full iOS exploit chains and a total of 23 exploits. The core technical value of this exploit kit lies in its comprehensive collection of iOS exploits, with the most advanced ones using non-public exploitation techniques and mitigation bypasses.

The Coruna exploit kit provides another example of how sophisticated capabilities proliferate. Over the course of 2025, GTIG tracked its use in highly targeted operations initially conducted by a customer of a surveillance vendor, then observed its deployment in watering hole attacks targeting Ukrainian users by UNC6353, a suspected Russian espionage group. We then retrieved the complete exploit kit when it was later used in broad-scale campaigns by UNC6691, a financially motivated threat actor operating from China. […]

Andy Greenberg, Wired:

Conspicuously absent from Google’s report is any mention of who the original surveillance company “customer” that deployed Coruna may have been. But the mobile security company iVerify, which also analyzed a version of Coruna it obtained from one of the infected Chinese sites, suggests the code may well have started life as a hacking kit built for or purchased by the US government. Google and iVerify both note that Coruna contains multiple components previously used in a hacking operation known as “Triangulation” that was discovered targeting Russian cybersecurity firm Kaspersky in 2023, which the Russian government claimed was the work of the NSA. (The US government didn’t respond to Russia’s claim.)

I am so curious to know how this thing made it outside the carefully guarded digital walls of the U.S. government or a contractor. While a rare event, it is not the first time the classified weapons of espionage have become public.

⌥ Permalink

Joseph Cox, 404 Media:

Customs and Border Protection (CBP) bought data from the online advertising ecosystem to track peoples’ precise movements over time, in a process that often involves siphoning data from ordinary apps like video games, dating services, and fitness trackers, according to an internal Department of Homeland Security (DHS) document obtained by 404 Media.

[…]

Although CBP described the move as a pilot, the DHS Office of the Inspector General (OIG) later found both CBP and ICE did not limit themselves to non-operational use. The OIG found that CBP, ICE, and the Secret Service all illegally used the smartphone location data, and found a CBP official used the data to track coworkers with no investigative purpose. CBP and ICE went on to repeatedly purchase access to location data.

There are people out there who will insist, to this day, that behaviourally targeted advertising is not actually a mechanism for surveillance despite all the evidence showing it is, in fact, an essential component.

⌥ Permalink

Naipanoi Lepapa, Ahmed Abdigadir, and Julia Lindblom, Svenska Dagbladet:

The workers in Kenya say that it feels uncomfortable to go to work. They tell us about deeply private video clips, which appear to come straight out of Western homes, from people who use the glasses in their everyday lives.

Several describe video material showing bathroom visits, sex and other intimate moments.

Another worker talks about people coming out of bathrooms.

It is appalling that massively rich corporations like Meta continue to offload critical tasks like these onto people who receive little support or pay. I recently finished “Ghost Work” by Mary L. Gray and Siddharth Suri and, while not my favourite book nor surfacing anything conceptually new, is worth your time. Meta can and should be doing far better, but can avoid association with labour atrocities better than, say, Nike in the 1990s in part because I doubt most people think too much about human intervention in artificial intelligence. Meta does not celebrate the hard work of its contract labour in Kenya; it does not even acknowledge them.

Speaking of not acknowledging the human labour involved, this story is the obvious nightmare you would expect. Some of these incidents of sensitive video recordings appear to be accidental, while others are seemingly deliberate. Without excusing the people who seem to be recording creepy videos on purpose, I assume few people would have believed it would be seen by someone at a company they probably have not heard about.

At first glance, it appears that we have significant control over our data. It states that voice recordings may only be saved and used for improvement or training of other Meta products if the user actively agrees.

But for the AI assistant to function, voice, text, image and sometimes video must be processed and may be shared onwards. This data processing is done automatically and cannot be turned off.

This is the kind of thing I would expect would be bundled into the additional diagnostic information Meta asks if you would like to opt into sharing. But Meta says this “does not include the photos and videos captured by your glasses”. That is, as this investigation found, part of the mandatory data collection.

This is offensive on behalf of users who might be less likely to consent if they had this full information. But it is also offensive to their romantic partners, friends, acquaintances, and passers-by, none of whom agreed to have their image or conversations adjudicated by these contractors.

⌥ Permalink

In a WWDC 2011 session, Dan Schimpf explained some of the goals of the refreshed design for Aqua in Mac OS X Lion were “meant to focus the user attention on the active window content”. This sentiment was echoed by John Siracusa in his review of Lion for Ars Technica:

Apple says that its goal with the Lion user interface was to highlight content by de-emphasizing the surrounding user interface elements.

When Apple redesigned Mac OS X again in 2014 with Yosemite, it promised…

[…] a fresh modern look where controls are clearer, smarter and easier to understand, and streamlined toolbars put the focus on your content without compromising functionality.

Then, when it revealed the Big Sur redesign in 2020, it explained:

The entire experience feels more focused, fresh, and familiar, reducing visual complexity and bringing users’ content front and centre.

And you will never guess what it promised in 2025 with the announcement of MacOS Tahoe and Liquid Glass, as introduced by Alan Dye:

Our goal is a beautiful new design that brings joy and delight to every user experience. One that’s more personal, and puts greater focus on your content — all while still feeling instantly familiar.

It is not just Apple, either. Here is Microsoft’s Jensen Harris at Build 2011 describing a key goal for the company’s then-new Metro design language:

Metro-style apps have room to breathe. They’re not about the chrome, they’re about the content. […] For years, Windows was always about adding stuff. We added bars, and panes, and doodads, and widgets, and gadgets, and bars — and stuff everywhere. And that’s how we defined our U.I., based on what new widgets we added. Now, we’ve receded into the background, and the app is sitting out there on the stage.

And later, as Microsoft rolled out app updates with its Fluent Design language, it described them in familiar terms:

With the updated OneDrive, your content takes center stage. The improved visual design reduces clutter and distractions, allowing you to focus on what’s important – your content.

This is a laudable goal if the opposite is, I assume, increasing the amount of clutter in user interfaces and making them more distracting. Nobody wants that. Then again, while the objective may be quite reasonable, there are surely different ways of achieving it — but Apple has embraced a single strategy: make the interface blend into the document. (I will be focusing on MacOS here as it is the platform I am most familiar with.)

Here is what a Pages document looks like running under Mac OS X Lion:

Here is that same document in a newer version of Pages running on MacOS Catalina, with the Yosemite-era design language that replaced the one that came before:

Here it is in the last version of Pages on MacOS Tahoe, using the design language introduced with Big Sur:

And, finally, the newest version of Pages on MacOS Tahoe using the current Liquid Glass visual design language:

There are welcome improvements in newer versions of this comparison, like the introduction of the “Format” panel on the right-hand side, which makes better use of widescreen landscape-oriented displays, and allows for larger controls. While I admire the density of the Lion-era screenshot, the mini-sized controls in that formatting menu are harder to click.¹

Overall, however, what Apple has done to Pages over this period of time is representative of a broader trend of minimizing the delineation of user interface elements from each other and the document itself. This is the only tool in the toolbox, and I am skeptical it achieves what Apple intends.

Compare again the two more recent screenshots against the ones that came before, and focus on the toolbar at the top of each. In the older two, there is a well-defined separation between the toolbar — the window itself — and the document. In the Big Sur visual language, however, the toolbar is the same bright white as the document. By Tahoe and the Liquid Glass language, there is barely a distinction; the buttons simply float over the document. And, bizarrely, that degrades further with the “Reduce Transparency” accessibility preference enabled:

(Also, no, your eyes do not deceive you: the icons in the drop cap menu, barely visible in the lower-right, are indeed pixellated.)

For me, this means a constant distraction from my document because the whole window has a similar visual language. As the toolbar and its buttons become one with the document, they lose their ability to fade into the background. In the two older examples, the contrast of the well-defined toolbar allows me to treat them as an entirely separate thing I do not need to pay attention to.

This is further justified by the lower contrast within those two older toolbars. In Lion, the grey background and moderately saturated colours are a quiet reminder of tools that are available without them being intrusive. The mix of shapes is a sufficient differentiator, something Apple threw away in the following screenshot. By making all the buttons literal and with the same bright background, the toolbar becomes a little more distracting — but at least it does not blend into the document. Without the context of the previous screenshot, the colours of each icon seem almost random, and I find the yellow-on-white “Table” button difficult to distinguish at a glance from the black-on-yellow-on-white “Comment” button.

The Big Sur-era design language is, frankly, an atrocious regression. The heterogeneous shapes may have returned, but in the form of monochromatic medium-grey icons set against a uniform white background. The icons are not bad, per se — though putting “Add Page” and “Insert” next to each other in this default toolbar layout, both represented by a plus sign, is a little confusing. But I will bet you would not guess that some of these are buttons, while others are pop-up buttons with a submenu.

Finally, there is Liquid Glass which, in its default form, has more contrast than the previous example; with “Reduce Transparency” enabled, which is how I use MacOS, it has even less. The buttons themselves have a greater amount of internal contrast with bigger, darker grey icons on a white background. This is preferential within the context of the toolbar compared to the thin, small, and low-contrast buttons in the past example, but it also means this toolbar has similar contrast to the document itself.

I would not go so far as to argue that Pages ’09 has a perfect user interface and that everything since has been a regression. The average colours used for the icon fill in both older toolbars generally fails accessibility contrast checks which, remarkably, the Big Sur design will pass. The icons in Pages ’09 rely on dark outlines and unique shapes to have sufficient contrast with the toolbar background. However, Apple has since discarded most variables it could change to design these interfaces. Every button contains an icon of a single uniform colour, within barely defined holding containers of the same shape, and without text labels by default.

This monochromatic look means any splash of colour is distracting. The yellow accent used in Pages is garish — though, thankfully, something that can mostly be mitigated by changing the Theme Colour in System Settings, under Appearance. (Unfortunately, the yellow background remains on the “Update” button in the most recent version of Pages regardless of the system accent colour.) But perhaps you also noticed the purple icon in the Liquid Glass screenshot above. Here is the full toolbar:

Those purple icons signify features that are part of Apple Creator Studio, a paid subscription to Pages and other applications that allows you to — in the order they are presented above — generate an image, artificially boost the resolution of an image, and access a stock image library. If you would like to insert one of your own images into your Pages document, that feature has been moved to the paperclip icon. Yes, it is a menu and not a button, despite lacking the disclosure triangle of the zoom menu right beside it, and it also reminds you about the “Content Hub” and “Generate Image” features. In Pages under Lion, colour was used in the icons to help guide the user as they complete a task — click the green thing to add a shape; click the darker yellow thing to add a table. Colour is not being used in the newer version to signify these are A.I. features, as the “Writing Tools” icon remains dark grey. In this version, the coloured icons are there to guide the user to premium add-ons regardless of whether they are currently paying for them.

I decided to focus on Pages for this comparison because it has lived so many different lives in MacOS. However, it is perhaps an imperfect representation for the rest of the system. Across Mac OS X Lion, for example, the toolbars of first-party applications like Finder and Preview almost exclusively use monochromatic icons. This has been true since Mac OS X Leopard, which also introduced barely differentiated folder icons. Some toolbars in Tiger, introduced two years prior, featured icons inside uniform capsule shapes. These were questionable ideas at the time, but they still retained defining characteristics. The capsules, for example, may have had a uniform shape, but contained within were full-colour icons. Most importantly, they were all clearly controls that were differentiated from the document.

Perhaps Apple has some user studies that suggest otherwise, but I cannot see how dialling back the lines between interface and document is supposed to be beneficial for the user. It does not, in my use, result in less distraction while I am working in these apps. In fact, it often does the opposite. I do not think the prescription is rolling back to a decade-old design language. However, I think Apple should consider exploring the wealth of variables it can change to differentiate tools within toolbars, and to more clearly delineate window chrome from document.

I have a document open in BBEdit right now named “2025-06-22 – MacOS SaaS.markdown”. I started drafting this thing last year about how Apple has transitioned its operating systems to something closer to a software-as-a-service model. I was trying to describe how the difference between major versions has become generally more modest since many features are rolled out across the year, and how — particularly on Apple’s non-Mac platforms — updates are more-or-less forced since the company stops digitally certifying older versions.

It is not a perfect comparison and not quite a fully-developed idea — note the difference between the filename and the last sentence above — but I thought it was going somewhere. Of course, you had no idea about this because I never published, which is why it must have seemed strange when I dropped a reference to software-as-a-service in the middle of my piece about software quality:

There was a time when remaining on an older major version of an operating system or some piece of software meant you traded the excitement of new features for the predictability of stability. That trade-off no longer exists; software-as-a-service means an older version is just old, not necessarily more reliable.

Riccardo Mori was understandably confused by this:

[…] I very much enjoy using older Mac OS versions, but not being able to browse the Web properly and securely, not being able to correctly sign in to check a Gmail account, not being able to fetch some RSS feeds because you can’t authenticate securely or establish a secure connection is very frustrating. Not having Dropbox work on my 2009 MacBook Pro running OS X 10.11 El Capitan is a minor annoyance and means I just won’t have access to certain personal files and that I’ll have to sync manually whatever I do on this other machine.

But if I put these two factors aside, there’s nothing about those older Macs, nothing about the older Mac OS versions they run that makes them less reliable. […]

What Mori explains as this paragraph continues is what I had meant to write at the time. What I should have written was this (emphasis mine):

There was a time when remaining on an older major version of an operating system or some piece of software meant you traded the excitement of new features for the predictability of stability. That trade-off no longer exists; an operating system on a software-as-a-service treadmill means an older version is just old, not necessarily more reliable.

The cycle of having a major new version ready to preview by June and shipping in September means the amount of time Apple spends focusing on the current version must necessarily shrink. How many teams at the company do you suppose are, right now, working on MacOS 26 when WWDC is a little over three months away? Engineering efforts are undoubtably beginning to prioritize MacOS 27. There are new features to prepare, after all.

So, yes, what Mori writes is what I was trying to express. I wish I had given that sentence a little more thought. Do read Mori’s piece — the second part, “On Software Frugality”, is thought-provoking.

⌥ Permalink

Jon Hicks, last year:

Music apps leave me wanting.

While I collect albums both physically (Vinyl + CD) and digitally (from Bandcamp), there are still missing pieces that streaming services provide: discovering new music, sharing playlists and seeing what friends are playing so that I can try their recommendations. They’re a valuable part of my listening habits, but none of them feel like ‘the one’. […]

I only stumbled across this today, but it remains a wonderful encapsulation of the state of music apps today. I share Hicks’ criteria, though I would add three things for myself:

1. More expansive metadata. I would like genres that work more like tags. An artist may generally make records in one genre, but different albums have different influences. Even individual songs may considerably differ in sound and style. This is the kind of thing that would help me make playlists or find songs that sound better together.

   This would be a management challenge across the tens of thousands of songs in my library, but I feel like integration with RateYourMusic and other databases might help partially automate this.

2. iPhone syncing over a wire. One of Hicks’ criteria is streaming and local library in the same app, and I completely agree. But I do not want anything — especially iPhone syncing — to be predicated on an assumption I have Apple’s first-party iCloud Music stuff turned on.

3. No lock-in. I want to be able to point it at my existing library and for things to just work. I would like to be able to import my entire setup from Music — all my playlists, including smart playlists, plus all my stats and ratings — and I would like it to be stored in a format some other application could read if I ever need to move to a different client in the future.

There are many indie apps that get close to this. I checked out Radiccio recently, but it unfortunately does not work with the iMac on which my music library is stored. Maybe that is the fourth criteria: backwards compatibility as far as possible.

Nobody has ever said I am easy to please.

⌥ Permalink

Germain Gauthier, et al., in a recent peer-reviewed paper in Nature:

Feed algorithms are widely suspected to influence political attitudes. However, previous evidence from switching off the algorithm on Meta platforms found no political effects. Here we present results from a 2023 field experiment on Elon Musk’s platform X shedding light on this puzzle. We assigned active US-based users randomly to either an algorithmic or a chronological feed for 7 weeks, measuring political attitudes and online behaviour. Switching from a chronological to an algorithmic feed increased engagement and shifted political opinion towards more conservative positions, particularly regarding policy priorities, perceptions of criminal investigations into Donald Trump and views on the war in Ukraine. In contrast, switching from the algorithmic to the chronological feed had no comparable effects. Neither switching the algorithm on nor switching it off significantly affected affective polarization or self-reported partisanship. […]

One can be pedantic about the use of “algorithmic” and “the algorithm” to describe a particular set of rules for recommending tweets, given that you could also say a reverse-chronological timeline is its own kind of algorithm. A simple one, to be sure, but an algorithm. I will not quibble with this.

Here is one thing I will be pedantic about, though: this study is not an examination of the “political effects of X’s feed algorithm”, as the title of the study suggests. It was conducted in 2023 — just a little bit after Elon Musk bought the platform and when it was still named Twitter. That is a long time ago in online platform terms, and the recommendations engine has probably changed a lot since — but almost certainly not in the direction of political even-handedness — even though the GitHub commit log suggests it has not been.

This study’s design seems better to me than a report published shortly after the 2024 U.S. presidential election, which I found limited and unconvincing.

There should always be a way for users to set a reverse-chronological timeline, and to opt out of recommendations features. We should be suspicious of any platform that refuses to trust us with control over our own experience.

⌥ Permalink

Samantha Ryan, “VP of Content” at Meta’s Reality Labs:

We’ve recently made some pretty big changes, including right-sizing our Reality Labs investment to ensure that our efforts remain sustainable over time. We’ve been in this space for over a decade, and we aren’t going anywhere. We’re in it for the long haul.

By “right-sizing”, Ryan means laying off ten percent of the Reality Labs workforce, and pouring money into the Ray-Ban partnership instead of metaverse initiatives. By “in it for the long haul”, Ryan means shifting the definition of the “metaverse” to meet Mark Zuckerberg’s latest obsession. They did not whiff by renaming the entire company around a crappy update to Second Life; you just are not getting it.

Ryan:

Our goal remains constant: to empower developers and creators as they build long-term, sustainable businesses. We used to have a pretty well-defined audience for VR, but as we’ve grown, we’ve attracted new audiences — who want different things — and the onus is on us to make sure that each of these distinct groups can find the apps and games that appeal to them.

That’s why we’re changing our roadmaps to increase your chances for success. We’re explicitly separating our Quest VR platform from our Worlds platform in order to create more space for both products to grow. We’re doubling down on the VR developer ecosystem while shifting the focus of Worlds to be almost exclusively mobile. By breaking things down into two distinct platforms, we’ll be better able to clearly focus on each.

Meta can say it is “doubling down on the V.R. developer ecosystem” all it wants, but it announced in January it would be shutting down its work-focused V.R. app with only a month’s notice, and it has cancelled third-party headsets. Now, it is saying Horizon Worlds is basically a phone app. Last February, Andrew Bosworth wrote in a memo about the importance of this very strategy:

[…] And Horizon Worlds on mobile absolutely has to break out for our long term plans to have a chance. […]

As I write this, Meta Horizon is the fifty-seventh most popular free game in the Canadian App Store, just two spots behind Hole.io, “the most addictive black hole game”. Maybe people do not, in general, want to wear a computer on their entire head — not for the thousands of dollars Apple is charging, and not for the hundreds Meta is.

⌥ Permalink

Omar Shahine:

For years, I’ve wanted a personal assistant. Someone who knows my preferences, manages my inbox, tracks my packages, and helps my family stay organized. The problem? Good assistants are expensive, require training, and still need constant direction.

So I built one. His name is Lobster. 🦞

The key insight that made this work wasn’t technical—it was conceptual. I stopped thinking “AI chatbot” and started thinking “new hire.”

I think this analogy is downright perfect.

When I first read this piece, my mind started to spin with all the things I could offload to my own digital personal assistant. Imagine how much time I could save by… wait. What could I use it for? Shahine says it helps summarize recent emails, figure out travel details, find event tickets, and more, all through iMessage conversations. This is a remarkable technical achievement. But what it drove home for me is how little I could ultimately relate to the scenarios presented by Shahine, even as I am trying to plan dinner with friends and a couple of trips later in the year.

Perhaps the same is true for you, too. Take a moment and think about what tasks you would give a personal assistant that can only work through software. Is it a long list? Is delegating checking your email saving you time? If you automate your vacation planning, does it make you happier than figuring that out alongside your partner or family? I am not saying Shahine is wrong or misguided. I just cannot see my life in this, and I do not think I am alone.

⌥ Permalink