Business trip to Hell

At 01:46 -0600 on February 15th, I was woken up by a chorus of beeping noises. Power had gone out, and two UPS systems plus the home alarm system were keen that I should know about it.

I was aware of the weather forecast and figured there was a good chance the outage would exceed the 15 minute capacity of the UPS batteries. (Now that’s an understatement.) I wandered downstairs and did a controlled shutdown of my work laptop and home server. I left the PS4 to fend for itself; if Lara Croft had to re-raid a tomb that was no big deal. I marveled at the thick carpet of snow visible out of the window, then I went back to bed.

When I woke up again around 06:00, the power was still out, and the house was uncomfortably cold. I realized that although we have gas heating, it doesn’t work if there’s no electricity to power the fans that blow the heated air. We were without heat as well as electricity.

The budgerigar is a hardy little bird. In the wild, they live in giant flocks in the central outback of Australia. It can get down to -5˚C in Alice Springs in winter, but a more typical range is from just above freezing to 45˚C, so budgies are generally happy with Texas weather. However, it’s also the case that they’re opportunistic breeders, and in winter Australian farmers sometimes have to shovel dead budgies out of water troughs. Pet budgies, of course, are domesticated, and likely not as hardy as their wild cousins.

What with heat rising, by midday downstairs was noticeably colder than upstairs, and our budgies were visibly shivering in their cage. The overnight temperature was predicted to drop to -14˚C. Rothko carried the cage upstairs, and we started discussing what we were going to do.

At this point, the roads were under a good 5cm of snow, more in places. Early in the storm some of it had melted and then re-frozen, so beneath the snow lurked random invisible sheets of ice. While news reports assure me that Austin does have gritting trucks and snowplows, I have literally never seen one. Also, since Austin is in the Texas hill country, there are lots of exciting hills. If we were going to go somewhere to ride out the storm, we weren’t going to go there in the Prius.

It’s at times like this that it really pays to have a strong social network. Rothko started pinging all of our friends to see who had heat, who had four-wheel drive. Our friend Joel was the closest person who still had heat; he offered to let us stay at his house, assuming we could get to it. It was just over 8km away.

Fortunately, we also know Michael, who is a long-distance truck driver. As well as an 18 wheeler, he has a big pickup with an extended cab — and keeps a set of snow tires for it. He was running mercy missions and could evacuate us, as long as we were ready. We scrambled to get as ready as possible.

As I put on my boots and my serious winter coat, I found myself thinking that we were lucky to have lived in New England and visited Minnesota in mid-winter. It means that we have all the clothing for -14˚C. A mere -14 with 5cm of snow is no big deal to a Minnesotan; I’ve been there on a day when the wind chill took the temperature below -40 degrees, which I never have any trouble remembering because it happens to be where the Celsius and Fahrenheit scales cross over. Here’s another data point: below -8˚C is the point at which your nose hairs start to freeze when you inhale.

The average Texan is not so prepared for prolonged sub-zero weather. We’ve had snow in Austin before, but for it to stay on the ground for multiple days is pretty much unheard of. The last time it has been this cold for this long was back in 1989. (The 2011 cold snap was mild by comparison, barely a centimeter of snow.) I couldn’t help thinking that a lot of people were going to die. Not only did they lack the gear, they also likely lacked an understanding of just how deadly cold weather can be.

And then there’s the driving. Texas has some of the worst drivers in regular weather, and most of them have no clue about how to drive safely in rain, let alone ice. People think their SUVs make them invincible, and they’ll speed down the freeway in the outside lane. Any time there’s icy weather you’ll see a bunch of them flipped over.

Our rescuers arrived at the house. The main birdcage would fit inside the truck’s cab, fortunately, so we didn’t have to transfer the birds to one of the travel cages. Michael took a careful route, avoiding hills — most hills seemed to have one or more stranded vehicles part of the way up them, blocking traffic. We were at Joel’s house before dark, and got the birdcage up into the spare room where we would be sleeping. A couple more of his friends arrived shortly afterward, along with their dog and cat, and the house was at capacity: five humans, four dogs, three birds, and a cat. I got into a pair of thick waffle-weave pajamas, then into a thermal union suit with hood, crawled under some blankets, and slept like the dead.

On Tuesday, 4 million people in Texas were without power. For most of them, that meant they were without heat, as forced air heating is the norm. Stores were without power, which meant many people were low on food. Water pipes had frozen, and many were also without water. Natural gas supplies were close to failing.

Some of the more Trumpian Republicans were quick to point the finger at wind power, saying that the problem was reliance on wind turbines that had frozen over. Texas’s wind turbines indeed froze; however, that had a lot to do with the fact that the state had opted not to bother paying for the optional cold weather package. I’ve driven past huge wind farms in Minnesota, they work just fine in the middle of winter.

Only 7% of the winter energy supply for Texas was expected to be sourced from wind power anyway, and lost wind power was just 13% of the lost energy capacity. Most of the problem was non-renewable sources failing. Gas lines froze; pumps froze; instrumentation systems failed.

Think nuclear is the answer? One of the newest reactors in the country failed because two feedwater pumps froze up; a safety tripped because of low steam pressure, and the unit shut down. The reactor was built with the turbines exposed to the open air, because it’s not like it will ever get cold in Texas, right?

The same cost-cutting on infrastructure is why the water supplies failed, it’s why the roads weren’t clear, it’s why there wasn’t enough spare capacity to keep the electricity grid running.

Oh, and about that grid. You might have thought that Texas would have been able to buy power from neighboring states; but no, Texas has its own power grid, isolated from the rest of the US. That’s deliberate policy, so that the federal government can’t have any say in regulating the system and forcing operators to do profit-reducing things like installing cold-weather equipment.

Texas deregulated its electrical grid in 2002, a scheme architected by William W Hogan, author of smug articles about how the free market ensures that Texas will never have power generation problems like California. The grid is run by an organization called ERCOT, a corporation with a board made up mostly of representatives from the big private power generation companies.

People in ERCOT areas have a choice of electricity suppliers, pay more for their electricity, and have more blackouts. (When I worked at IBM, the propensity of the Dallas sites to lose power was a regular problem.) People in Austin and San Antonio have power generation run by their local government; they pay less, get better prices for any excess solar-generated power they feed back to the grid, and get bigger rebates for energy efficiency measures. That’s why San Antonio and Austin have more solar power deployed than the rest of the state.

In most parts of the US, power companies are required to have 15% excess capacity in reserve. But thanks to deregulation and lack of federal oversight, Texas power companies are not required to have any spare capacity. So when power stations started going offline, ERCOT quickly found itself minutes away from the entire Texas grid failing. The massive shutdown that resulted was the system operating exactly the way William W Hogan designed it to.

While Austin had planned to use rotating outages to avoid exceeding its generation capacity, ERCOT told the city they weren’t allowed to and had to keep the power off indefinitely. The parts of Austin that got to keep power were those connecting vital infrastructure, like a hospital — or the downtown government building.

It may or may not be significant that our friend Joel’s house is less than a mile from ERCOT’s offices.

Monday had been a holiday, Tuesday was a workday. I’d grabbed a laptop on the way out the door, and fortunately I’d also grabbed my headset, because it turned out I was going to have to handle any online technical meetings — our CEO was without power, keeping warm by gathering the family around a fireplace.

If you’re interested in conference call headsets, I can definitely recommend the Poly Blackwire 8225. I’d picked it after seeing a YouTube video where a Poly employee has an intelligible conversation while running a blender on his desk. If you need to be able to lead international conference calls with important potential customers while sitting next to a cage full of angry parakeets in a house full of barking dogs, the Poly 8225 will not disappoint. (It also works fine with Linux as long as you have a Mac or a Windows box available for the initial setup.)

Getting enough focus to do any coding proved impossible, but I was at least able to deal with e-mail and support tickets and a design meeting as well. It wasn’t a productive work week, by any means, but nothing blew up. In fact, it was curiously like some sort of hellish business trip, tired and stressed and sleeping in an unfamiliar bed and wanting nothing more than to be back in the office.

Some time on Wednesday the security system for the house came back online, according to the remote control app on my phone. I reasoned that power had been restored. With the power back, the heating would come on automatically and start to warm the place back up to habitable temperature — our old heating system used to need someone to go up into the attic and reset it every now and again, but the new HVAC system we’d been forced to install in the hellish summer of 2020 had been completely reliable so far.

The downside of the heat likely being back was that it was time to start worrying about burst pipes. Unfortunately, the roads were thoroughly iced over and it seemed unwise to try to return home. Power seemed to go off and on for the next day or so, or maybe the network connection was failing? Hard to say.

Investigating the local neighborhood revealed that supermarkets were either closed because of lack of power or were surrounded by long lines of desperate people waiting for over an hour in the freezing cold to try to get food. Convenience stores were mostly operating on a “cash only” basis, even when they had power. Fortunately, we had enough food to last for days, so we all settled in for another day and tried not to think about burst pipes.

On Thursday the temperature rose slightly above freezing, and it looked like the roads might be safe enough for regular vehicles. Joel drove us over to check out the situation. We saw crashed and abandoned vehicles, including a pickup that had spun out trying to exit the freeway, collided with the concrete walls, and been left almost blocking the exit.

I approached the front door with trepidation. We were lucky. We were very, very lucky. The house had warmed up to normal temperature. There was no visible sign of flooding anywhere. Amazingly, every single faucet, toilet and shower seemed to work. There was even hot water.

Snow was falling, more snow was predicted. There was going to be another sub-zero night. However, we were pretty sure that the house would stay warm enough overnight even if the power failed again, and Friday was predicted to be back to normal temperatures. We decided to return home.

This time the driving was going to be my problem. Again I found myself feeling strangely fortunate to have experienced Minnesota winters — I’ve driven in a blizzard, on snow and ice, in the middle of the night, so as long as we carefully avoided hills I was pretty sure we’d be fine. We got a shovel and broom out, broke up the sheets of ice on the sloping driveway to our garage, and cleared two paths down to the concrete where the tires would need to go.

At Joel’s house, we cleared another path so that we wouldn’t have to carry the birdcage over treacherous patches of ice. We had to lay it down flat in the trunk of the car, but at least with a hatchback we had that option. I drove us home carefully, we put our avian companions back in their usual spot and let them out of the cage, and then we crashed into bed.

Friday, everything was still OK. Other than having to boil our water, things were pretty much normal. Again, we’ve been lucky — there are many people still without water, some still without electricity. We’ve got plenty of food to get us through the weekend, we can wait until Monday or Tuesday to restock; some people need the supermarkets’ dwindling supplies much more than we do. I checked with friends to see if anyone was still in need of heat, water, or a chance to recharge batteries.

It’s now midday on Saturday. Snow is still melting, roads are pretty much clear. We’re doing our best to avoid using water, though I had to give in and take a 2 minute shower to shampoo my hair after being forced to go over a week without washing it. The electricity situation is officially back to normal, though Austin Energy is asking people to conserve still.

Rick Perry has said that that Texans would rather go for more than 3 days without electricity than have the federal government be allowed to regulate the Texas power grid. I suspect he has misread the room. This crisis has been a massive failure by ERCOT and the Texas legislature, many people have died, and even Greg Abbott seems to realize that heads will have to roll. ERCOT scrubbed all the names from their web site, but were forced to restore them when it was pointed out that it’s considered public information.

Still, I’m highly confident that Texas Republicans won’t actually learn anything from this experience. Once there are some resignations and a few token “reforms”, (big) business will go back to usual and we’ll be left unprepared for the next big winter storm. And thanks to climate change, it won’t be as long a wait as it was this time.