April 10, 2023
How are we still getting caught in the rain?
Technologically speaking, we live in a time of plenty. Today, I can ask a chatbot to render The Canterbury Tales as if written by Taylor Swift or to help me write a factually inaccurate autobiography. With three swipes, I can summon almost everyone listed in my phone and see their confused faces via an impromptu video chat. My life is a gluttonous smorgasbord of information, and I am on the all-you-can-eat plan. But there is one specific corner where technological advances haven’t kept up: weather apps.
Weather forecasts are always a game of prediction and probabilities, but these apps seem to fail more often than they should. At best, they perform about as well as meteorologists, but some of the most popular ones fare much worse. The cult favorite Dark Sky, for example, which shut down earlier this year and was rolled into the Apple Weather app, accurately predicted the high temperature in my zip code only 39 percent of the time, according to ForecastAdvisor, which evaluates online weather providers. The Weather Channel’s app, by comparison, comes in at 83 percent. The Apple app, although not rated by ForecastAdvisor, has a reputation for off-the-mark forecasts and has been consistently criticized for presenting faulty radar screens, mixing up precipitation totals, or, as it did last week, breaking altogether. Dozens of times, the Apple Weather app has lulled me into a false sense of security, leaving me wet and betrayed after a run, bike ride, or round of golf.
People love to complain about weather forecasts, dating back to when local-news meteorologists were the primary source for those planning their morning commutes. But the apps have produced a new level of frustration, at least judging by hundreds of cranky tweets over the past decade. Nearly two decades into the smartphone era—when anyone can theoretically harness the power of government weather data and dissect dozens of complex, real-time charts and models—we are still getting caught in the rain.
Weather apps are not all the same. There are tens of thousands of them, from the simply designed Apple Weather to the expensive, complex, data-rich Windy.App. But all of these forecasts are working off of similar data, which are pulled from places such as the National Oceanic and Atmospheric Administration (NOAA) and the European Centre for Medium-Range Weather Forecasts. Traditional meteorologists interpret these models based on their training as well as their gut instinct and past regional weather patterns, and different weather apps and services tend to use their own secret sauce of algorithms to divine their predictions. On an average day, you’re probably going to see a similar forecast from app to app and on television. But when it comes to how people feel about weather apps, these edge cases—which usually take place during severe weather events—are what stick in a person’s mind. “Eighty percent of the year, a weather app is going to work fine,” Matt Lanza, a forecaster who runs Houston’s Space City Weather, told me. “But it’s that 20 percent where people get burned that’s a problem.”
No people on the planet have a more tortured and conflicted relationship with weather apps than those who interpret forecasting models for a living. “My wife is married to a meteorologist, and she will straight up question me if her favorite weather app says something different than my forecast,” Lanza told me. “That’s how ingrained these services have become in most peoples’ lives.” The basic issue with weather apps, he argues, is that many of them remove a crucial component of a good, reliable forecast: a human interpreter who can relay caveats about models or offer a range of outcomes instead of a definitive forecast.
Lanza explained the human touch of a meteorologist using the example of a so-called high-resolution forecasting model that can predict only 18 hours out. It is generally quite good, he told me, at predicting rain and thunderstorms—“but every so often it runs too hot and over-indexes the chances of a bad storm.” This model, if left to its own devices, will project showers and thunderstorms blanketing the region for hours when, in reality, the storm might only cause 30 minutes of rain in an isolated area of the mapped region. “The problem is when you take the model data and push it directly into the app with no human interpretation,” he said. “Because you’re not going to get nuance from these apps at all. And that can mean a difference between a chance of rain all day and it’s going to rain all day.”
But even this explanation has caveats; all weather apps are different, and their forecasts have varying levels of sophistication. Some pipe model data right in, whereas others are curated using artificial intelligence. Peter Neilley, the Weather Channel’s director of weather forecasting sciences and technologies, said in an email that the company’s app incorporates “billions of weather data points,” adding that “our expert team of meteorologists does oversee and correct the process as needed.”
Weather apps might be less reliable for another reason too. When it comes to predicting severe weather such as snow, small changes in atmospheric moisture—the type of change an experienced forecaster might notice—can cause huge variances in precipitation outcomes. An app with no human curation might choose to average the model’s range of outcomes, producing a forecast that doesn’t reflect the dynamic situation on the ground. Or consider cities with microclimates: “Today, in Chicago, the lakefront will sit in the lower 40s, and the suburbs will be 50-plus degrees,” Greg Dutra, a meteorologist at ABC 7 Chicago, told me. “Often, the difference is even more stark—20-degree swings over just miles.” These sometimes subtle temperature disparities can mean very different forecasts for people living in the same region—something that one-size-fits-all weather apps don’t always pick up.
Naturally, meteorologists think that what they do is superior to forecasting by algorithm alone, but even weather-app creators told me that the challenges are real. “It’s impossible for a weather-data provider to be accurate everywhere in the world,” Brian Mueller, the founder of the app Carrot Weather, told me. His solution to the problem of app-based imprecision is to give users more ability to choose what they see when they open Carrot, letting them customize what specific weather information the app surfaces as well as what data sources the app will draw from. Mueller said that he learned from Dark Sky’s success how important beautiful, detailed radar maps were—both as a source of weather data and for entertainment purposes. In fact, meteorology seems to be only part of the allure when it comes to building a beloved weather app. Carrot has a pleasant design interface, with bright colors and Easter eggs scattered throughout, such as geography challenges based off of its weather maps. He’s also hooked Carrot up to ChatGPT to allow people to chat with the app’s fictional personality.
But what if these detailed models and dizzying maps, in the hands of weather rubes like myself, are the real problem? “The general public has access to more weather information than ever, and I’d posit that that’s a bad thing,” Chris Misenis, a weather-forecasting consultant in North Carolina who goes by the name “Weather Moose,” told me. “You can go to PivotalWeather.com right now and pull up just about any model simulation you want.” He argues that these data are fine to look at if you know how to interpret them, but for people who aren’t trained to analyze them, they are at best worthless and at worst dangerous.
In fact, forecasts are better than ever, Andrew Blum, a journalist and the author of the book The Weather Machine: A Journey Inside the Forecast, told me. “But arguably, we are less prepared to understand,” he said, “and act upon that improvement—and a forecast is only as good as our ability to make decisions with it.” Indeed, even academic research around weather apps suggests that apps fail worst when they give users a false sense of certainty around forecasting. A 2016 paper for the Royal Meteorological Society argued that “the current way of conveying forecasts in the most common apps is guilty of ‘immodesty’ (‘not admitting that sometimes predictions may fail’) and ‘impoverishment’ (‘not addressing the broader context in which forecasts … are made’).”
The conflicted relationship that people have with weather apps may simply be a manifestation of the information overload that dominates all facets of modern life. These products grant anyone with a phone access to an overwhelming amount of information that can be wildly complex. Greg Dutra shared one such public high-resolution model from the NOAA with me that was full of indecipherable links to jargony terms such as “0-2 km max vertical vorticity.” Weather apps seem to respond mostly to this fire hose of data in two ways: By boiling them down to a reductive “partly sunny” icon, or by bombarding the user with information they might not need or understand. At its worst, a modern weather app seems to flatter people, entrusting them to do their own research even if they’re not equipped. I’m not too proud to admit that some of the fun of toying around with Dark Sky’s beautiful radar or Windy.App’s endless array of models is the feeling of role-playing as a meteorologist. But the truth is that I don’t really know what I’m looking at.
What people seem to be looking for in a weather app is something they can justify blindly trusting and letting into their lives—after all, it’s often the first thing you check when you roll over in bed in the morning. According to the 56,400 ratings of Carrot in Apple’s App Store, its die-hard fans find the app entertaining and even endearing. “Love my psychotic, yet surprisingly accurate weather app,” one five-star review reads. Although many people need reliable forecasting, true loyalty comes from a weather app that makes people feel good when they open it.
Our weather-app ambivalence is a strange pull between feeling grateful for instant access to information and simultaneously navigating a sense of guilt and confusion about how the experience is also, somehow, dissatisfying—a bit like staring down Netflix’s endless library and feeling as if there’s nothing to watch. Weather apps aren’t getting worse. In fact they’re only getting more advanced, inputting more and more data and offering them to us to consume. Which, of course, might be why they feel worse.