You are right, the countries that develop the models tend to tune it for their country’s weather. They tend to tune things that make the most improvement to their country, e.g., for Canada it might be snow storms, for USA it might be hurricanes?? But they tune physics, numerical methods, and parameters. These are still location agnostic in that you might get different accuracy for different terrains and weather phenomena, e.g., lakes vs valleys vs sea side, storms vs high pressure systems.
It’s not that simple. You have to first precisely define what you want. “best for my location” is too broad. If you want to know this for all model predictions, then you have a mountain to climb.
I have thought about doing this for my location but I was simply going to look at current temperature (not temperature in the future - that’s another dimension). I would just manually combine the model predictions and see what kind of accuracy my crude MOS gives. If it’s accurate enough - then great, if not then I’d decide to give up or use better statistical techniques.
I would do this using a simple Python program but you could do this is Excel.
Sure, sure, as with anything, we can focus on aggregates or specifics. Like anyone else, some metrics–temperature, precipitation–are more important to me than others.
Wait, how is current temperature a prediction?? Presumably, forecasting accuracy is defined by the deviation of forecasts from actual (when the forecasted datetime becomes current), no?
Let’s say GFS predicts the current temperature to be 27.7C but the thermometer in my house says 30.8C - that is a difference of 3.1C. Assuming the GFS model has a temperature bias (as opposed to a random error) for my location, I can just add this 3.1C bias to all GFS predictions. Boom - you have a very basic MOS predictor and you can apply it to future GFS predictions.
Tomorrow at the same time, GFS predicts 27.7 again, so I’ll add the bias and predict it’ll be 30.8.
The day after at GFS predicts 27.2C, so I’ll add the bias and predict it’ll be 30.3.
Now you can keep calculating the bias at different times of the day and over the next week to get a more refined value for the bias.
You can calculate a bias for every hour of the day. You can calculate the bias depending on wind direction and speed.
Then you could calculate the bias for each of GDPS and ICON, then you can take the weighted average of these MOS predictions.
As you can see your MOS predictor can be as simple or as complex as you want.
This is why I suggested starting with a simple Excel spreadsheet.
How?? Isn’t the object of prediction something in the future? Do you mean ‘Let’s say GFS predicted the current temperature’? Or by “predicts” do you estimates?
Regardless, I don’t have a private weather station. I don’t even have an outdoor thermometer! I simply look at the current estimate on some app on my phone. And it’s against those current estimates that I was looking for a comparison of forecasting accuracy of the different models.
This is a very simple example of a MOS predictor for illustration. If you understand this concept, then you can extrapolate it to be more complex.
If we assume the bias in the simulation predictions are constant then we can apply it for all predictions in the future. In other words, GFS always predicts 3.1C too low.
Let assume that the bias is not constant, then we need to create a more complex MOS predictor. We record the temperature now and compare it to GFS prediction for the current time but for a forecast from 24 hours ago. For illustration, you might find GFS always predicts the temperature to be 2.3C too low for predictions 24 hours ahead. Now you have a MOS predictor that can correct predictions for 24 hours ahead.
Then you’re probably on a futile mission. Who are you going to trust for the correct temperature at your house? Wunderground? Accuweather? All these models are based on some weather station somewhere, probably your closest airport or a combination of weather stations around you.
If you’re really wanting to go to all this effort, then you should probably buy a cheap thermometer ($10-$20 from Amazon) and start an Excel spreadsheet.
I don’t feel I need to “trust” any of them, and am not fussed about nailing the exact temperature at my house, as I’m not home all day (hence not much of a need for an outdoor thermometer). I’m happy with a general temp estimate in my area, and in the back of my head I sorta know that, depending on topograohy, surrounding vegetation and construction, some places therein–even some rooms in my house!–are warmer or colder than others. So I’m happy with regional temp readings (most likely coming from an airport nearby) in some generic weather app. And when I want to have a more granular reading, from private weather stations nearby, I look at Wunderground.
But I feel I didn’t get across what I was looking for (and please feel free to disengage, if this doesn’t interest you). All I was asking for was a site or a study that backtested models forecasts (against current, i.e. actual, measurements/estimates, of course), and compared them on various stats (not only bias, but also accuracy, consistency, etc). Presumably, every mathematical model is calibrated to minimise error; I just wanted to know what the error rates are for each them in my location.
Those sites and studies (and competitions) exist but without being precise on what “best” is you may not find what you’re looking for because no prediction is the best for all variable and all times for all season and for all locations. For example, one might be score 9 for temperature and 5 for precipitation, and another might score 6 for temperature and 8 for precipitation, so how do you weight those scores to get a total? And that is just two variables.
Of course. That’s mostly true of any product. Any product benchmarking exercise ranks and compares competitors along various criteria. That’s the intro of any such study: defining the metrics and dimensions of comparison. We, consumers, generally typically take those on board, and weigh the criteria: what’s more important to me, horsepower or fuel economy? Temp or precipitation accuracy? That’s just par for the course.