NSSL/SPC Spring Experiment Blog

Sneak Peak Part 3: Modeled vs Observed reports

2011-12-20T22:01:00.003-06:00

I went ahead and used some educated guesses to develop model proxies for severe storms in the model. But how do those modeled reports compare to observed reports? This question, at least the way it is addressed here, yields an interesting result. Lets go to the figures:

The 2 images show the barchart of all the dates on the left, with the Modeled reports (top), observed reports close to modeled storms (middle) and the natural log of the pixels of each storm (or area; bottom) on the right. The 1st image has the modeled storm reports selected and it should be pretty obvious I have chosen unwisely (either the variable or the value) for my hail proxy (the reports with a 2 in the string). Interestingly, the area is skewed to the right or very large objects tend to be associated with model storms.

Also note that modeled severe storms are largest in the ensemble for 24 May with 27 Apr coming in 6th. 24 May appears first in percent of storms on that date with the 27 Apr outbreak coming in 15th place (i.e. having a lot of storms that are not severe).

Changing our perspective and highlighting the observed reports that are close to modeled storms, the storm area distribution switches to the left or smallest storm area.

The modeled storms to verify has 25 May followed by 27 Apr coming in with the most observed reports close by. 24 May lags behind in 5th place. In a relative sense, 27 Apr and 25 May switch places, with 24 May coming in 9th place.

These unique perspectives highlight two subtle but interesting points:
1. Modeled severe storms are more typically larger (i.e. well resolved),
2. Observed reports are more typically associated with smaller storms.

I believe there are a few factors at play here including the volume and spacing of reports on any particular day, and of course how well the model performs. 25 May and 27 Apr had lots of reports so they stand out. Plus all the issues associated with reports in general (timing and location uncertainty). But I think one thing also at work here is that these models have difficulty maintaining storms in the warm sector and tend to produce small, short-lived storms. This is relatively bad news for skill; but perhaps a decent clue for forecasters. I say clue because we really need a larger sample across a lot of different convective modes to make any firm conclusions.

I should address the hail issue noted above. I arbitrarily selected an integrated hail mixing ratio of 30 as the proxy for severe. I chose this value after checking out the 3 severe variable (hourly max UH > 100 m s-2 for tornadoes, hourly max wind > 25.7 m s-1, hourly max hail > 30) distributions. After highlighting UH at various thresholds it became pretty clear that hail and UH were correlated. So I think we need to look for a better variable so we can relate hail-fall to modeled variables.

Sneak Peak 2: Outbreak comparison

2011-12-19T23:34:00.000-06:00

I ran my code over the entire 2011 HWT data set to compare the two outbreaks from 27 April and 24 May amidst all the other days. These outbreaks were not that similar ... or were they?

In the first example, I am comparing the model storms that verified via storm reports with 40% for 27 April and only 17% for 24 May but 37% for 25 May. 25 May also had a lot of storm reports including a large number of tornado reports. Note the distribution of UHobj (upper left) is skewed toward lower values. The natural log of the pixel count per object (middle right) is also skewed toward lower values.
[If I further dice up the data set, requiring UHobj exceed 60, then 27 April has ~12%, 24 May has 7.8%, 25 May has 4% of the respective storms on those days (not shown). ]

In the second example, if I only select the UHobj greater than 60, the storm percentages for 27 Apr are 25%, 24 May are 35%, and 25 May are 8%. The natural log of the pixel count per object (middle right) is also skewed toward higher values. Hail and Wind parameters (middle left and bottom left, respectively) shift to higher values as well.

Very interesting interplay exists here since 24 May did not subjectively verify well (too late, not very many supercells). 27 Apr verified well, but had a different convective mode of sorts (linear with embedded supercells). 25 May I honestly cannot recall other than the large number of reports that day.

Comments welcome.

Sneak Peak from the past

2011-12-17T17:22:00.001-06:00

So after the Weather Ready Nation: A Vital Conversation Workshop, I finally have some code and visualization software working. So here is a sneak peak, using the software Mondrian and an object identification algorithm that I wrote in Fortran, applied via NCL. Storm objects were defined using a double threshold, double area technique. Basically you set the minimum Composite Reflectivity threshold, and use the second threshold to ensure you have a true storm. The area thresholds apply to the reflectivity thresholds so that you restrict storm sizes (essentially as a filter to reduce noise from very small storms).

So we have a few ensemble members from 27 April generated by CAPS which I was intent on mining. The volume of data is large but the number of variables was restricted to some environmental and storm centric perspectives. I added in the storm report data from SPC (soon I will have the observed storms).

In the upper left is a barchart of my cryptic recording of observed storm reports, below that is the histogram of hourly maximum surface wind speed, and below that is the integrated hail mixing ratio parameter. The two scatter plots in the middle show the (top) CAPE-0-6km Shear product versus the hourly maximum updraft helicity obtained from a similar object algorithm that intersects with the storm, and the (bottom) 0-1km Storm Relative Helicity vs the LCL height. The plots to the right show the (top) histogram of model forecast hour, (bottom) sorted ensemble member spinogram*, and (bottom inset) the log of the pixel count of the storms.

The red highlighted storms used a CASH value greater than 30 000 and UHobj greater than 50. So we can see interactively on all the plots, where these storms appear in each distribution. The highlighted storms represent 24.04 percent of the sample of 2271 storms identified from the 17 ensemble members over the 23 hour period from 1400 UTC to 1200 UTC.

Although the contributions from each member are nearly equivalent (not shown; cannot be gleaned from the spinogram easily), some members contribute more of their storms to this parameter space (sorted from highest to lowest in the member spinogram). The peak time for storms in this environment was at 2100 UTC with the 3 highest hours being from 2000-2200 UTC. Only about half of the modeled storms had observed storm reports within 45km**. This storm environment contained the majority of high hail values though the hail distribution has hints of being bimodal. The majority of these storms had very low LCL heights (below 500 m) though most were below 1500m.

I anticipate using these tools and software for the upcoming HWT. We will be able to do next day verification using storm reports (assuming storm reports are updated via the WFO's timely) and I hope to also do a strict comparison to observed storms. I still have work to do in order to approach distributions oriented verification.

*The spinogram in this case represents a bar chart where the length of the bar is converted to 100 percent and the width of the bar is the sample size. The red highlighting now represents the within category percentage.

**I also had to do a +/- 1 hour time period. An initial attempt to verify the tornado reports in comparison to the tornado tracks yielded a bit of spatial error. This will need to be quantified.

Forecast Soundings: A Look to the Future (Literally)

2011-07-26T23:47:00.002-05:00

One of the data visualization tools we utilized in the HWT-EFP this year is a way to view ensemble soundings. I put together a blog post about how we did this on my personal web site and thought I'd share that post here!

You can find the post here: http://www.patricktmarsh.com/2011/07/forecast-soundings-a-look-to-the-future/

More Data Visualization

2011-06-22T15:41:00.003-05:00

As jimmyc touched on in his last post, one of the struggles facing the Hazardous Weather Testbed is how to visualize the incredibly large datasets that are being generated. With well over 60 model runs available to HWT Experimental Forecast Program participants, the ability to synthesize large volumes of data very quickly is a must. Historically we have utilized a meteorological visualization package known as NAWIPS, which is the same software that the Storm Prediction Center uses for their operations. Unfortunately, NAWIPS was not designed with the idea it would be handling the large datasets that are currently being generated.

To help mitigate this, we utilized the Internet as much as possible. One webpage that I put together is a highly dynamical, CI forecast and observations webpage. This webpage allowed users to create 3, 4, 6, or 9 panel plots, with CI probabilities of any of 28 ensemble members, NSSL-WRF, or observations. Furthermore, users had the ability to overlay the raw CI points from any of the ensemble members, NSSL-WRF, or observations to see how the points contributed to the underlying probabilities. We even enabled it so that users could overlay the human forecasts to see how it compared to any of the numerical guidance or observations. This webpage turned out to be a huge hit with visitors, not only because it allowed for quick visualization of a large amount of data, but because it also allowed visitors to interrogate the ensemble from anywhere -- not just in the HWT.

One of the things we could do with this website is evaluate the performance of individual members of the ensemble. We could also evaluate how varying the PBL schemes affected the probabilities of CI. Again, the website is a great way to sift through a large amount of data in a relatively short amount of time.

Visualization

2011-06-12T22:33:00.000-05:00

We all did some web displays for various components of the CI desk. I built a few web displays based on object identification of precipitation areas. I counted up the objects per hour for all ensemble or all physics members (separate web pages) in order to 1) rapidly visualize the entire membership and 2) to add a non-map based perspective of when interesting things are happening. It also allows the full perspective of the variability in time, and variability of position and size of the objects.

The goal was to examine the models in multiple ways simultaneously and still investigate the individual members. This, in theory, should be more satisfying for forecasters as they get more comfortable with ensemble probabilities. It could alleviate data overload by giving a focused look at select variables within the ensemble. Variables that already have meaning and implied depth. Information that is easy to extract and reference.

The basic idea as implemented was to show the object count chart and upon mousing over a grid cell you can call up a map of the area with the precipitation field. At the upper and right most axes, you call up an animation of either all the models at a specific time OR one model at all times. The same concept was applied to updraft helicity.

I applied the same idea to the convection initiation points only this time there were no objects, just the raw number of points. I had not had time to visualize this prior to the experiment, so we used this as a way to compare two of the definitions in test mode.

The ideas were great, but in the end there were a few issues. The graphics were good in some instances because we started with no precipitation or updraft helicity or CI points. But if the region already had storms then interpretation was difficult, at least in terms of the object counts. This was a big issue with the CI points, especially as the counts increased well above 400, for a 400 by 400 km sub domain.

Another display I worked hard on was the so-called pdf generator. The idea was to use the ensemble to reproduce what we were doing, namely putting our CI point on the map where we thought the first storm would be. Great in principle, but automating this was problematic because we could choose our time window o fit the situation of the day. The other complication was that sometimes we had to make our domain small or big, depending on how much pre-existing convection was around. This happened quite frequently so the graphic was less applicable, but still very appealing. It will take some refinement but I think we can make this a part of the verification of our human forecasts.

I found this type of web display to be very useful and very quick. It also allows us to change our perspective from just data mining to information mining and consequently to think more about visualization of the forecast data. There is much work to be done in this regard and I hope some of these ideas can be further built upon for visualization and Information Mining so they can be more relevant to forecasters.

Certainty, doubt, and verification

2011-06-08T22:27:00.000-05:00

Today's forecast on CI focused on the area from northeast KS southwest along a front down towards the TX-OK panhandles. It was straightforward enough. How far southwest will the cap break? Will there be enough moisture in the warm sector near the frontal convergence? Will the dryline serve as a focus for CI, given the development of a dry slot present just ahead of the dryline along the southern extent of the front and a transition zone (reduced moisture zone)?

So we went to work mining the members of the ensemble, scrutinizing the deterministic models for surface moisture evolution, examining the convergence fields, and looking at ensemble soundings. The conclusion from the morning was two moderate risk areas: one in northeast KS and another covering the triple point, dryline, and cold front. The afternoon forecast backed off the dryline-triple point given the observed dry slot and the dry sounding from LMN at 1800 UTC.

The other issue was that the dryline area was so dry and the PBL so deep that convective temperature would be reached but with minimal CAPE (10-50 J kg-1). The dry LMN sounding was assumed to be representative of the larger mesoscale environment. This was wrong, as the 00 UTC sounding at LMN indicated an increase in moisture by 6 g/kg aloft and 3 at the surface.

Another aspect to this case was our scrutiny of the boundary layer and the presence of open-cell convection and horizontal convective rolls. We discussed, again, that at 4km grid spacing we are close to resolving these types of features. We are close because of the scale of the rolls (in order to resolve them they need to be larger than 7times the grid spacing) which scales with the boundary layer depth. So a day like today where the PBL is deep, the rolls should be close to resolvable. On the other hand, there is a need for additional diffusion in light wind conditions and when this does not happen, the scale of the rolls collapses to the scale of the grid. In order to believe the model we must take these considerations into account. In order to discount the model, we are unsure what to look for besides indications of "noise" (e.g. features barely resolved on the grid, scales of the rolls being close to 5 times the grid spacing).

The HCRs were present today as per this image from Wichita:

However, just because HCRs were present does not mean I can prove they were instrumental in CI. So when we saw the forecast today for HCRs along the front, and storms developed subsequently, we had some potential evidence. Given the distance from the radar, it may be difficult if not impossible, to prove that HCRs intersected the front, and contributed to CI.

This brings up another major point: In order to really know what happened today we need a lot of observational data. Major field project data. Not just surface data, but soundings, profilers, and low level radar data. On the scale of The Thunderstorm Project, only for numerical weather prediction. How else can we say with any certainty that the features we were using to make our forecast were present and contributing to CI? This is the scope of data collection we would require for months in order to get a sufficient amount of cases to verify the models (state variables and processes such as HCRs). Truly an expensive undertaking, yet one where a number of people could benefit from one data set and the field of NWP could improve tremendously. And lets not forget about forecasters who could benefit from having better models, better understanding, and better tools to help them.

I will update the blog after we verify this case tomorrow morning.

Wrong but verifiable

2011-06-06T23:21:00.000-05:00

The fine resolution guidance we are analyzing can get the forecast wrong yet probabilistically verify. It may seem strange but the models do not have to be perfect, they just have to be smooth enough (tuned, bias corrected) to be reliable. The smoothing is done on purpose to account for the fact that the discretized equations can not resolve more than 5-7 times the grid spacing. It is also done because the models have little skill below 10-14 times the grid spacing. As has been explained to me, this is approximately the scale at which the forecasts become statistically reliable. An example forecast of a 10 percent probability, in the reliable sense, will verify 10 percent of the time.

This makes competing with the model tough unless we have skill at deriving not only similar probabilities, but placing those probabilities in close proximity in space-time relative to observations. Re-wording this statement: Draw the radar at forecast hour X probabilistically. If you draw those probabilities to cover a large area you wont necessarily verify. But if you know the number of storms, their intensity, their longevity, and place them close to what was observed you can verify as well as the models. Which means, humans can be just as wrong but still verify their forecast well.

Let us think through drawing the radar. This is exactly what we are trying to do, in a limited sense, in the HWT for the Convection Initiation and Severe Storms Desks over 3 hour periods. The trick is the 3 hour period over which the models and forecasters can effectively smooth their forecasts. We isolate the areas of interest, and try to use the best forecast guidance to come up with a mental model of what is possible and probable. We try to add detail to that area by increasing the probabilities in some areas and removing some for other areas. But we still feel we are ignoring certain details. In CI, we feel like we should be trying to capture episodes. An episode is where CI occurs in close proximity to other CI in a certain time frame presumable because of a similar physical mechanism.

By doing this we are essentially trying to provide context and perspective but also a sense of understanding and anticipation. By knowing the mechanism we hope to either look for that mechanism or symptoms of that mechanism in observations in the hopes of anticipating CI. We also hope to be able to identify failure modes.

In speaking with forecasters for the last few weeks, there is a general feeling that it is very difficult to both accept and reject the model guidance. The models don't have to be perfect in individual fields (correct values or low RMS error) but rather just need to be relatively correct (errors can cancel). How can we realistically predict model success or model failure? Can we predict when forecasters will get this assessment incorrect?

Adding Value

2011-06-06T21:17:00.003-05:00

Today the CI (convective initiation) forecast team opted to forecast for northern Nebraska, much of South Dakota, southern and eastern North Dakota, and far west-central Minnesota for the 3-hr window of 21-00 UTC. The general setup was one with an anomalously deep trough ejecting northeast over the intermountain West. Low-level moisture was not all that particularly deep as a strong, blocking ridge had persisted over the southern and eastern United States for much of the past week. With that said, the strength of the ascent associated with the ejecting trough, the presence of a deepening surface low, and a strengthening surface front was such that most numerical models insisted that precipitation would break out across the CI forecast domain. The $64,000 question was, "Where?".

One model in particular, the High-Resolution Rapid Refresh (HRRR) model insisted that robust storm development would occur across central and north-eastern South Dakota during the late afternoon hours. It just so happened that this CI episode fell in outside the CI team's forecast of a "Moderate Risk" of convective initiation. As the CI forecast team poured over more forecast information than any single individual could possibly retain, we could not make sense of the how or why the HRRR model was producing precipitation where it was. The environment would (should?) be characterized by decreasing low-level convergence as the low-level wind fields responded to the strengthening surface low to the west. Furthermore, the surface front (and other boundaries) were well removed from the area. Still, several runs of the HRRR insisted storms would develop there.

It's situations like this where humans can still improve upon storm-scale numerical models. By monitoring observations, and using the most powerful computers in existence (our brains), humans can add value to numerical forecasts. Knowing when to go against a model, or knowing when it is important to worry about the nitty-gritty details of a forecast, are important traits that good forecasters have to have. Numerical forecasts are rapidly approaching the point where on a day-to-day basis, humans are hard pressed to beat them. And, in my opinion, forecasters should not be spending much time trying to determine if the models are wrong by 1 degree Fahrenheit for afternoon high temperatures in the middle of summer in Oklahoma. Even if the human is correct and improves the forecast, was there much value added? Contrast this with a forecaster improving the forecast by 1F when dealing with temperatures around 31-33F and precipitation forecast. In this case the human can add a lot of value to the forecast. Great forecasters know when to to accept numerical guidance, and when there is an opportunity to improve upon it (and then actually improve it). Today, that's just what the CI forecast team did. The HRRR was wrong in it's depiction of thunderstorms developing in northeast South Dakota by 00 UTC (7 PM CDT), and the humans were right...

...and as I write this post at 9:30 PM CDT, a lone supercell moves slowly eastward across northeastern South Dakota. Maybe the HRRR wasn't as wrong as I thought...

Timing

2011-06-02T21:06:00.000-05:00

It is remarkably difficult to predict convection initiation. It appears we can predict, most times (see yesterdays post for a failure), the area under consideration. We have attempted to pick the time period, in 3 hour windows, and have been met with some interesting successes and failures. Today had 2 such examples.

We predicted a time window from 16-19 UTC along the North Carolina/South Carolina/ Tennessee area for terrain induced convection and along the sea breeze front. The terrain induced storms went up around 18 UTC, nearly 2 hours after the model was generating storms. The sea breeze did not initiate storms, but further inland in central South Carolina there was one lone storm.

The other area was in South Dakota/North Dakota/Nebraska for storms long the cold front and dryline. We picked a window between 21-00 UTC. It appears storms initiated right around 00 UTC in South Dakota but little activity in North Dakota as the dryline surged into our risk area. Again the suite of models had suggested quick initiation starting in the 21-22 UTC time frame, including the update models.

In both cases we could isolate the areas reasonably well. We even understood the mechanisms by which convection would initiate, including the dryline, the transition zone, and the where the edge of the deeper moisture resided in the Dakotas. For the Carolinas we knew the terrain would be a favored location for elevated heating in the moist air mass along a weak, old frontal zone. We knew the sea breeze could be weak in terms of convergence, and we knew that only a few storms would potentially develop. What we could not adequately do, was predict the timing of the lid removal associated with the forcing mechanisms.

It is often observed in soundings that the lid is removed via surface heating and moistening, via cooling aloft, or both processes. It is also reasonable to suspect that low level lifting could be aiding in cooling aloft (as opposed to cold advection). Without observations along such boundaries it is difficult to know what exactly is happening along them, or even to infer that our models correctly depict the process by which the lid is overcome. We have been looking at the ensemble of physics members which vary the boundary layer scheme, but today was the first day we attempted to use them in the forecast process.

It was successful in terms of incorporating them, but as far as achieving understanding, that will have to come later. It is clear that understanding the various structures we see, and relating them to the times of storm initiations will be a worthwhile effort. Whether this will be helpful to forecasting, even in hindsight, is still unknown.

When too much is not enough

2011-06-01T21:23:00.001-05:00

Going into HWT today, I was thinking about and hoping for a straightforward (e.g. easy) forecast for storms. I was hoping for one clean slate area. An area where previous storms would not be an issue, where storms would take their time forming, and where the storms that do form would be at least partially predicted by the suite of model guidance we have at our disposal. Last time I think that.

The issues for today were not particularly difficult, just complex. The ensemble that we work with was doing its job, but the relatively weak forcing for ascent in an unstable environment was leading to convection initiation early and often. The resulting convection produced outflow boundaries that triggered more convection. This area of convection was across NM, CO, KS, and NE. It became difficult to rely on these forecasts because of all of this convection in order to make subsequent forecasts of what might occur this evening in NE/SD/IA along the presumed location of a warm front.

We ended up trying to sum up not only the persistent signals from the ensemble, but also every single deterministic model we could get our hands on. We even used all the 12 UTC NAM, 15 UTC SREF, RUC, HRRR, NASA WRF, NSSL WRF, NCAR WRF, etc. We could find significant differences with observations from all of these forecast models (not exactly a rare occurrence) which justified putting little weight on the details and attempting to figure out, via pattern recognition, what could happen. We were not very confident in the end, knowing that no matter what we forecast or when, we were destined to bust.

Ensemble wise, they did their job in providing spread, but it was still somehow not enough. Perhaps it was not the right kind or the right amount of spread. We will find out tomorrow how well (or poorly) we did on this quite challenging forecast. In the end though, we had so much data to digest and process, that the information we were trying to extract became muddied. Without clear signals from the ensemble, how does a forecaster extract the information and process that into a scenario? Furthermore, how can the forecaster apply that scenario to the current observations to assess if that scenario is plausible?

I will leave you with the current radar and ask quite simply: What will the radar look like in 3 hours?

UPDATE: Here is what the radar looked like 3 hours later:

Nothing like our forecast for new storms. But that is the challenge when you are making forecasts like these.

This week in CI

2011-05-28T00:07:00.000-05:00

The week was another potpourri of convection initiation challenges ranging from evening convection in WY/SD/ND/NE to afternoon in PA/NY back over to OK/TX/KS for a few days. We encountered many similar events as we had the previous week struggling with timing of the onset of convection. But we consistently can place good categorical outlooks over the region, and have consistently anticipated the correct location of first storms. I think the current perception is that we identify the mechanisms and thus the episodes of convection, but timing the features remains a big challenge. The models tend to not be consistent (at least in the aggregate) for at least two reasons: There is no weather event that is identical to any other, and the process by which CI occurs can vary considerably.

The processes that can lead to CI were discussed on Friday and include:
1. a sufficient lifting mechanism (e.g. a boundary),
2. sufficient instability in the column (e.g. CAPE),
3. instability that can be quickly realized (e.g. low level CAPE or weak CIN or low LCL or small LFC relative to the LCL),
4. a deep moist layer (e.g. reduced dry air entrainment),
5. a weakening cap (e.g. cooling aloft).

That is quite a few ingredients to consider quickly. Any errors in the models then can be amplified to either promote or hinder CI. In the last 2 weeks, we had at least similar simulations along the dryline in OK/TX where the models produced storms where none were observed. Only a few storms were produced by the model that were longer lasting, but the model also produced what we have called CI failure: where storms initiate but do not last very long. Using this information we can quickly assess that it was difficult for the model to produce storms in the aggregate. How we use this information remains a challenge, because storms were produced. It is quite difficult to verify the processes we are seeing in the model and thus either develop confidence in them or determine that the model is just prolific in developing some of these features.

What is becoming quite clear, is that we need far more output fields to adequately scrutinize the models. However, given the self imposed time constraints, we need a data visualization system that can handle lots of variables, perform calculations on the fly, and deal with many ensemble members. We have been introduced to the ALPS system from GSD and it seems to be up to the challenge for the rapid visualization and the unique display capabilities for which it was designed (e.g. large ensembles).

We also saw more of what the DTC is offering in terms of traditional verification, object based verification, and neighborhood object based verification. There is just so much to look at it, that it is overwhelming day to day. I hope to look through this in the post experiment analysis in great detail. There is alot of information buried in that data that is very useful (e.g. day to day) and will be useful (e.g. aggregate statistics). This is truly a good component of the experiment, but there is much work to be done to make it immediately relevant to forecasting, even though the traditional impact is post experiment. Helping every component fill an immediate niche is always a challenge. And that is what experiments are for: identifying challenges and finding creative ways to help forecasting efforts.

Tornado Outbreak

2011-05-26T20:41:00.000-05:00

I am posting late this week. It has been a wild ride in the HWT. The convection initiation desk has been active and Tuesday was no exception. The threat for a tornado outbreak was clear. The questions we faced for forecasting the initiation of storms were:
1. What time would the first storms form?
2. Where would they be?
3. How many episodes would there be?

This last question requires a little explanation. We always struggle with the criteria that denotes convection initiation. Likewise we struggle with how to define the multiple areas and multiple times at which deep moist convection initiates. This type of problem is "eliminated" when you issue a product for a long enough time period. Take the convective outlook for example. Since the risk is defined for the entire convective day you can account for the uncertainty in time by drawing a larger risk area and subsequently refining it. But as you narrow down your time window (from 1 day to 3 hours or even 1 hour) the problems can become significant.

In our case, the issue for the day was compounded because the dryline placement in the models was significantly east of the observed position by the time we started making our forecast. We attempted to account for this fact and as such had to adopt to a feature relative perspective of CI along the dryline. However, the mental picture you are assembling of the CI process (location, timing, number of episodes, number of storms) is tied not just to the boundaries you are considering, but the presumed environment in which they will form.

The feature relative environment then would necessarily be in error because we simply do not have enough observations to account for the model error. We did realize that shallow moisture, which was shown on morning soundings, was not going to be the environment in which our storms formed. Surface dew points were higher and staying near 68 in the warm sector. We later confirmed this with soundings at LMN which showed the moist layer increase in depth with time.

So we knew we had two areas of initial storm formation, one in the panhandle of OK and into KS along the cold front to the west and triple point to the east. The other area was along the dryline in OK and TX. We had to decide how far south storms would initiate. As we figuring all of this out, we had to look at the current satellite imagery since that was the only tool which was accounting for the correct dryline placement and estimate how far east it might travel, or mix out to in order to make the forecast.

Sure enough, the warm sector had multiple cloud streets ahead of the dryline. Our 4km model suite is not really capable of resolving cloud streets but we still needed to make our forecast roughly 1-2 hours before CI. So in a sense we were not making a forecast as much as we were making a longer more uncertain nowcast (probably not abnormal given the inherent unpredictability of warm season convection). Most people put the first storm in KS and would end up being quite accurate in placement. Some of us went ahead of the dryline in west central OK and were also correct.

There was one more episode in southern OK and then another in TX later on. This case will require some careful analysis to verify the forecast, other than subjective assessments. Today we got to see some of the potential objective methods via DTC, showing MODE plots of this case. The object identification of reflectivity via neighborhood and also merging and matching were quite interesting and should foster vigorous discussion.

Last but not least, the number of models we interrogated continued to increase, yet we were feeling confident in understanding this wide variety of models using all of the visualization tools including the more rapid web-based plots, and the use of the sub-hourly convectively active fields. We are getting quite good at distilling information from this very large dataset. There are so many opportunities for quantifying model skill that we will be busy for a long time.

It was interesting to be under the threat of tornadoes and to be in the forecast path of them. It was quite a day, especially since the remnant of the hook echo moved over Norman showering debris over the area picked up from the Goldsby Tornado. The NWC was roughly 3-5 miles away from the dissipation point of that Tornado.

Quick Post

2011-05-22T22:08:00.000-05:00

I have blogged here about scales of CI but this weekend was a great example.
Saturday:

These storms formed in close proximity to the dryline where the southern most supercell went up pretty quickly and the other to the North and West went up much slower, remained small and then only the closest storm to the supercell formed into one. But the contrast is obvious. Even after breaking the cap, the storms remained small for an hour or so, and a few remained small for 2.

Today, we saw turkey towers along the dryline for quite a while (2 hours-ish) in OK and then everything went up. But it is interesting to see the different scales, even at the "cloud scale" where things tend be uneven and random, skinny and wide, slow and fast. It makes you wonder what the atmospheric structure is, especially when our tools tell us the atmosphere is uncapped, but the storms just don't explode.

Looks like a pretty active southern Plains week is just beginning, as evidenced by the 43 tornado reports today and the 20 yesterday.

Relative skill

2011-05-21T12:40:00.000-05:00

During the Thursday CI forecast we decided to forecast for a late show down in western Texas where the models were indicating storms would develop. The models had a complex evolution of the dryline and moisture return from SW OK all the way down past Midland, TX. There was a dryline that would be slowly moving southeast and what we thought could be either a moisture return surge or bore of some kind moving northwest. CI was being indicated by most models with large spatial spread (from Childress down to Midland) and some timing spread (from 03 to 07 UTC) depending on the model. More on this later.

Mind you, none of these models had correctly predicted the evolution of convection earlier in the day along the TX panhandle-western OK border. In fact the dryline in the model was well into OK and snaked back southwest and remained there until after 23 UTC. The dryline actually made it to the border area, but then retreated after the storms formed southwest of SW OK in TX. This was probably because of the outflow from these storms propagating westward. This signal was not at all apparent in the models, maybe because of the dryline position. The storms that did form had some very distinct behavior with storms that formed to the north side of the 1st initiation episode moving North, not east like in the models. The southern storms were big HP supercells, slowly moving east northeast, and continually developing in SW OK and points further SW into TX (though only really the first few storms were big; the others were small in close proximity to the big storms - a scale enigma). We had highlighted the areas to the south in our morning forecast, along with an area in KS to the North but left a sight risk of CI in between. So while our distinct moderate risk areas would sort of verify in principle (being two counties further to the east than observed) we still did not have the overall scenario correct.

That scenario being, storms developing in our region and moving away, with the possibility for secondary development along the dryline a bit later. Furthermore we expected the storms to the north to develop in our moderate risk area and move east. When in fact the OK storms moved into our KS region just prior to our northern KS moderate area verifying well with an unanticipated arcing line of convection. This was a sequence of events that we simply could not have anticipated. We have discussed many times having the need to "draw what the radar will look like in 3 hours". This was one of those days where we could not have had any skill whatsoever in accomplishing that task.

Drawing the radar in 1 or 2 or 3 hours is exactly what we have avoided doing given our 3 hour forecast product. We and the models, simply do not have that kind of skill at the scales where it will be required to have added value. This is not so much a model failure or even human failure. It is an operational reality that we simply don't have enough time to efficiently and quickly mine the model data to extract enough information to make a forecast product. More on this later.

Back to the overnight convection. So once these SW OK supercells had been established we sought other model guidance, notably the HRRR from 16 or 17 UTC. By then it had picked upon the current signal and was showing a similar enough to observations evolution. This forecast would end up being the closest solution, but to be honest was still not that different way down in TX to the ensemble which was a 24-30 hour forecast. They all said the same thing: dryline boundary and moisture surge would collide, CI would ensue within 2 hours into a big line of convective storms that would last all night and make Fridays forecast very difficult.

Sure enough these boundary collisions did happen. From the surface stations point of view, winds in the dry air had been blowing SW all day with temps in the upper 80's low 90s. After 02 UTC, the winds to the NW had backed and were now blowing W with some blowing W-NW. While at the dryline, winds were still SW but weakening. Ahead of the dryline, they were SE and weak. By 0400 UTC, the moisture surge intensified from weak SE winds to strong SE winds, with the dew point at CDS increasing from 34 to 63 in that hour. On radar the boundaries could be seen down in Midland, as very distinct with a clear separation:

You can see CI already ongoing to the north where the boundaries have already collided and the zipper effect was in progress further southwest but it took nearly 2 more hours.

Also note the "gravity waves" that formed in the upper level frontal zone within a region of 100 knots of vertical shear back in NM. Quite a spectacular event. Let me also note that the 00 UTC ensemble and other models DID NOT pick up this event, until 3 hours later than shown by the last radar image. Spin up may have played a significant role in this part of the forecast. As you can see, the issues we face are impressive on a number of levels, spatial, and temporal scales. We verified our forecast of this event with the help of the ensemble and the HRRR and the NSSL WRF. To reiterate the point of the previous post: It is difficult to know when to trust the models. But in this case we put our faith in the models and it worked out, whereas in the previous forecast, we put our faith in the models and we had some relative skill, but not enough to add value.

Not so fast

2011-05-18T23:26:00.000-05:00

What can I say. We verified our Tuesday forecasts and felt pretty good capturing the elevated convection over KS and OK at about the right time (1st lightning strike was 45 minutes before our period start time) for an elevated convection event and we rocked it out in CO during the day.

Which brings us to Wednesday. A slight risk in OK with a triple point from a warm front, stationary front and dryline in place. The dryline was forecast to be nice and strong, with horizontal convective rolls (HCRs) present on the north side interacting with the dryline circulation, and more HCRs in the dry air behind the dry line further south. The end result was a series of CI failures (indicated exclusively by accumulated precipitation) along the dryline, but eventually classic right moving supercells (1 or 2 dominant and long lived) were present in the ensemble. The HCRs were detected via some of the unique output variables we are experimenting with, particularly vertical velocity at 1.1 km AGL.

The complicating factor was that not all members had supercells, perhaps 1 in 3. We also had a situation where CI was very sensitive to the distribution of mixing ratio in the warm sector. It appeared we had two different behaviors along the dryline southward, but a definite moisture pool was dominant to the north. This pool was in the area where the dryline bent back westward and where the HCRs directly interacted with the dryline/warm front boundary. There was very little vertical velocity in the warm sector. Not sure why this was the case, assuming the PBL heights were not much lower than 1.1 km.

But lets be serious. There was not a whole lot of over-forecasting by the models. Storms did attempt to form. They just didn't last very long or get very strong. Nor could we really call them storms (having met no single definition of CI other than some coherent weak reflectivity). In this case it appears the strongest forcing (the dryline and HCRs) was separated from where the realizable instability was present. We analyzed where, when, and how the instability could be realized (in great detail) in the model. We could not verify these models with observations because we don't have very many sounding sites nor do we have frequent launches. What we could not do is pinpoint where this forecast was going wrong.

The KOUN sounding is presented below:

Note that this sounding has nearly 3000 J/kg CAPE with 43 knot deep layer shear, and strong (31 knot) 0-1km shear. An ideal sounding for supercells, and possibly tornadoes. But Norman is well away from where the dryline was setup in western OK. There are no sounding sites in SW OK or NW OK. If we look at LMN which is north of OUN and happens to be north of the warm front:

We see very little instability because of cooler surface temperature and dew point and an elevated capped layer. Modifying this sounding for surface conditions at OUN would indicate strong instability and no cap using the virtual parcel. It is unlikely to be this easy as there is probably some mesoscale variability that has not been sampled in this area.

What is clear is that the forecasters were very much willing to buy into a reasonable solution. What we lacked was a solid reason to not believe the models. I am assuming we should first believe the models and that is perhaps not the best starting point. So lets reverse that thought: what reasons did we use to believe the models? I won't speak for the group, but we should address this question when we review this event tomorrow.

Another point: Assume the model is not a poor representation of observations. What if it was very close? How could we recognize the potentially small errors which could lead to the development of storms or the lack there of? These are really fundamental questions that still need to be addressed.

On a day like today it would have been highly valuable to have soundings in key storm-scale locations near the dryline in the warm sector to the east, and to the immediate north between the dryline and warm front. At the very least we can ascertain if the model was depicting the stratification correctly and also the moisture content, instability and inhibition. It would have been great to have a fine scale radar to measure the winds around the dryline to look for HCRs.

Opportunities

2011-05-17T20:34:00.000-05:00

Every Monday we get a new group of participants and this week we spent time discussing all the issues from their perspective. From an aviation, airport, and airplane perspective to towering cumulus from the mountains. We then discussed the forecast implications from verification, to model data mining, to practical use of forecast data and again learned the lesson that forecasters already know: you only have so much time before your forecast is due and it better contain the answer!

Our forecast has evolved into a 3 hour categorical product of convection initiation where we determine the domain and the time window. We then go a step further and forecast individually a location where we think the first storm will be in our domain during our time period. We then forecast the time we think it will occur and our uncertainty. Then we assign our confidence that CI will occur within 25 miles of our point. It might sound easy, but it takes some serious practice to spin up 10 people on observations, current and experimental guidance, AND lots of discussion about the scenario or scenarios at play. We have a pretty easy going group, but we do engage in negotiations about where to draw our consensus categorical risk and over what time period we are choosing. It is a great experience to hear everyone's interpretation of the uncertainty on any number of factors at play.

Monday was a great practice day with a hurried morning forecast, and terrain induced CI forecast in the afternoon. Tuesday we were off to a good start with all products nominal, 10+ forecasters, and action back out on the Plains with plenty of uncertainty. The highlights from today included the introduction of ensemble soundings via the BUFKIT display (Thanks Patrick Marsh!). This garnered a lot of attention and will be yet another new, exciting, and valuable visualization tool. The aviation forecasters shared tremendous insights about their experiences and even showed a movie of what they face as airplane traffic gets shuffled around thunderstorms. It was a glimpse of exactly the sorts of problems we hope to address with these experimental models.

These problems are all associated with the CI problem, on every scale (cloud, storm, squall line scales). The movie highlighted the issue of where and when new convection would fill in the gaps, or simple occupy more available air space, or block and airport arrival, or when convection would begin to fizzle. Addressing these issues is part of the challenge and developing guidance relies almost exclusively on how we define convection initiation in the models and observations. We have some great guidance and it is clear that as we address more of the challenges of generic CI we will require even more guidance to account for the sheer number of possibilities of where (dx, dy and dz), when, how, and if CI occurs.

As an example, we issued our first night time elevated convection forecast. As it turns out, we could be verifying this by observing rain in OK tonight. Our experimental guidance was inadequate as we have very little data aloft except soundings from the fine resolution models. So we looked at more regular models while using what was available from the fine resolution models, like reflectivity and CI points. This highlights a unique operational challenge that we all face: Data overload and time intensive information extraction. The forecast verification for tonight should be quite revealing and should provide more insight than I am prepared to discuss this evening.

When seeing is believing

2011-05-15T23:58:00.000-05:00

The HWT is examining some fairly sophisticated model simulations over a big domain. One question that frequently arises is: Can we trust the model over here if it is wrong over there?

What does "wrong" mean in these somewhat new models? Wrong in the sense that convection is absent or wrong in the sense that convection is too widespread? Perhaps, a particular feature is moving too slow or too fast. Can you really throw out the whole simulation if a part is "wrong"? Or do you just need time to figure out what is good/bad and extract what you can? Afterall the model is capable of detail that is not available anywhere else. That includes observations.

So Thursday and Friday we discussed how wrong the models have been. The features missed, the features misrepresented, the features absent. Yet each day we were able to extract important information. We were careful about what we should believe. On Friday, though, it was a different story. The NSSL WRF simulated satellite imagery was spot on. That is 14 hours into the simulation where the upper low, its attendant surface cold front were almost identical.

Our domain was northern AR, southern MO, western TN and MS. The models were not in agreement mind you. The different boundary layer schemes clustered into two groups: all the schemes were going for the northern AR initiation, and a second group, the TKE based schemes were also going for the southern part of the cold front. Another signal I was paying attention was post-frontal convergence that was showing up. I made note of it but I never went back to check all the simulations but I wanted to keep that threat in the forecast. Turns out, the TKE schemes hit on all of these features. The northern storms initiated similar to model consensus, the southern storms initiated as well, and so did the secondary episode behind the front (at least from the radar perspective).

The second domain of the day was Savannah GA, in the afternoon. This was an event involving convection possibly moving in from the west, the sea breeze front penetrating far inland along the east, a sea breeze fron the west FL and gulf coast sea breeze penetrating even farther inland, and a highly organized boundary layer sandwiched in between. The models had little in the way of 30 dBz 1km reflectivity at hourly intervals. The new CI algorithms showed that CI was occurring along all of the aforementioned features:
1. Along the sea breezes,
2. in the boundary layer along horizontal convective rolls,
3. along the intersections of 1 and 2,
4. and finally along the outflow entering into our domain.

We went for it and there was much rejoicing. We watched all afternoon as those storms developed along radar fine lines, and along the sea breeze. This was a victory for the models. These storms ended up reaching severe levels as a few reports came in.

As far as adding value on days like this, I am less certain. Our value was in extracting information. There is much to add value to. At this stage, we are still learning. It is impossible to draw what the radar will look like in 3 hours (unless there is nothing there). But I think as we assemble the capabilities of these models, we will be able to visualize what the radar might look like. As our group discussed, convection in the atmosphere appears random. But only because we have never seen the underlying organization.

It is elusive because our observing systems do not see uniformly. We see vertical profiles, time series at a location, and snap shots of clouds. We see wind velocity coming towards or away from radars. We see bugs caught in convergence lines (radar fine lines). So these models provide a new means to see. Maybe we see things we know are there. Maybe we are seeing new things that we don't even know to look for. Since we can not explain them we are not looking for them. We expect to see more cool stuff this week.

Thanks to all the forecasters this week who both endured us trying to figure out our practical, time limited forecast product, and who taught us how to interrogate their unique tools and visualizations. We begin anew tomorrow with a whole new crop of people, a little better organized, with more new stuff on display, and more complex forecasts to issue.

I know it when I see it and other discussions

2011-05-11T22:55:00.000-05:00

We had many discussions over the last two days. One was regarding CI definitions. Of the variety of opinions we heard, a storm was defined by:
1. Whether lightning occurred,
2. a coherent, continuous thunderstorm that eventually reached a significant low level reflectivity threshold (40-45 dBz) within 30 minutes,
3. any combination of 1 or 2, which also produced severe weather (e.g. it was just a storm, but a severe storm)

These variations on the theme are exactly what we were considering for the experiment, the experimental algorithms from the model, and the forecast verification we had played with prior to the experiment.

Another conversation involved what forecasters would use from the experimental suite of variables. The variables would need to be robust, easy to interpret (e.g. quick to interpret and understand), and clear. This is a tough sell from the research side of things, but it is totally understandable from a forecaster perspective. Forecasters have limited time in an environment of data overload in which to extract (or mine) information from the various models. They have very specific goals too, from nowcasting (e.g. 1-2 hours; especially on days like today where models miss a significant component of what is currently happening), to forecasting (6-24 hours), to long lead forecasting 1-8 days.

We also spent some time discussing the Tuesday short wave trough in terms of satellite data and radiosonde data. I argued that many people believe that satellite data and its assimilation is much more important now (Data volume, coverage, and quality control) than radiosondes. It was mentioned that radiosondes are very important on the mesoscale especially in the 0-24 hour possibly 48 hour forecasts. Still more opinions were expressed that some forecasters have questioned the need for twice a day soundings. Opinions in the HWT ranged from soundings are important, to soundings should be launched more often, to sounding should be launched more often at different times. It is plausible that some of our NWP difficulty may be due to launching soundings at transition times of the boundary layer.

I am of the opinion that if model suites are launched 4 times per day that soundings should be launched at least 4 times per day, especially now where cycling data assimilation is common practice. This would return our field to the 1950's era where 4 times per day soundings were launched at 3,9, 15, and 21 UTC.

Lastly, we discussed the issue of what happens when a portion of the forecast domain is totally out to lunch? Like today where the NM convection was not represented. I think I will talk about that tomorrow once we verify our forecast in the OK area from today. Stay Tuned.

It's complicated

2011-05-11T21:05:00.000-05:00

As expected, it was quite a challenge to pick domains for days 2 and 3. Day 2 was characterized by 3 potential areas of CI: Ohio to South Carolina, Minnesota and Iowa, and Texas. We were trying to determine how to deal with pre-existing convection: whether it was in our domain already or would be in our domain during our assumed CI time. As a result, we determined that the Ohio to South Carolina domain was not going to be as clean-slate as Texas or Minnesota. So we voted out SC.

We were left with Texas (presumed dryline CI) and Minnesota (presumed warm front/occlusion zone). Texas was voted in first but we ended up making the MN forecast in the afternoon. Data for this day did not flow freely, so we used whatever was available (NSSL-WRF, operational models, etc).

The complication for TX was an un-initialized short wave trough emanating from the subtropical jet across Mexico and moving northward. This feature was contributing to a north to south band of precipitation and eventually triggered a storm in central and eastern OK, well to the east of our domain. The NSSL WRF did not produce the short wave trough and thus evolved eastern TX much differently than what actually occurred despite having the subtropical jet in that area. So we were gutsy in picking this domain despite this short wave passing through our area. We were still thinking that the dryline could fire later on but once we completed our spatial confidence forecast (a bunch of 30 percents and one 10 percent) and our timing confidence (~+/- 90 minutes) it was apparent we were not very confident.

This was an acceptable challenge as we slowly began to assemble our spatial forecast, settling on a 3 hour period in which we restrict ourselves to worrying only about new, fresh convection by spatially identifying regions within our domain where convection is already present. This way we don't have to worry about secondary convection directly related to pre-existing convection. We also decided that every forecaster would enter a spot on the map where they thought the first storms would develop (within 25 miles of their point). This makes the forecast fun and competitive and gets everyone thinking not just about a general forecast but about the scenario (or scenarios if there are multiple in the domain).

The next stop on this days adventure was MN/IA/Dakotas. This was challenging for multiple reasons:
1. The short wave trough moving north into OK/KS and its associated short wave ridge moving north northeast
2. the dryline and cold front to the west of MN/IA,
3. the cold upper low in the Dakotas moving east north east.

The focus was clear and the domain was to be RWF. This time we used a bigger domain in acknowledgement of the complex scenario that could unfold. You had the model initiating convection along the warm front, along the cold front in NE on a secondary moisture surge associated with the short wave trough, and a persistent signal of CI over Lake Superior (which we ignored).

We ended up drawing a rather large slight risk extending down into IA and NE from the main lobe in MN with a moderate area extending from south central MN into northern IA. After viewing multiple new products including simulated satellite imagery (water vapor and band differencing from the NSSL WRF and the Nearcast moisture and equivalent potential temperature difference, it was decided that CI was probably with everyone going above 50 percent confidence.

In Minnesota we did quite well, both by showing a gap near Omaha where the moist surge was expected but did not materialize until after our 0-3 UTC time period. Once the moisture arrived ... CI. In MN CI began just prior to 23 UTC encompassing some of our moderate risk even down into IA, yet these "Storms" in IA were part of the CI episode but would not be objectively classified as storms from a reflectivity and lifetime perspective, but they did produce lightning.

The verification for Texas was quite bad. Convection formed to the east early, and to the west much later than anticipated associated with a southern moisture surge into NM from the upper level low migrating into the area nearly 11 hours after our forecast period start.

As it turns out, we awoke this morning to a moderate risk area in OK, but the NM convection was totally missed by the majority of model guidance! The dryline was in Texas still but now this convection was moving toward our CDS centerpoint and we hoped that the convection would move east. A review of the ensemble indicated some members had some weak signals of this convection, but it became obvious that it was not the same. We did key in on the fact that despite the missed convection in the TX panhandle the models were persistent in secondary initiation despite the now-developing convection in southern TX. We outlooked the area around western OK and parts of TX.

In the afternoon, we looked in more detail at the simulated satellite imagery, nearcast, and the CIRA CI algorithm for an area in and around Indiana. This was by far the most complicated and intellectually stimulating area. We analyzed the ensemble control member for some new variables that we output near the boundary layer top (1.2 km AGL roughly): WDT: the number of time steps in the last hour where w exceeded 0.25 m/s and convergence . We could see some obvious boundaries as observed, with a unique perspective on warm sector open celled convection.

In addition we used the 3 hour probabilities of CI that have been developed specifically for CI since these match our chosen 3 hour time periods. We have noticed significant areal coverage from the ensemble probabilities which heavily weight the pre-existing convection CI points. Thus it has been difficult to assign the actual new CI probabilities since we cant distinguish the probability fields if two close proximity CI events are in the area around where we wish to forecast. That being said, we have found them useful in these messy situations. We await a clean day to see how much a difference that makes.

Day 1 in the 2011 HWT EFP

2011-05-09T21:44:00.000-05:00

What a great start to the HWT. There were troubles, and troubleshooters. We had plenty of forecasters and plenty of forecast problems. All in all it was quite a challenge.

The convection initiation (CI) team had some great discussion on the CI definition including all the ways in which CI gets complicated. For example, visually we can identify individual clouds, or cloud areas on satellite. When using radar, we might select areas of high reflectivity that last for say 30 minutes. In the NWP models, we rely on quantitative values at a single grid point at two instances in time.

We also have the issue of whether CI is part of a larger episode (close in space and/or time by other storms) or developing as a direct result of previous convection (ahead of a squall line). In these relative cases, visually identifying new storms might be easily accomplished, but in the model atmosphere (in a grid point centric algorithm) new CI points may be all over the place, say as gravity waves or outflow achieve just enough separation to be classified as new (thus CI) even though it might simple be redevelopment. From a probability standpoint, spatial probabilities of CI may thus be larger around existing convection. Does this enhanced probability, ahead of the line, signal actual new storm development?

Trying to establish an apples to apples comparison between model and human forecasts of such discrete events is a major challenge. We are testing 3 model definitions of CI to see their viability from the perspective of forecasters, and we will also evaluate object based approaches to CI.

Of course we cannot talk about where CI might be without talking about when! When will the first storm form? This gets back to your definition of CI. Should the storm produce lightning to be classified a storm? How about reaching a threshold reflectivity? How about requiring it that it last a certain amount of time? The standard definition of storms relies on its mode (ordinary, multicell, supercell); all having a unique evolution with the placement of the updraft and precipitation fall out. But what about storm intensity (however you define it)?

I should also acknowledge that defining all of this can be quite subjective and is relevant to individual users of a CI forecast. So we are definition dependent, but most people know it when they see it. Lets consider two viewpoints: The severe storm forecaster and an aviation forecaster. The severe storm forecaster wants to know about where and when a storm may form so they can decide the potential threat thus leading to a product (mesoscale discussion for specific hail, wind, tornado threats) provided that storm or CI episode is long lived. The aviation forecaster might be concerned with the sudden appearance of cumulonimbus which could pose an immediate threat to aircraft. But they are also concerned with the resulting coverage of new storms (diverting traffic, shutting down airports, planning new traffic routes or patterns) and the motion, expansion, and decay of existing storms.

And lastly it will be important for us to establish what skill the models and forecasters have with respect to CI. This is not a new area of study, but it is one where lots of complexity, vagaries of definitions, and also a lack of understanding contribute to making this one of the greatest forecast challenges.

As we refine what our forecast will consist of, we will report back on how our forecast product evolved. The more we forecast, the more we learn.

Comparing Experimental QPF Outlooks with Hi-res model/radar data

2010-05-21T14:44:00.008-05:00

As part of the 2010 HWT Spring Experiment (EFP), two new forecast components were added; Aviation and QPF. After analyzing various high-resolution (hi-res) models, outlooks were created for QPF expected to exceed 0.50 and 1 inch for two time periods (18-00 UTC and 00-06 UTC) for the Day 1 period. In the image below, you will see an example of one of these hi-res (WRF-HRRR) models 6-hour total precipitation overlayed with a "SLIGHT RISK" threat area for QPF exceeding 0.50 inch in the same 6-hour period. Each day during the Experiment, the morning forecast (threat area) was completed by 1530 UTC.

A screen capture of the latest composite reflectivity data is attached to show how the forecast is verifying to this point.

A "SLIGHT RISK" threat area is defined by 25 percent of the threat area expected to reach or exceed a specific amount (i.e. 0.50 inch).

David Nadler
Warning Coordination Meteorologist
NWS Huntsville AL

Another review from the week of May 18-24

2009-07-13T08:52:00.002-05:00

First, as others have mentioned, special thanks to Steve and Jack for hosting the 2009 Spring Experiment. It's of tremendous benefit to be able to focus on meteorology and consideration of many factors without the distractions of email/phones. Here is a reproduction of portions of a summary I prepared for my forecast staff:

There are considerable issues regarding mesoscale (and potentially stormscale) models being able to reproduce observed convective initiation, morphology, and evolution. Whether it be near-term assistance in high impact decision support services, or long term initiatives such as Warn on Forecast, the future success of improving services beyond detection systems (radar/satellite) depends on the ability of models to portray reality. Current challenges for models include: 1. ability to resolve features, even at high (1 km) resolution; 2. computational limitations; 3. initial conditions and data assimilation; 4. remaining poor understanding of physics and representation of physics in models; and 5. verification of non-linear, object-like features. Identifying where focus should be placed in improving the models is one main goal I found with the EFP. For example, given some set amount of computational resources, should one very high resolution model be run, or an ensemble of lower resolution models? What observations/assimilation/initial conditions seem most critical? Is a spatially correct forecast of a squall line, but with a timing error of 3 hours a “good” forecast? Is it a better forecast than a poor spatial representation of the squall line (e.g., a blob) with spot-on timing?

The main challenge we faced was literally a once in 30-year event of poor convective conditions across the CONUS during the week – the first time SPC did not issue a watch for that corresponding week since 1979, the previous low being 10 watches for the same week in 1984! We did have pulse convection and a few isolated supercells in eastern Montana, eastern Wyoming, and western Nebraska, but it was a challenge for the models to even develop (or in some cases overdevelop) convection. After reflecting on our outlooks, verification, and evaluation/discussion of the models, some general conclusions can be drawn:

1. Mesoscale models are a long way from consistently depicting convective initiation, morphology, and evolution. Lou Wicker and Morris Weisman estimate it might be 10-15 years before models are at the level where confidence in their projections would be enough to warn for storms before they develop, presuming the inherent chaos of convection or lack of initial conditions/data assimilation even allows predictability. Progress will surely be made in computational power, better initial conditions (e.g., radar/satellite/land surface), and to some extent model physics, but there will remain a significant role for forecasters in the foreseeable future.

2. Defining forecast quality is extremely difficult when considering individual supercells, MCSs, and other convective features. What may be an excellent 12 hour outlook forecast of severe convection for a County Warning Area could be nearly useless for a hub airport TAF forecast. Timing and location are just as critical as meteorologically correct morphology of convection. Ensembles may be able to distinguish the most likely convective mode, but offer only modest assistance in timing and location.

3. The best verifying models through a 36-hour forecast seem to be those with 3-4 km grid spacing, “lukewarm” started with radar data physically balanced through the assimilation process. The initialization/assimilation scheme seems to have more of an influence than differences in the model physics. Ensemble methods (portraying “paintball splats” of a particular radar echo or other variable threshold), seem to offer some additional guidance beyond single deterministic runs, although it’s very hard to assess the viability/quality of the solutions envelope when making an outlook or forecast. The Method for Object-based Diagnostic Evaluation (MODE) is a developing program for meaningful forecast verification, based on the notion of objects (i.e., a supercell, an MCS) rather than a grid point based scheme.

Overall, the EFP experience was personally rewarding – an opportunity to get away from typical WFO operations and into applied research. The HWT facility and SPC/NSSL staff were fantastic and made for a high-level scientific, yet relaxed, environment. I strongly encourage anyone interested in severe convection from a forecast/outlook/model perspective to consider the EFP in future years.

-Jon Zeitler
NWS Austin/San Antonio, TX

A Few Thoughts Regarding the HWT Spring Experiment – Mike Fowle

2009-06-11T11:00:00.001-05:00

As others have mentioned, I want to thank the SPC and NSSL for coordinating the program once again this year. As a first year participant, I found the experience both challenging and rewarding. Although the overall magnitude/coverage of convective weather continued to be generally below normal – there were still plenty of forecast challenges to keep us busy throughout the week.

Now for some observations (mainly subjective) about a few of the issues we encountered:

1. Having completed a verification project on an early version of the MM5 (6KM horizontal grid spacing) back in the late 1990s, it was interesting to see the current evolution of mesoscale modeling. While there have been many changes/improvements (e.g. microphysics schemes, PBL schemes, radar assimilation, etc) it was evident to me that many of the same problems we encountered (sensitivity to initial conditions, sensitivity to model physics, parameterization of features, upscale growth, etc) still haunt this generation of mesoscale models.

2. With the increase in computer power, we are now able to run models with a horizontal grid resolution of 1km over a large domain in an operational setting! However, examining the 1km output did not seem to add much (if any) value over 4km models – especially considering the extra computational expense.

3. All of the high resolution models still appear to struggle when the synoptic scale forcing is weak. In other words, modeling convective evolution dominated by cold pool propagation remains extremely challenging.

4. The output from the high resolution models remains strongly correlated to that of the parent model used to initialize. Furthermore, if you don’t have the synoptic scale conditions reasonably well forecast, you have little hope in modeling the mesoscale with any accuracy.

5. Not surprising, each model cycle tended to produce a wide variety of solutions (especially during weak forcing regimes) – with seemingly little continuity amongst individual deterministic members (even with the same ICs), or from run to run. Sensitivity to ICs and the lack of spatial and temporal observations on the mesoscale remains a daunting issue!

Even with some of these issues, on most days the high resolution models still provided valuable guidance to forecasters – most notably regarding storm initiation, storm mode, and overall storm coverage. Although the location/timing of features may not be exactly correct, seeing the overall “character of the convection” can still be of great utility to forecasters especially considering they are not available in the current suite of operational models (i.e. NAM/GFS).

From a field office perspective – one of the big challenges I see in the future is how to best incorporate high resolution model guidance into the “forecast funnel.” Being that many forecasters already feel we are at (or even past!) the point of data overload, they need proof that these models can be of utility in the forecast process. Moreover, I believe that on an average day, most forecasters can/will devote at most 30-60 min to interrogate this guidance. Is this sufficient time? During the experiment we were devoting a few hours to evaluating the models – and I still felt we were only scratching the surface.

Next, what is the best method to view the data? A single deterministic run? Multiple deterministic runs? Probabalistic guidance from storm scale ensembles? Model post products (i.e. surrogate severe)? Some combination of the above? In addition, what fields give forecasters the most bang for the buck? Simulated reflectivity, divergence, winds, UH, updraft/downdraft strength? Obviously many of these questions have yet to be answered, however what is clear to me is that significant training is going to be required regarding both what to view, and how to view it.

In terms of verification, the object based methodology that DTC is developing is an interesting concept. Although still in its infancy, I like the idea and do see some definite utility. However, as we noted during the evaluation, it still appears as though this methodology may be best suited for a “case study” approach rather than an aggregate (i.e. seasonal) evaluation (at least at this point).

As echoed by others, it was a privilege to be a participant in this year’s program and I would jump at the opportunity to attend in future years. In my humble opinion, I think the mesoscale models have proved long ago that they do have utility in the forecast process – if used in the proper context. There are obvious challenges to embark upon in the years to come, and I look forward to seeing the continued evolution of techniques/technology in future years.

11-15 May 2009 Spring Experiment

2009-06-08T07:46:00.002-05:00

I would like to thank SPC, NSSL, and others for the invitation to participate in the 2009 HWT. As most of you are aware (either through discussions with me or your experiences sitting at an airport for hours waiting for a delayed flight) the socioeconomic impacts of convective weather on the aviation industry are substantial. Attending the HWT is a way to grasp where the edge of the science is and establish a reality check for myself and to share with others as we work towards NextGen. It is an intriguing experience to learn from both operational forecaster feedback and our own forecasts we developed at HWT that the solution to increased accuracy does not correlate to higher model resolution as some might believe. Having an HWT week with an aviation focus is a good way for this research to get some exposure in a capacity that it may not have been designed for but illustrates potential utility and benefit.
I have pointed out things like Simulated Reflectivity from the HWT last year that has gained attention based on potential utility in using the data at a high level to get an understanding of what the National Airspace System scenario might look like for the day. Although it may never verify due its deterministic nature and is somewhat noisy it is good visual aid for an Air Traffic Flow Manager (non meteorologist) to get a quick frame of reference on potential systemic impacts.
Other forecasts like the Probability of >=40dbz intensity within 25 miles of point and the 18 member ensemble for 40dbz intensity could have enormous value for the aviation industry as 40dbz is also the level of intensity that aircraft no longer penetrate thus causing deviations and ultimately delays. Research on convective mode was another area I found myself intrigued with as convective mode from an aviation perspective provides a frame of reference to determine the permeability of the convective constraint. Discreet cells and linear convection can be equally disruptive but would be managed very differently if forecast with a high degree of skill so modal info for aviation is equally important as location and timing.
Thanks for a great week!
John

Reid Hawkins' view of the June 1-5 HWT Spring Experiment

2009-06-07T13:47:00.004-05:00

As I sit here at the Will Rogers Airport in Oklahoma City waiting on a plane that is 2 hours late, I wanted to reflect on my experiences with the 2009 HWT Spring Weather Experiment. This reflection will be in more in the style of stream of consciences so I hope someone out there can follow it.

1^st a well deserved round of applause for Steve W, Jack, and Matt on steering us through the plethora of numerical models and the objective verification techniques of DTC. Our week started off with a rather well behaved and straight forward event over the northern Mississippi Valley into the northern plains. The second day was a highly frustrating forecast over Oklahoma, Kansas, and Northern Texas where overnight convection and gravity wave played havoc to the forecast and lack of convection over Oklahoma. The third day was even more frustrating with a weaker forced case south of an east-west stationary front from northern Virginia and back to Kentucky. The final case was a high plains case from Wyoming and Nebraska southward to the Texas Panhandle.

For our week of evaluating the models, my first impression was the number of models that provided a whole host of solutions. Through experience from the staff, they steered us to look at the reflectivity fields, outflows and updrafts instead of digging ourselves into a myriad of model fields that no one could have possibly looked at in the short time we had to prepare a forecast outlook. After shifting my paradigm to this style of forecasting which was somewhat uncomfortable, it was a comfortable feeling when we saw similar results from the models. This was not a common event as most of the cases were marginal or weakly forced.

One concern I have is that I did not see a huge bang for the money in the 1 km Caps model runs vs. the 4 km Caps models. There was a huge discussion about the assimilation of the data into the models and my thoughts are that until we sample the atmosphere with higher resolution, frequency and more accurately then I do not see where the higher scale models will provide better results for forecast operations. This is just an opinion and I hope the modelers can prove me wrong.

Another concern, I have is the way the data is displayed to the forecaster. With the wealth of data that is available and our current display techniques, I am afraid this has or will force many forecasters to find a comfort level of what data types to use. This means there may be valuable data sets to view but due to comfort level and time constraints these data sets may never be used.

Overall, the week was extremely enlightening on seeing the techniques that are being developed to help the forecaster. In time as the development envelop is pushed, I expect to see great information delivered to the operational desk. I am somewhat disheartened but not surprised on the lack of help we saw in weakly forced environments.

My impressions from the week May 11-15th

2009-05-22T10:07:00.002-05:00

Finally got a chance to type some words (I hope not to many) after getting back to the UK.

First I would like to say thank you for a very enjoyable week in which I feel that I've come away learning a great deal about the particular difficulties of predicting severe convection in the United States and gained more insight into the challenges that new storm-permitting models are bringing. I was left with huge admiration for the skill of the SPC forecasters; particularly synthesising such a large volume of diverse information quickly (observational and model) to produce impressive forecasts, and to see how an understanding of the important (and sometimes subtle) atmosphereic processes and conceptual models are used in the decision-making process.
I was also struck by the wealth of storm-permitting numerical models that were available and how remarkably good some of the model forecasts were, and can appreciate the amount of effort that is required to get so many systems and products up, running and visible.

One interesting discussion we touched on briefly was how probabilistic forecasts should be interpreted and verified. The issue raised was whether it is sensible to verify a single probabilistic forecast.
So if the probability of a severe event is say 5% or 15% does that mean the forecast is poor if there were no events recorded inside those contours? It could be argued that if the probabilities are as low as that, then it is not a poor forecast if events are missed, because in a reliability sense we would expect to miss more than we get. But the probabilities given were meant to represent the chance of an event occurring within 25 miles of a location, so if the area is much larger than that it implies much larger probabilities of something occurring somewhere within that area. So it may be justifiable to assess the forecast of whether something occurred within the warning area as if it is almost deterministic, which may be why it seemed intuitively correct to do it like that in the forecast assessments. The problem then is that the verification approach does not really assess what the forecast was trying to convey.
An alternative is to predict the probability of something happening within the area enclosed by a contour (rather than within a radius around points inside the contour), which would then be less ambiguous for the forecast assessment. The problem then is that the probabilities will vary with the size of the area as well as the perceived risk (larger area = larger probability), which means that for contoured probabilities, any inner contours that are supposed to represent a higher risk (15% contour inside 5% contour), can't really represent a higher risk at all if they cover a smaller area (which they invariably will!). So at the end of all this I'm still left pondering.

The practically perfect probability forecasts of updraft helicity were impressive for the forecast periods we looked at. Even the forecasts that weren't so good seemed to appear better when the information was presented in that way and they seemed to enclose the main areas of risk very well.

The character of the 4km and 1km forecasts seemed to be similar to what we have seen in the UK (although we don't get the same severity of storms of course). The 1km could produce a somewhat unrealistic speckling of showers before organising on to larger scales. Some of the speckling was precipitation produced from very shallow convection in the boundary layer below the lid and was spurious. (We've also seen in 1km forecasts what appear to be boundary-layer roles producing rainfall when they shouldn't, but the cloud bands do verify against satellite imagery even though the rainfall in wrong).
The 4km models appeared to have a delay in initiation in weakly forced situations (e.g. 14th May) which wasn't apparent in the strongly forced cases (e.g. 13th May). It appeared to me that the 1km forecasts were more likely to generate bow-echoes that propagate ahead (compared to 4km) and on balance this seemed overdone. There was also an occasion from the previous Friday when the 1km forecast generated a MCV correctly when the other models couldn't, so perhaps it indicates that more organised behaviour is more likely at 1km than 4km - and sometimes this is beneficial and sometimes it is not. It implies that there may be general microphysics or turbulence issues across models that need to be addressed.

It was noticeable that the high-resolution models were not being interpreted literally by the participants - in the sense that it was understood that a particular storm would not occur exactly as predicted and it was the characteristics of the storms (linear or supercell etc) and the area of activity that was deemed most relevant. Having an ensemble helped to emphasise that any single realisation would not be exactly correct. This is reassuring as a danger might be that kilometre-scale models are taken too much a face value because the rainfall looks so realistic (i.e. just like radar).

It seemed to me that the spread of the CAPS 4km ensemble wasn't particularly large for the few cases we looked at - the differences were mostly local variability, probably because there wasn't much variation in the larger-scale forcing. The differences between different models seemed greater on the whole. The members that stood out as being most unlike the rest were the ones that developed a faster-propagating bow-echo. This was also a characteristic of the 1km model and was maybe a benefit of having the 1km model as it did give a different perspective, or added to the confusion, however you look at it! One of the main things that came up was the shear volume of information that was available and the difficulty of mentally assimilating all that information in a short space of time. The ensemble products were found to be useful I thought - particularly for guidance in where to draw the probability lines. However, it was thought too time-consuming to be able investigate the dynamical reasons why some members were doing something and another members something else (although forecaster intuition did go a long way) . Getting the balance between a relevant overview and sufficient detail is tricky I guess and won't be solved overnight. Perhaps 20 members were too many.

The MODE verification system gave intuitively sensible results and definitely works better than traditional point-based measures. It would be very interesting to see how MODE statistics compared with the human assessment of some of the models over the whole experiment.

One of the things I came away with was that the larger-scale (mesoscale rather than local or storm scale) dynamical features appear to have played a dominant role whatever other processes may also be at work. The envelope of activity is mostly down to the location of the fronts and upper-level troughs. If they are wrong then the forecast will be poor whatever the resolution. An ensemble should capture that larger-scale uncertainty.

Thanks once again. Hope the rest of the experiment goes well.
Nigel Roberts

Wednesday

2009-05-20T16:10:00.002-05:00

Wednesday morning we talked about severe reports. Is "practically perfect" the way to go? How do we deal with people-sparse regions? Can or should we add an uncertainty to the location and time and veracity of each report?

With our current capability, we can't reliably forecast whether a storm will be a wind or hail producer. I think that is what John Hart suggested.

Yesterday, the 0-4Z forecast ensemble was too eager to produce high updraft helicity severe weather for the first 20-0Z forecast period, but the 0-4 Z period was forecast almost perfectly. Ryan said UH is usually better than surface wind and hail (graupel).

The MODE area ratio is not as useful as area "bias". Ratio is small-over-big and doesn't tell you if the forecast is biased high or low.

I summarized bias, GSS, and MODE results for yesterday. For CSI, Radar assimilation jumped out to an early lead, but joined the control run at near-zero after 3 hours. MODE had some spotty matches, but no clear winner.

Dave Ahijevych

Tuesday

2009-05-20T16:09:00.001-05:00

We talked about Monday evening weather in Montana and how the forecasts went.

NMM had a high false alarm rate but was the only model to correctly predict the severe storm in ID. ARW missed the storm by 200km to the southeast. Another comparison we always make here at the HWT is the 0Z vs the 12Z runs of thhe NMM. For this day, I think the 12Z NMM was much better than the 0Z. Steve thought it was "somewhat" better. The 12Z run had less false alarms in central MT and captured the ID storm area better.

The probability matched mean is an interesting way of summarizing the ensemble of forecasts. It has the spread of the ensemble, but the sharpness of an individual run. I'm not sure but I was told to check out Ebert/McBride for a reference on this.

The Monday forecast was pretty insignificant, so we also talked about the high-end derecho on Friday May 8. There were some differences between the 1 and 4-km CAPS solutions of this event. (I forget what they were).

David Ahijevych

monday

2009-05-20T16:07:00.001-05:00

Here's my summary of yesterday's (Monday's) activity at the HWT.

This is a very quiet week for mid-May.

The DTC has been well-received. I presented Jamie's verification .ppt and gave it away to several interested people. The need for objective verification is great. There so many models and little time to analyze everything after the fact. Mike Coniglio led two discussions of Friday's MODE verification output. The CAPS model without radar data assimilation lagged behind the CAPS model with radar data assimilation. MMI was similar for the two models, but the MODE centroid distance was a distinguishing factor. The model that lagged behind had greater centroid distance. This wouldn't have been possible to quantify with conventional verification metrics.

We also subjectively evalutated the Friday storms over the centeral U.S. The 0Z NMM had a false alarm storm in the morning that disrupted the afternoon forecast. The simulated squall line was much weaker than the observation. This was not as much of a problem with the NSSL model. The 12Z NMM was not a whole lot better with convective mode and individual storm evolution, but its 0-2 h and 6-12 h forecasts had better storm placement than the older 0Z NMM.

As an aside, ARW runs with Thompson microphysics have less intense simulated radar reflectivity than observed.

For Monday afternoon and evening's severe weather forecast, we chose Billings, MT as the center point. It was the only place to have the possibility of severe weather. We broke up into 2 teams and came up with a less than 5% chance. Two actual reports were northwest of our predicted zone in northern Idaho. Radar indicated some small storms in our predicted zone.

Dave Ahijevych

Model Evaluation Tools

2009-05-20T13:13:00.002-05:00

I would like to thank all of the HWT personnel for a fun and interesting week - May 10-15. The experience was well worth it. How quickly I (being in the research community) have lost touch with the daily challenges that an operational forecaster faces. It was good to get back to those roots with a little hand analysis of maps!

I would like to thank you for engaging with the DTC and helping us to evaluate MET/MODE during the Spring Experiment. It is great to have eyes looking at this on a daily basis to give us some good feedback on how the tools are performing. It seemed that while I was there the participants were encouraged by the performance of MODE and its ability to capture objectively what forecasters felt subjectively. This is a great first step towards more meaningful forecast evaluations which we hope, ultimately, feedback to improve overall forecasts by removing systematic biases.

Please feel free to visit the DTC's HWT page at: http://www.dtcenter.org/plots/hwt/

You were all great hosts. Thanks again!

Posted by Jamie W.

Recap of Week 2 from a Forecaster's Perspective

2009-05-18T07:50:00.004-05:00

After spending a week at the HWT, I must say I'm encouraged to see how far the NWP world has come in recent years. For instance, in an effort to keep my mind occupied on my flight to Norman last week, I thought it would be neat to read a paper produced by the SPC on the Super Tornado Outbreak of 1974. If I remember correctly, the old LFM model had a model grid spacing of 190.5 km! After reading this and then coming to the HWT as seeing model output on a scale as low as 1 km is absolutely amazing in my opinion. This is a testament to all the model developers out there who work diligently on a daily basis to produce better models for forecasters in the field. If nothing less, the HWT opportunity made me realize and appreciate the efforts of the model developers more so than I had ever done previously.

Although these models can provide increased guidance for basic severe wx guidance, such as convective mode and intensity, the models only show output (simulated refl, updraft helicity, etc.) on a very small scale. If taken at face value, critical forecasting decisions can be made without having an adequate handle on the overall synoptic and mesoscale pattern. Thus, even with all the high resolution model output, one must still interrogate the atmosphere utilizing a forecast funnel methodology in an effort to develop a convective mode framework to work from. Sadly, if high resolution model output is taken at face value without any 'behind the scenes" work beforehand, I can see many blown/missed forecasts as forecasters would be forecasting "blind." Many factors must be taken into account when developing a convective forecast and unfortunately just looking at the new high res model output will likely lead to more questions than answers. In order to answer these questions, a detailed analysis done prior can allow one to see why a particular model may be producing one thing as opposed to the other. Looking back at some of the old severe wx forecasting handbooks, one thing remains clear, much can be gained on the developing synoptic/mesoscale patterns through pattern recognition. Some of the old bow echo/derecho papers (Johns and Hirt, 1987) and a whole list of others have reiterated the fact that much can be gained by recognizing the overall synoptic pattern. How many times last week were the models producing a bow type signature during the overnight hours? Situations like these commonly need deep vertical shear and unfortunately not much shear was available for organized cold pools when the H50 flow was only 5-10 knots. This is just one instance where having a good conceptual model in the back of your mind can assist in the forecasting process.

As for the models, more often than not, I was pleased by the 4-km AFWA runs. For the activity that developed on the Tue (05/12), the 00/12 UTC AFWA runs had better handle the low-level moisture intrusion up the Palo Duro Canyon just SE of AMA. A supercell resulted which led to several wind/hail reports. A look back at the Practically Perfect Forecast based on updraft helicity the following day had a bullseye centered over the area based on the AFWA output. This is more than likely a testament to different initial conditions as the AFWA utilizes the NASA LIS data. This can pay huge dividends for offices along the TX Caprock where these low-level moisture intrusions have been documented to assist in tornadogenesis across the canyon locations along with a backed wind profile (meso-low formation).

Posted by Chris G.

Spring Experiment Week 3 Participants

2009-05-17T19:20:00.004-05:00

The Spring Experiment organizers would like to welcome the following participants to Week 3 of the 2009 NSSL/SPC Spring Experiment:

Dave Ahijevich (NCAR/DTC, Boulder, CO)
Lance Bosart and Tom Galarneau (University at Albany-SUNY)
Geoff Manikin (NOAA/NWS/NCEP EMC, Camp Springs, MD)
Morris Weisman (NCAR, Boulder CO)
Jon Zeitler (NOAA/NWS San Antonio/Austin, TX)
Jack Hales (NOAA/NWS/NCEP SPC)
John Hart (NOAA/NWS/NCEP SPC)
Jon Racy (NOAA/NWS/NCEP SPC)

Anatomy of a Well Forecast Bow Echo, Part II

2009-05-14T09:24:00.005-05:00

A Cautionary Note about Deterministic Guidance from High-Resolution NWP Models (posted by GregC on behalf of David Bright).

Figure 1. 13-hour WRF-NMM forecast of simulated reflectively (1 KM AGL) valid at 1300 UTC 8 May 2009 (left), and verifying observed base reflectively and severe thunderstorm warning polygons valid at 1300 UTC 8 May 2009 (right).

The 13-hour WRF-NMM forecast of the Missouri Bow Echo (see earlier post with this title) is remarkable in both its accuracy and structure, particularly given the severity of the event. As model forecasts go, it appears to be a perfect piece of numerical guidance. But shifting the grid about 450 miles to the east, the exact same 13-hour WRF-NMM completely missed the MCS (albeit less severe) moving through eastern Tennessee. So while there is little doubt that the model provided an essentially perfect prediction of the intense bow echo over southern Missouri, in a purely deterministic sense, the same model provided little-to-no short-term convective guidance with respect to convective mode and QPF over much of Tennessee.

The information provided by these high-resolution NWP models is revolutionary, and will likely lead to a quantum increase in high-impact services provided by the NWS. But let's be careful not to oversell the capabilities of a single, deterministic model forecast. In order to fully realize the potential of future NWS forecasting and warning services, an ensemble of convective-resolving models will be required to address the uncertainty that accompanies all weather forecasts. The HWT has evaluated convective-resolving models over a large portion of the CONUS for the past several years, and it is encouraging to see the improvements these models have made in high-impact convective guidance and in their ability to predict intense, realistic convective structures such as the bow echo over southern Missouri. But a single high-resolution NWP forecast, regardless of its ability to reproduce intense convective structures, is unlikely to meet the future uncertainty requirements of the entire NWS at all times and locations. That said, the development and evaluation of these convection resolving models is and will continue to be an essential part of future high-impact, life saving, decision support services provided by the NWS, likely realized through a blend of deterministic guidance, well constructed ensemble systems, and related ensemble interrogation tools.

Forecast experiment--May 12 thoughts

2009-05-12T21:27:00.002-05:00

The forecast experiment centered on ABI (Abilene, TX) today to catch development along the dryline and other moisture gradients, including a warm front eroding the morning's stratus northward. After analyzing and discussing the 12Z sounding plan views, we had about an hour on the schedule to create the 20-00 UTC and 00-04 UTC forecasts. It turns out we took a bit more than 90 minutes, since there were three scenarios we considered: (a) Convective initiation over the mountainous terrain of southwest Texas, coverage, and mode as forcing moved east; (b) Same initiation, coverage, and mode problem but over the South Plains and panhandle of Texas and extreme southwest Oklahoma, and; (c) what to think about some model members' convection forecasts over southeastern Texas.

The forecast team I participated in wrestled with forcing mechanisms since the shear profiles were less than robust over the central and southern portions of west Texas and looked to be favorable in the panhandle, South Plains, and extreme southwest Oklahoma. We (there were six of us on the team today) settled on two initiation scenarios--the first in southwest Texas in the 20-21 UTC time frame as a shortwave moved toward El Paso and a second toward the space of Texas between Amarillo and Lubbock with a second jet streak moving into the area. The third area was discounted based on standard theories of organized severe convection.

For ensemble displays, we used

The probability of 40 dBZ reflectivities. This helped focus our attention on the areas of concern and timing scenarios. This is probably the quickest way to assess the result of each model's integration rather than interrogating multiple plan views, soundings, and postage stamp images of significant fields.
Spaghetti outlines of 40 dBZ model-derived reflectivity. This is a noisy but useful depiction.
Max reflectivity from the ensemble members to assess in a very rough manner storm instensity, and
Max updraft helicity to assess the likelihood of severe weather.

From these we went to individual high-res (1-4 km) models and the observations to modify the threat area. It's interesting to note that most participants prefer to use a single high-res or a familiar lower-res (e.g., 12 km WRF-NMM) as assess potential mode and than use the ensembles to place mental "possibilities" around what I'd call the "individual's most probable" forecast.

The deterministic and ensemble runs suggested that southern storms would like be isolated and probably end short after 01 UTC. In the north we believed more organized convection, possibly a couple of clusters, was likely due to the presence of better 0-6 km shear. The was some issue as to how far east the convection would progress by 04 UTC, with the 1 km models suggesting propagation as far east as I-35. The operational SPC forecaster acting as our team's guide tempered our enthusiasm with a little climatology, so the east edge of our forecast area was kept a little west (upstream) of the most aggressive model's 04 UTC position for convection.

We believed there was a significant hail threat over this area given the shear, mid-level lapse rates, and NAM-KF model soundings suggesting analogs of 2+" hail cases. SPC's significant hail paramenter on its mesoanalysis page also centered a threat in this area.

As I write this, we were not far enough west with our initiation and the severe threat is continuing a bit north of the area we anticipated. Tomorrow's review ought to be quite enlightening, assuming we can keep our brains focused on the review and not on Wednesday's expected event!

-- Bruce E, forecasting on the West team today

Anatomy of a Well Forecast Bow Echo

2009-05-08T23:32:00.007-05:00

Above is an example of one of the forecasts from the Spring Experiment models from Friday. This bow echo moved across southwest Missouri early Friday morning and these images are centered on Joplin, MO (JLN). On the left is the 13h forecast from the WRF-NMM 4km model initialized at 00Z 08-May-2009 and valid at 13Z. On the right is the verifying 1km base reflectivity image with the model fields for winds overlaid on the radar. The barbs in each of the images are the model's instantaneous 10m winds in knots (with the grid skipped to lessen the clutter). The isotachs are plotted from the WRF "history variables" for maximum U,V 10m winds (no grid skip). These are the maximum 10m wind speeds in the model over the past hour ending at 13Z.

Instantaneous 10m winds in the model at 13z, near the rotating bow head, are at least 50 knots. The maximum model 10m winds over the past hour range from 60-70 knots near and north of the weak echo channel and around the comma-head of the bow.

This was only one of several exceptional forecasts of this feature from the models being evaluated in this year's Spring Experiment. To see more output on this case and more, check out the Spring Program website here:

http://hwt.nssl.noaa.gov/Spring_2009/index.php

-GregC

WRF/CAPS Reflectivity Loops on V2 Domain

2009-05-08T14:14:00.004-05:00

Plots of WRF/CAPS 1km AGL simulated reflectivity are now available using "V2" as the centerpoint in the URL. The following V2 domain image combinations should work as of today's 00z and 12z model runs:

At 00Z:
NSSL4 / AFWA4 / NMM4 / NCAR3
or
CAPS1 / NCAR3 / NMM4 / NSSL4

At 12Z, you can compare prior runs:
NMM4 / NMM4_12 / AFWA4 / AFWA4_12
or
CAPS1 / NCAR3 / NCAR3_12 / NMM4_12

Mix and match the models to build the loop you need based on the URLs, above. You can also add &date=YYYYmmdd to review another date.

The 00Z NCAR run goes out to forecast hour 48 while the other 00Z runs go to forecast hour 36. The loop URLs, above have a start hour of 12z go out 24h. This can be adjusted by using the &starthr and &frames variables in the URL.

The 12Z NMM goes to forecast hour 36 while the NCAR and AFWA versions of the WRF only go out 24h. These loops also start at 12Z and go out 24h. Note, some imagery will drop out when looping beyond the available forecasts from that model. Imagery may also be uavailable if model data fails to arrive prior to the execution of the cron job for the image generation script. These loop pages should not to be considered operational and will not always be available.

Thanks to Ryan for building such an adaptable web-based interface that allows us to build these thumbnail image loops!

Th 7 May 2009 Noteworthy aspects

2009-05-07T15:48:00.002-05:00

Centerpoint for tda's activity is KGMJ in extreme ne OK. With the last 2 days activity being farther E, this area was quiet yesterday, so no significant preexisting convection was present to complicate matters. The exception to this was some midlevel convection that at 12Z today was over ern KS into MO.

Early on we identified at the surface 2 weak transition zones (these were too diffuse to call them "boundaries". The first of these was along the Red River, separating shallow cool, moist air over OK (partly the consequence of the persistent cloud cover and pcpn the previous 2d) from more humid air (dew pt > 21C) over n TX. The second was a weak e-w zone of confluence marking a weak sfc front from roughly IL, srn IA across much of NE. The parent sfc low with this feature was over srn MB this mrng, if memory serves correctly.

Aloft, strong zonal flow crosses the W coast and the wrn 2/3 of the CONUS. This is providing a shear environment plenty adequate to support supercells. The temperature contrast up and down the W coast of the CONUS at 12Z this mrng was noteworthy: +13C at KNKX (San Diego) to -11C at KUIL (Quilliute). Only weak disturbances are present in this flow, revealed by 6.7 micron water-vapor imagery primarily.
There aprs to be a weak upper lvl PV perturbation in the flow at 12z this mrng centered nrn NV, srn ID and aligned E-W. Associated with this is enhanced N to S temp gradient across NV at 700mb and evidence (from METARs) of a weak surge of slightly cooler surface air heading ewd along I-80 in srn WY. This surge can be argued to have some relationship to the cooler air that this mrng was over the Nrn high plains behind the aformentioned sfc front.

So, we had in place early this mrng warming conditions over the Southwest, very moist, high CAPE air over the southern plains, and no distinct sfc boundaries or marked upper-air features. This made for another challenging forecast day for today.

Our outlooks for the 2000 - 0000Z and 0000 - 0400Z periods this mrng focused on two areas. The first and most important was KS and MO, where indications were from the 00Z Th 7 May initialized hi-res models (CAPS ensembles, NMM4 from NCEP, NSSL ARW3) that activity would initiate in the general vcty of Great Bend KS in late aftn and move into MO durg the evening. The proximate cause of initiation appeared likely to be [based on 10-m wind field from the CAPS control runs and the other so-called "deterministic" (poor term, but I use it for lack of a better one) models] convergence alg a wind shift derived from the frontal confluent zone noted earlier in NE. Updraft-helicity and CAPE-shear parameters argued that there was a chc of sig severe with this stuff.
During the forecast praparation period it became more apparent that the frontal confluent region at the surface noted earlier was going to be a focus of activity, if there was going to be activity in this area. Complicating the picture somewhat was the mid-level stuff noted earlier as being over ern KS into MO.
We noted mesoscale pressure perturbations and fluctuations with this, suggesting that there was some associated wind perturbation, probably just above the surface in early morning. Falling pressure in ern KS and rising pressure in MO appeared to be associated with acceleration northward of the OK low level moisture into KS during the morning, aprnt on visible satellite imagery as a tongue of the OK Sc advancing into central and e KS. The possibility of outflow cooling at low levels over MO reinforced the decision to go with initiation over central to ern KS.

Another factor was the behavior of both the NCAR 3km (initialized from RUC at 00Z and using GFS LBCs), and the 12Z initialized HRRR. Both these models had shown initiation of radar reflectivity over central KS before 15Z, and that the resulting storms would move eastward. Since there was no evidence for such initiation by then in the observations, and it seemed unlikely that such initiation would occur before, say, 1900 to 2000Z, the west team, at least, largely discounted these fcsts as providing useful guidance.

(As an aside, in the Sc over OK and KS mid-morning through mid aftn there were southward-propagating waves in this low-level Sc. I speculate that these were manifestations of low-level bore-like features, initiated by penetrative downdrafts from the earlier convection, though that these continued obvious into mid-afternoon raises some questions about this.)

Both teams, then, pretty much bought off on the scenario of the 00Z NAM-based intialized hi-res models in their forecasts, tho the teams differed in details. Both teams figured that this was worth a 15% chance of svr ... no higher due to a lot of concern that storms would not even form ... and a 10% conditional probability of sig severe.

The second area of concern was along the Red River in AR and SE OK, where the aforementioned 00Z NAM-based initialized models were indicating moderately intense storms forming in aftn, but that storms this area would become weaker after 00Z. The W team put a 5% chc of svr this area, but the E team did not consider this area worth even that much.

These fcsts were completed by 1600-1620Z.

By 1800Z or so clumping of sfc-rooted Cu was occurring along I-80 WSW of KOMA along a WSW-ENE line. These cells were partly flagged by the GOES-R algorithm for detecting growing clouds along boundaries. (I would have to question whether this algorithm added anything to what an experienced forecaster knowledgeable about the overall convective environment could provide.) This convective development contributed to the final forecasts for the 20 - 24Z period and the 24 (00)Z - 04Z period having the areas of forecast severe shifted Nwd from the preliminary forecasts, touching the nrn border of our forecast domain near the IA-MO border. This was confirmed by the 12z-initialized forecasts, as well as teh 16Z HRRR. The 12Z NMM from EMC in particular had initiation very close to where it actually occurred, and subsequent movement of the activity into MO with rotating updrafts. However, we tended to discount the massive outflow generated by this large cell.

We had less confidence in the liklihood of sig severe in preparing our final forecasts. The temp-dew-point spreads in the air farther N were roughly mid 80s to mid 50s, arguing that strong tornadoes would be very unlikely. So, the main threat for sig severe would have to come from wind gusts or hail. Accordingly the sig severe areas were reduced somewhat and the E team might have eliminated theirs altogether (don't remember attm).

A couple of observations.
The NMM 4km from NCEP initialized at 12Z today also initiated a cell near CDS that put out a circularly spreading outflow that covered much of OK after a few hours.
The HRRR seems to be producing overall too much coverage of radar reflectivity, though in terms of forecast reflectivity I believe it is doing a little better than the CAPS forecasts initialized same time. (NOTE that these forecasts are really not directly comparable since different model configurations (phyics options, lateral boundary conditions, grid spacings, domain sizes, etc.)

Comments and corrections welcome ...
John Brown

Another "messy" day.

2009-05-06T15:45:00.000-05:00

Another day with ongoing storms to mess-up the afternoon convective environment. Attention was focused over the southeastern and mid-Atlantic states. While the observations led to plenty of forecast uncertainty, today was notable in that it was the first day of the Spring Experiment where the different 0000 UTC models showed some consistency...but of course just because they were consistent doesn't mean they'll produce good forecasts.

One of the more intriguing forecasts was from the 4-km WRF-NMM which developed several discrete, rotating storms along a warm front located in northern North Carolina. The other models also produced storms in this region, but they were smaller, more numerous, and not nearly as organized. The behavior of the NMM can probably be attributed to its surface wind field, which predicted a more easterly wind component north of the warm front (compared to the other models), thus, enhancing shear along the boundary. As of this posting, there are numerous storms in that area, but I do not believe any are supercellular.

Once again, the 1200 UTC model runs were out to lunch and didn't provide any helpful guidance, probably due to ongoing convection influencing initialization. I don't think either forecasting team used the 1200 UTC model runs as guidance for the final forecast products.

It seems like we're just waiting for a day without morning convection. Perhaps then, the 1200 UTC models will have a better handle on the situation and can provide useful guidance.

Craig S

A first for the 2009 Spring Experiment!

2009-05-06T14:53:00.001-05:00

It looks like we have full 18-06z loops from all available data from 00Z, May 6, 2009!

This is a first for the 2009 SE.

With the center point change to CLT today we are outside the domain of the 00Z CAPS CNA and C0A models.
GEMPAK just plots a blank image with a title for those frames. Everyone else appears to exist for the
18z-06z period. This includes the following 00Z runs:

WRF-AFWA
WRF-NCAR
WRF-NMM
WRF-NSSL
WRF-CAPS1
CAPS-SSEF-ALL (18 members)
CAPS-CNA - no grids in selected domain and not shown in the loop links, below.
CAPS-C0A - no grids in selected domain and not shown in the loop links, below.

Here are the links to the loops for this 00Z run of the models (the verifying base/composite reflectivity will fill-in tonight):

6-panel 1KM REFL w/ verifying BREF w/ time:
http://hwt.nssl.noaa.gov/Spring_2009/loop_6panel.php?date=20090506&image1=nssl&image2=afwa&image3=ncar&image4=nmm&image5=caps1&image6=brefr

6-panel Composite REFL w/ verifying BREF w/ time:
http://hwt.nssl.noaa.gov/Spring_2009/loop_6panel.php?date=20090506&image1=nssl_cref&image2=afwa_cref&image3=ncar_cref&image4=nmm_cref&image5=caps1_cref&image6=crefr

4-panel Synthetic Severe (no HRLY REFMAX from AFWA4 until next week) w/ verifying hrly reports w/ time:
http://hwt.nssl.noaa.gov/Spring_2009/loop_4panel.php?date=20090506&image1=nssl_ssvr&image2=nmm_ssvr&image3=afwa_ssvr&image4=lsr

Where to focus?

2009-05-06T09:01:00.000-05:00

Gulf Coast states look pretty worked over...early day reflectivity pattern shows serpentine line of convection over this area..."junked up". May focus on NC-SC where they may get some heating.

I blog

2009-05-04T18:04:00.000-05:00

OK, I think I'm blogging. We had a decent first day in the Spring Experiment. Things are starting to take shape and we will soon have lot's of interesting things to blog about. Hope there was no serious storm damage in Norfolk today.

HRRR Update

2009-05-03T17:22:00.002-05:00

Looks like the WRF3-HRRR (high-rez rapid refresh) has been coming in. I had to update datatype.tbl for the $MODEL/wrf3hrrr/ grid file name. Also, there does not appear to be a 1km REFL parameter in the GEMPAK output. There is REFC (composite reflectivity) and also the history parameters that allow the generation of the SSVR plots.

So, as of the 12Z run May 3, 2009 the following thumbnail image names are available:

Composite reflectivity (compare to crefr):

date_runtime/WRF-HRRR3_CREF_YYYYmmdd12_f000.gif through f012.gif

and...

Hourly synthetic severe (compare to lsr/tlsr):

WRF-HRRR3_SSVR_YYYYmmdd_f000-012.gif

For May 4th: the severe potential across much of the US looks marginal (it figures). However, there is some possibility for high-based or elevated convective initiation across parts of southeast CO into KS/OK/TX areas later in the day and perhaps again overnight. There is also the chance for an isolated LP storm with heating across TX but the chance is low. With that in mind, I decided to set the center point at CDS for Monday.

The weekend saw some remarkable storms! I tried to capture as much as possible. LIT was a good pick for Saturday's actvity and I picked TCL for the long-track bowing MCS/Derecho event on Sunday. The WRFs from NSSL and NMM both showed some amazinglt accurate simulations and were integral to forecast decisions made at SPC. The convective evolution/mode on both days was relatively well handled. This was especially true on Saturday afternoon/evening across the Arklatex:

http://hwt.nssl.noaa.gov/Spring_2009/loop_3panel.php?date=20090502&image1=nssl_00&image2=brefr&image3=nmm_00&starthr=14&frames=22

The 18-24h forecasts form the NSSL and NMM WRFs valid for Sunday afternoon were also really interesting to review. I think the NMM takes the prize in this comparison with its incredibly accurate depcition of the two linear systems over the MS/AL area Sunday morning. However, the forecast was about 3h too slow.

Compare this 17h forecast from the 4km WRF-NMM:

http://hwt.nssl.noaa.gov/Spring_2009/webimages/thumbnails/20090503_0000/WRF-NMM4_1KM-REFL_2009050300_f017.gif

with this radar image about 3 hours earlier:

http://hwt.nssl.noaa.gov/Spring_2009/webimages/thumbnails/20090503/20090503_14_brefr.gif

One last observation:

The 12Z NMM run on Friday and Saturday was poor guidance and seemed to suffer from the cold start. I would shy away from the 12Z NMM for making decisions about storm initiation based on what I saw Friday and again Saturday. There was no 12Z NMM run on Sunday morning. Not sure what happened there.

That's it. One more forecast shift on Monday and then back to some hacking on Spring Experiment graphics!

-GregC

Status on 40+ dBZ accumulation plots

2009-04-30T23:23:00.001-05:00

Initial success on the accumulating 40+ dBZ 1KM REFL from the models tonight. At left is an image from the WRF-NSSL4 that accumulates 40+ simulated reflectivity from 18z (f018) through 06z (f030). I still have to clean up the labeling and add logic to do the 12z runs.

While the observed base reflectivity data for this plot is running, don't expect these model plots to be up and running for at least another a day. -GregC