In a Landslide Test, the Lowest-Error Model Missed Every Warning Event

The lowest-error model in a landslide ablation study missed every warning event.

At Slovenia's Urbas landslide, a purely autoregressive model produced the lowest median daily-displacement error among six progressively reduced models: 1.82 millimeters. Yet its F1 score for detecting upward crossings of the study's two-millimeter warning threshold was 0.000. It detected none of the test set's 11 positive events.

A multi-task neural network performed a little worse on the average-error metric, with median mean absolute error of 1.85 millimeters, but reached median F1 of 0.526 for warning events. The contrast exposes a central design problem in disaster AI: a model can look accurate on ordinary days while failing at the rare moments that matter most.

The result is not an operational warning system. It is a retrospective, single-site test showing why models intended for public safety should be trained and judged against the decisions people actually need to make.

Research references: [1]

A Mountain Monitored Above a Community

Urbas is a slow-moving, deep-seated landslide in the Karavanke mountains of northwestern Slovenia. It sits above Koroška Bela, a settlement of roughly 2,200 people with a history of debris flows.

The mapped landslide covers about 177,000 square meters, and researchers estimate that its sliding mass could reach 1.5 million cubic meters. Its lower section is particularly active and could mobilize into a debris flow.

Continuous monitoring began in 2019. Four solar-powered Global Navigation Satellite System units measure surface movement, supported by piezometers, rain gauges and other environmental observations. The study modeled the daily displacement measured at GNSS4, a station at the landslide toe where movement is most active.

The public-safety stakes are real. During extreme rainfall in August 2023, a smaller slide in the lower Urbas area broke alarm loops and activated sirens. Slovenian police reported that civil protection ordered overnight evacuation of about 900 residents from the threatened part of Koroška Bela. A geological post-event report independently describes the activated landslide, broken loop system and intervention.

That event was not a trial of the new model. The research asks whether sensor data could eventually add an anticipatory layer before a mechanical alarm is triggered.

Research references: [1] [2] [3]

Predicting Movement and Warning Transitions Together

The researchers trained convolutional neural networks to predict next-day displacement from recent landslide movement, rainfall, groundwater, snow, temperature and earthquake indicators.

The single-task model predicted displacement alone. The multi-task version learned three related objectives at once: next-day displacement, next-day change in maximum groundwater level, and whether movement would cross from below to above a predefined warning threshold.

The threshold was two millimeters of daily displacement. The authors selected it empirically as an early indicator of possible acceleration, above estimated measurement error and low enough to provide positive cases for training. It was not an evacuation trigger or an operational public-warning standard.

The complete displacement record contained 1,562 daily points from October 2019 through October 2024. Researchers trained through March 2024 and evaluated on the remaining period. The final test set contained only 11 positive threshold crossings, which makes every warning metric sensitive to a small number of cases.

Research references: [1]

Why Mean Error Hid the Failure

Mean absolute error rewards predictions that stay close to observed daily movement. When ordinary or receding days dominate the record, a model can score well by following the prevailing pattern.

The ablation study removed external variables in stages. The final M5 version used only recent displacement, with no rainfall, groundwater, snow, temperature or earthquake inputs. Its median MAE of 1.82 millimeters was the lowest among the six ablation versions.

But its F1, precision and recall for threshold crossings were all zero.

The complete multi-task model reported median MAE of 1.85 millimeters and median warning-event F1 of 0.526 across 15 training runs. Its median precision was 0.625 and recall 0.455. The best multi-task run reached MAE of 1.79 millimeters and F1 of 0.636, with precision and recall both 0.636.

The single-task neural network reached similar median MAE of 1.88 millimeters but lower median F1 of 0.316.

The result should not be presented as a sweeping victory for deep learning. A conventional ARIMAX model reached F1 of 0.632, close to the best multi-task run, although its MAE was higher at 2.46 millimeters. Linear regression reached F1 of 0.571.

The clearest conclusion is not that one architecture won. It is that the choice of target and metric changed which model appeared useful.

Research references: [1]

Rainfall Carried the Strongest Signal

The ablation sequence also showed how external conditions affected warning detection.

Removing earthquake indicators reduced median F1 from 0.526 to 0.476. Removing snow and temperature as well reduced it to 0.421. A model retaining only rainfall alongside recent displacement scored 0.375. When rainfall was removed too, the autoregressive model fell to zero warning-event F1.

The paper's statistical analysis identified rainfall as the main external driver of Urbas acceleration. Snowmelt and earthquake inputs produced small median improvements, but the overall performance differences associated with those inputs were not statistically significant.

That dependence makes weather-forecast quality important. The researchers simulated next-day rainfall estimates biased by 25, 50 and 100 percent. Increasing rainfall by up to 50 percent did not significantly reduce threshold-detection performance. Underestimating rainfall did reduce detection significantly.

This was an artificial sensitivity test, not a live weather-forecast evaluation. During model assessment, the system used measured contemporaneous weather values. Prospective use would need real forecast inputs and their real error distribution.

Research references: [1]

What the Study Does Not Establish

This evaluation used one primary GNSS target at one landslide. It was retrospective, not a prospective early-warning trial.

Missing sensor records were cleaned and imputed, including several extended gaps. The selected threshold was experimental. Geological triggers, useful thresholds, sensor quality and warning protocols may differ at other sites.

Most importantly, the authors state that current prediction accuracy does not meet operational landslide early-warning requirements. Median recall of 0.455 means the multi-task model still missed more than half of the positive test events across its median run.

No vendor product was evaluated. The study did not measure warning delivery, evacuation decisions, false-alarm burden, casualties prevented, property protected or response costs.

The data are available from the corresponding author on reasonable request. The publisher record does not identify a public study-specific code repository. The work was funded by Slovenian research programs, the European Union's Green.DAT.AI project and the Interreg AIMS project. The authors declared no competing interests.

Research references: [1]

A Better Design Target for Warning AI

The near-term opportunity is not autonomous evacuation. It is a human-supervised forecast layer that helps experts interpret the sensor network.

A useful interface could show predicted movement, threshold-crossing probability, recent rainfall and groundwater trends, input quality and uncertainty together. It should flag missing or imputed data and remain subordinate to physical monitoring, geotechnical judgment and public authority.

Before operational use, the approach needs prospective testing with real weather forecasts, more threshold events, additional sensors and redundancy, predefined alert protocols, measurement of missed events and false alarms, and external validation at other landslides.

The broader lesson already travels well: optimize the decision people need to make, not only the statistic that is easiest to minimize.