AI and time-series for detecting landslides from optical imagery

Overview

Earthquakes in mountainous areas can trigger thousands of landslides causing significant damage and disruption (Fig 1). Efforts to understand the controls on these landslides rely heavily on manually mapped landslide inventories but collecting these inventories is time consuming and relies heavily on costly high-resolution imagery. Lower resolution (e.g. Landsat) imagery offers near global coverage, extending over multiple decades with high temporal resolution but relatively low spatial resolution (Fig 2).
If an effective approach could be found to take advantage of these lower resolution datasets this could considerably increase the archive of landslide inventories. It might also improve the speed and accuracy with which landslide information could be provided to those responding to an earthquake.
Over the last decade, computer vision and artificial intelligence has made rapid progress in identifying everything from cars to cats. This has been enabled by progress in deep learning using a type of neural network dubbed a ‘Convolutional Neural Network’ (e.g. Fig 3; Zhang et al. 2016). Whilst these types of networks are now abundantly used by tech giants such as Google and Facebook, their application in a geographic context remains sparse (Marochov et al., 2020).
This project will examine two alternative approaches to landslide detection: 1) a multi-temporal approach that harnesses the rich time series of available medium resolution imagery and that can be implemented within cloud computing infrastructure (e.g. Google Earth Engine); and 2) a neural network driven approach that will approach the human ability for generalized feature detection but without having the range of cognitive biases that affect human pattern matching.
Specific objectives are:
O1: Develop new classification tools for use with widely available medium resolution imagery.
O2: Tailor a subset of these tools to generate rapid landslide maps following large earthquakes.
O3: Test the classification tools against existing landslide inventories in a diverse range of environments.
O4: Demonstrate the impact of extending the set of mapped landslide inventories by applying the new methods to several earthquakes where landslide inventories have not been available.

Methodology

Time-series methods: using Landsat and Sentinel time series within GEE, building on recently developed pixel-wise NDVI differencing methods (Fig 4; Milledge et al., 2021) and exploring more refined object based methods within the constraints of GEE.
Neural Network methods: Convolutional Neural Networks (CNN) will be implemented in the Keras API using the TensorFlow libraries as a backend (open-source tools from Google inc.). CNNs will be trained from existing landslide inventories to develop an AI capable of recognizing image regions where landslides are present. From there, class activation maps will be used to delineate the landslide in the same manner that a human operator would.
Testing against observed landslide inventories: classifiers will be tested against open-access hand-mapped landslide inventories (>20 inventories available each containing >5000 landslides, e.g. Fig 3). Testing will include not only standard pixel and object based metrics but also landslide specific metrics such as: size distributions, spatial density maps.
Application to historic earthquakes: the new classifiers will be applied to a set of existing earthquakes to generate landslide inventories where none currently exist; then use them to test existing theory and models in these new settings. Example applications might include: the relationship between earthquake magnitude and landslide response of Marc et al. (2016); statistical and mechanistic susceptibility models (e.g. Nowicki-Jessee et al., 2018); and the method of Meunier et al. (2013) to use landslide distributions to reveal earthquake characteristics.

Project Timeline

Year 1

Familiarising with AI and traditional classification methods and selecting a suitable initial approach drawing on in house expertise; familiarising with landslide characteristics and the requirements of the research and disaster response communities; first iteration of timeseries and AI classifiers.

Year 2

Refinement and testing of timeseries and training of AI classifiers using existing landslide inventories.

Year 3

Application to multiple historic earthquakes where medium resolution imagery exists. Testing existing theory and models in these new settings. Present results at AGU Fall meeting (Washington DC).

Year 3.5

Synthesise datasets; attend international conferences; publication and thesis writing.

Training
& Skills

Training in (a) research skills and techniques and (b) research environment are provided through four mechanisms: (i) a programme of taught modules; (ii) internal training ‘workshops’ that focus on key geographical research skills and techniques; (iii) input from supervisors; and (iv) School and academic Group seminars by visiting and internal speakers and presentations by postgraduate students themselves.
In addition to generic training offered by the University, the School also provides training through a series of in-house ‘workshops’. Engineering research postgraduates normally take the following Workshops: ‘Scientific Writing’, ‘Research Ethics (Theory)’, ‘Data Management’, ‘Time management’, ‘Document Management – Content and Layout’, ‘Introduction to Learning and Teaching’ during their first year. Students will likely undertake 2-4 taught modules tailored to their background and research focus from the MSc in ‘Mapping and Geospatial Data Science’. These modules are typically delivered in one intensive week so well suited for PhD students.
The candidate will receive training from supervisors on python coding (used for the CNN scripts). The candidate will also be funded for attendance at the conference of the British Machine Vision Association in year 2 and 3 in order to further develop their machine vision skills.

References & further reading

Carbonneau & Marochov. 2020. SEE_ICE: Glacial Landscape Classification with Deep Learning (1.0). https://doi.org/10.5281/zenodo.4081095
LeCun et al., 2015. Deep learning. Nature, 521.
Marochov, Stokes & Carbonneau. 2020. Image Classification of Marine-Terminating Outlet Glaciers using Deep Learning Methods, The Cryosphere Discuss, https://doi.org/10.5194/tc-2020-310
Marc et al., 2016. A seismologically consistent expression for the total area and volume of earthquake‐triggered landsliding. JGR-ES, 121.
Meunier et al., 2013. Landslide patterns reveal the sources of large earthquakes. EPSL, 363.
Milledge et al., 2021. Automated landslide detection outperforms manual mapping for several recent large earthquakes. NHESSD. https://doi.org/10.5194/nhess-2021-168
Nowicki Jessee et al. 2018. A global empirical model for near‐real‐time assessment of seismically induced landslides. JGR-ES, 123.
Zhang et al. 2016. Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geosci. Remote Sensing Magazine, 4.
Google Earth Engine change detection: https://cmscheip.users.earthengine.app/view/hazmapper
Google Earth Engine landslide detection: https://github.com/DavidMilledge/ALDI

Further Information

Dr David Milledge: david.milledge@newcastle.ac.uk
Dr Patrice Carbonneau: patrice.carbonneau@durham.ac.uk

Apply Now