Homepage

European Microscopy Congress 2020 Abstract Database

Deep learning assisted transmission electron microscopy.

Deep learning assisted transmission electron microscopy.

Session

DHA.1 - Deep learning for analysis and interpretation of microscopy imaging data

Authors

Matthew Helmi Leth Larsen (3), William Bang Lomholdt (2), Mukesh Narendran (3), Frederik Dahl (3), Jacob Madsen (5), Pei Liu (4), Ole Winther (1), Stig Helveg (3), Jakob Birkedal Wagner (2), Thomas Willum Hansen (2), Jakob Schiøtz (3)

Affiliations

1. DTU Compute, Technical University of Denmark
2. DTU Nanolab, Technical University of Denmark
3. DTU Physics, Technical University of Denmark
4. EMAT, University of Antwerp, 2020
5. Faculty of Physics, University of Vienna

Keywords

Catalysis

Deep-Learning

Imaging

Neural Networks

TEM

Abstract text

Figure 1. Surface diffusion - how often is a given atomic column occupied, as determined by a neural network.

Electron microscopy remains one of the fundamental components in materials design and exploration. The value of the results is however limited by manual large-scale data analysis, and by sample damages and other effects induced by the electron beam. Deep learning can hereby step in to serve as an automated analysis tool and for the analysis of lower signal-to-noise (S/N)-ratio images to minimise beam induced effects. We have a workflow implementing deep neural networks, trained on simulated transmission electron microscopy (TEM) images, with the ability to identify local structures in experimental images. This workflow is open source and available as Python modules [1, 2, 3]. The desire is to expand this into TEM software to allow for a user-friendly on-the-fly analysis tool.

Typically neural networks require vast amounts of data to optimise the process of feature learning. To generate a sufficient amount of data experimentally would be a time consuming and expensive process. Fortunately, reliable atomic-resolution TEM images can be simulated, providing an extensive set of training data at a low cost. This allows for an on-the-fly generation of a TEM image with random microscopy parameters chosen within a suitable range. This exposes the neural network to a highly diverse set of data and ensures it never meets the exact image again. This has proven to reliably train the neural network to identify positions of atoms and atomic columns in experimental clusters of gold nanoparticles, even in low S/N-ratio conditions. This opened up the possibility to detect ”events”, such as surface diffusion along the edges of the gold nanocluster presented in figure 1. Across several frames, the percentage of an atomic site being occupied (or the percentage of sites being occupied and then unoccupied in the next frame) can be extracted. This presented that the percentage of sites going from occupied to unoccupied, or vice versa, increased along the edges in an oxygen rich atmosphere in comparison to vacuum, implying an increase in diffusion [2].

By applying a focal series, at which a set of three images are generated at different focus, expands the network’s ability to identifying various chemical species. This is demonstrated for molybdenum disulfide, where the network is able to locate individual molybdenum and sulphur atoms in experimental TEM images of industry style molybdenum disulfide upon graphene substrates [4].

The neural networks hereby present promising possibilities in analysing more complex materials consisting of multiple species, bringing it a step forward towards implementation as an on-the-fly analysis tool for large-scale data analysis in TEM imaging.

References

[1]  Jacob Madsen et al. “Accuracy of surface strain measurements from transmission electron microscopy images of nanoparticles.” In: Adv. Struct. Chem. Imag. 3.1 (2017), p. 14. doi: http://dx.doi.org/10.1186/s40679-017-0047-0.

[2]  Jacob Madsen et al. “A Deep Learning Approach to Identify Local Structures in Atomic- Resolution Transmission Electron Microscopy Images.” In: Advanced Theory and Simula- tions. 1.8 (2018), p. 12. doi: http://dx.doi.org/10.1002/adts.201800037.

[3]  url: https://gitlab.com/schiotz/NeuralNetwork_HRTEM.

[4]  Christian Kisielowski et al. “Imaging MoS2 nanocatalysts with single-atom sensitivity.” In: Angewandte Chemie. 49.15 (2010), pp. 2708–10. doi: http://dx.doi.org/10.1002/anie. 200906752.