15 ADVANCED MATERIALS & PROCESSES | JANUARY/FEBRUARY 2024 The transmission electron microscope (TEM) is often considered the most complete characterization tool available to scientists who study materials’ microstructure, crystallography, composition, chemistry, and functional properties. Consequently, significant investments have been made toward improving the newest, most powerful generation of TEMs, which facilitate atomic-resolution imaging via aberration-correction, in-situ experimentation, multi-dimensional characterization (e.g., 4D STEM), and integration of application programming interfaces (APIs) that provide scripting capabilities. However, while the hardware and functionality of TEMs have advanced, little attention has been devoted to implementing new interactive, highly visual, and user-friendly software packages to help users operate the instruments and analyze data. In other words, limited interactive user assistance is available to human operators during data analysis to help streamline their decision-making processes. Development of interactive tools to assist with data analysis, particularly virtual or mixed-reality applications, could transform how users interact with TEMs during materials characterization and analysis. Virtual reality provides a totally immersive experience where 100% of a user’s vision is projected in a virtual world, but this situation causes users to lose awareness of their surrounding physical space. Mixed reality, on the other hand, differs in that users simultaneously function in their real-world environment and visualize and interact with three-dimensional computer-generated objects (called “holograms”) through hand gestures, eye tracking, and voice commands. The ability to maintain a connection to the “real world” using mixed reality is a key advantage compared to virtual reality headsets, which completely block the user’s view of the real world and limit their tactile relationship with their environment. In the context of materials research, using mixed reality augments a user’s interaction by allowing them to see a TEM column and its control panels, while also interacting with three-dimensional holograms that are projected into their field of view, as shown in Fig. 1. Thus, the mixed-reality application serves as an enhancement to the microscopist’s interface with the TEM, rather than a replacement. Introduction of mixed reality into a data-intensive research workflow is an unexplored aspect of materials characterization and analysis which could accelerate scientific discovery. BACKGROUND The overarching goal is to develop and implement mixed-reality applications to assist researchers during data collection, interpretation, and analysis of data while using TEMs. This mission is supported at Lehigh University and other partner institutions, in particular Louisiana State University (LSU) and The Ohio State University (OSU), through the Lehigh Presidential Nano|Human Interfaces (NHI) Initiative[1]. Furthermore, real-world usage of the mixed- reality application is underway in the Lightweight High Entropy Alloy Development (LHEAD) program[2]. The LHEAD program is a 2021 Cooperative Agreement between Lehigh, LSU, OSU, and the Army Research Laboratory, while the NHI Initiative was supported by former President John Simon and the Lehigh Administration in 2016. In both research programs, the Microsoft HoloLens 2 device was chosen as the mixed-reality framework to create user interfaces that can be tailored for individual users and/or analysis workflows. The implementation of mixed-reality analysis environments represents a drastic change from the Fig. 1 — A microscopist interacting with holograms during a live microscopy session. The microscopist’s point of view, while interacting with simulated images of materials from different zone axes, is projected in the region outlined in teal. common, and often tedious, methods to analyze microscopy data (e.g., images, diffraction patterns, energy dispersive spectroscopy spectra) using conventional computer screens. IDENTIFYING UNKNOWN PHASES A key outcome of efforts thus far is the creation of a mixed-reality application called the HoloMicroscope. It is designed to enhance the effectiveness of microscopists while using TEMs and to provide microscopists with auxiliary information to facilitate their decision-making and improve data analysis efficiency.
RkJQdWJsaXNoZXIy MTYyMzk3NQ==