January_2021_AMP_Digital

A D V A N C E D M A T E R I A L S & P R O C E S S E S | J A N U A R Y 2 0 2 1 1 7 Fig. 1 — Only a small fraction of a ML-based segmentation or classification pipeline is composed of the ML code (small black box in the middle). Surrounding infrastructure and especially questions arising from applications in materials science (red frames) are vast and complex. Figure adapted from reference [1] . A rtificial intelligence (AI) and ma- chine learning (ML) have made their way into materials science and are omnipresent. AI is a branch of computer science that is dedicated to developing machines or algorithms to perform tasks that typically demand hu- man intelligence. In ML, a subset of AI, computers are enabled to independent- ly learn patterns and regularities from the available data without having been pro- grammed especially for this task. Within ML, deep learning (DL) is currently a very popular and promising technology. Thus, there is a growing number of publications and conferences dedicating specific ses- sions to these topics, including in materi- als science. Currently, it seems that even without the appropriate material-specif- ic background, many data scientists are jumping on this bandwagon to use ML and DL as a panacea, but occasionally without precisely grasping the complex material-specific questions. In general, only a small fraction of a ML-based seg- mentation or classification pipeline is composed of the ML code itself (Fig. 1). And when dealing with materials science tasks, additional questions and challeng- es arise that must be carefully considered (red framed boxes in Fig. 1). However, materials scientists on the other hand, for whom tradition- al image processing methods such as thresholding are still the go-to solu- tion, often do not have the necessary AI know-how for a successful imple- mentation and application. Therefore, especially collaborations between materials and computer science and new approaches. Most of these image sets consist of natural scene images, and the ground truth assignment is not problematic. Figure 2 compares and contrasts typical natural scene imag- es, e.g., pet images (Fig. 2a), with mi- crostructural images (Fig. 2b) of varying steel microstructures obtained using a scanning electron microscope (SEM). This example is very striking as it illustrates quite clearly that the ground truth assignment for microstructure classification is much more complex and can easily include a subjective com- ponent added by the human expert. This leads us to the main question this article is concerned with: How can we assign the ground truth for microstruc- tural classification or segmentation in the most objective way? This includes, for example, defining classes, assigning images to these classes, and annotating image regions for segmentation. These are all difficult tasks when the human expert must solely rely on the visual ap- pearance of the microstructure, an as- pect that is experienced in varying ways by different human experts. At this point it should be empha- sized that the ground truth assignment of the available images or data should not be looked at in isolation but in the context of a holistic way of building the ML model, which starts with choosing suitable samples and establishing re- producible sample contrasting (Fig. 1). Considering all these facts, it becomes clear that computer science expertise alone is not sufficient to solve AI tasks in materials science, but that domain knowledge of materials science and metallography is indispensable. For this case study, bainitic micro- structures have been chosen to illus- trate the previously mentioned chal- lenges and approaches on how to overcome them. Bainite is an essential constituent of modern high-strength steels. It combines high strength with high toughness, making it interest- ing for a variety of applications [9] . In addition to the existing challenge of characterization, the classification of bainite poses difficulties. Challenges when dealing with bainite include the variety and amount of involved phases, interdisciplinary research can create new synergies and great potential for applications inmaterials science. Prom- inent examples for AI and ML in mate- rials science include the classification or segmentation of complex steel mi- crostructures [2-4] , processing-structure- property links, or ML-aided materials design and discovery [5,6] . Overviews of the spectrum of applications can be found in the references [7,8] . Areas in which ML offers deci- sive improvements are microstructure segmentation and classification. An essential and often mentioned advan- tage of ML is the increased objectivi- ty compared to human expert results. However, during building the ML mod- el in supervised learning, the so-called ground truth must be assigned by the human expert, i.e., the expert tells the ML algorithm which images or data points belong to which class. During this ground truth assignment, espe- cially in the case of complex micro- structures, a considerate subjective component can potentially be intro- duced that would negatively affect the performance of the entire segmenta- tion or classification pipeline. While other “ML-inherent” issues like overfit- ting or generalizability have been ex- tensively addressed, the problem of a subjective ground truth assignment for materials science applications is usual- ly not sufficiently discussed. To better understand this issue, it may be helpful to consider typical benchmark image and data sets that are available in com- puter science for learning or developing