A Benchmark Time Series Dataset for Semiconductor Fabrication Manufacturing Constructed using Component-based Discrete-Event Simulation Models
基于组件离散事件仿真模型的半导体制造基准时间序列数据集

Simulation of automated manufacturing processes is essential for the efficient use of complex and expensive machines. This requires effective coordinated decision-making due to a variety of factors including market conditions and processes spanning days to months with end-to-end controls of networked machines in minutes. Extensive resources are employed to create accurate physics-based manufacturing models to achieve these well-understood needs. In addition, designing, conducting, and evaluating simulation experiments are major undertakings. To reduce cost and time for practitioners, research in machine learning methods supporting building manufacturing digital twins is attracting greater emphasis among academic and industry researchers. There are many benefits of machine learning models for analytics. Executing data-based models compared with their counterpart physics-based models is expected to be computationally scale-free. The concept of models primarily generated from data is aimed at responding faster to the changes that can lead to knowing how best manufacturing systems should operate under short/long time horizon requirements. Furthermore, generated ML model libraries can be frequently updated as additional data becomes available relative to physics-based models. It is also possible to generate synthetic data using physics-based simulation of semiconductor factories. This class of data is strictly prescriptive and structured. The basic entities that undergo various kinds of processing include dies on wafers, wafers making lots, and wafer lots assembled into batches. Inventories, machines, and transportation are the entities that are used for processing semiconductor chips. Every entity can be ascribed one or more quantitative/qualitative variables. Every variable has a measurable value at any instance within a finite period. Other data are also measured and aggregated, such as average inventory volume, work-in-progress in machines, and wafers’ transportation routes. It is common for every machine to behave either deterministically or stochastically. The time designated to each entity in factories can be continuous or discrete. The collected data can have arbitrary accuracy and precision. Formal models of factories are suitable for creating sound datasets for use in developing regression and deep machine-learning models. Given the event-driven nature of manufacturing systems, Discrete-Event Simulation (DES) is widely used. This approach naturally defines inputs and outputs as events that can occur at any arbitrary time. The events can interrupt any process and the processes can be combined to form factories using well-formed input and output relationships. However, the development of ML models depends on having rich data combined with expert domain knowledge among other factors. While massive amounts of live data are collected from semiconductor factories (e.g., from work centers to enterprise supply chain systems [29]), it is challenging to generate ML models. In a semiconductor fabrication factory, data should be collected, organized, maintained, and synchronized across different individual and networked machines with varying logical/physical topologies in time and space. Given that collecting data from actual semiconductor factories is restricted due to the company’s proprietary constraints and the resources required for data engineering, benchmark datasets can be developed using simulations of physics-based models. Indeed, various efforts have been undertaken to collect data from physics-based models for use in statistical and machine-learning studies [15, 23].
自动化制造过程的模拟对于高效使用复杂而昂贵的机器是必不可少的。这需要根据各种因素进行有效的协调决策，这些因素包括市场条件和流程，时间跨度从几天到几个月，对联网机器的端到端控制只需几分钟。我们使用大量的资源来创建精确的基于物理的制造模型，以实现这些众所周知的需求。此外，设计、进行和评估模拟实验是一项重要的工作。为了减少从业者的成本和时间，支持建筑制造数字双胞胎的机器学习方法的研究正吸引着学术界和行业研究人员的更多重视。机器学习模型对分析有很多好处。与相应的基于物理的模型相比，执行基于数据的模型预计在计算上是无尺度的。主要由数据生成的模型的概念旨在更快地响应变化，从而了解制造系统应如何在短/长时间范围要求下最佳运行。此外，随着相对于基于物理的模型的额外数据变得可用，所生成的ML模型库可以被频繁地更新。也可以使用半导体工厂的基于物理的模拟来生成合成数据。这类数据具有严格的规范性和结构化。经过各种加工的基本实体包括晶片上的芯片、制造批次的晶片和组装成批次的晶片批次。库存、机器和运输是用于加工半导体芯片的实体。每个实体都可以归因于一个或多个定量/定性变量。 在有限的时间内，每个变量在任何情况下都有一个可测量值。其他数据也被测量和汇总，如平均库存量、机器中的在制品和晶圆的运输路线。每台机器的行为要么是确定性的，要么是随机的，这是很常见的。指定给工厂中每个实体的时间可以是连续的，也可以是离散的。采集的数据可以有任意的准确度和精确度。工厂的正式模型适合于创建合理的数据集，用于开发回归和深度机器学习模型。鉴于制造系统的事件驱动特性，离散事件仿真(DES)得到了广泛的应用。这种方法很自然地将输入和输出定义为可以在任意时间发生的事件。这些事件可以中断任何流程，并且可以使用格式良好的输入和输出关系将这些流程组合成工厂。然而，ML模型的发展依赖于拥有丰富的数据和专家领域的知识等因素。虽然从半导体工厂(例如，从工作中心到企业供应链系统[29])收集了大量的实时数据，但生成ML模型是具有挑战性的。在半导体制造工厂中，应该在时间和空间上具有不同逻辑/物理拓扑的不同的单独和联网的机器上收集、组织、维护和同步数据。考虑到从实际半导体工厂收集数据受到公司专有限制和数据工程所需资源的限制，基准数据集可以使用基于物理的模型的模拟来开发。 事实上，人们已经做出了各种努力来从基于物理的模型中收集数据，以用于统计和机器学习研究[ 15，23]。

Manufacturing systems can be modeled as continuous differential equations, discrete-time, and discrete event specification languages [30]. Following system theory, they can be modeled as a set of atomic and coupled components. Every atomic model has the means to specify input, output, and state variables and state transition, output, and timing functions. Every coupled component has its own input and output variables. It has a set of atomic and other coupled models. Atomic and coupled models communicate through sending and receiving inputs (input variables) and outputs (output variables). Simulation output (data) naturally is partitioned and belongs to the atomic and coupled components.
制造系统可以建模为连续微分方程、离散时间和离散事件规范语言[30]。根据系统理论，它们可以被建模为一组原子和耦合组件。每个原子模型都有指定输入、输出和状态变量以及状态转换、输出和时间函数的手段。每个耦合组件都有自己的输入和输出变量。它具有一组原子和其他耦合模型。原子和耦合模型通过发送和接收输入（输入变量）和输出（输出变量）进行通信。模拟输出（数据）自然地被分割并属于原子和耦合组件。

Machine learning (ML) has been pivotal in advancing the traditional semiconductor manufacturing process [13, 2]. As Jiang et al. [14] discusses, ML can be effectively utilized to improve the yield in semiconductor manufacturing. Various methods have been explored for data collection necessary for developing these ML models. For instance, Liu et al. [17] reviews the use of different ML algorithms and available datasets for enhancing semiconductor manufacturing. Shin and Park [26] derive data from manufacturing logs, while Saif M. Khan [24] highlights a publicly available dataset of the semiconductor manufacturing process. The dataset [18] offers time series data for a specific factory configuration, limiting its flexibility to alter factory settings and parameters.
机器学习（ML）在推动传统半导体制造过程方面发挥了关键作用[13, 2]。正如江等人[14]所讨论的，ML 可以有效地用于提高半导体制造中的产量。已经探索了用于开发这些 ML 模型所需的数据收集的各种方法。例如，刘等人[17]回顾了不同的 ML 算法和可用数据集，以增强半导体制造。Shin 和 Park[26]从制造日志中提取数据，而 Saif M. Khan[24]强调了一个公开可用的半导体制造过程数据集。该数据集[18]为特定工厂配置提供了时间序列数据，限制了其灵活性以更改工厂设置和参数。

In this research, we focus on using Discrete Event Simulation (DES) to generate data [6] for a given factory configuration, which can then be used to develop ML algorithms. Specifically, we aim to predict the overall throughput of the factory for a given time instance. Although Godahewa et al. [11] discusses various time series datasets and evaluation metrics along with the implementation of time series models, there is no exclusive dataset related to semiconductor manufacturing. The work by Singgih [27] uses the MiniFab model description [28] to develop a model in Anylogic. A dataset from this DES model is generated and used to develop ML classification models. We extend the functionalities of the simulation model to incorporate features such as repair state and wafer generation dataset. One key aspect of semiconductor manufacturing is predicting the throughput of a fabrication plant for a given time instance and factory configuration [7]. We perform data analysis with different factory configurations and analyze parameters such as throughput and turnaround time of a semiconductor manufacturing process at different time instances.
在这项研究中，我们专注于使用离散事件模拟（DES）来生成给定工厂配置的数据[6]，然后可以用这些数据来开发机器学习算法。具体来说，我们的目标是预测给定时间点工厂的总吞吐量。尽管 Godahewa 等人[11]讨论了各种时间序列数据集和评估指标，以及时间序列模型的实施，但没有专门与半导体制造相关的数据集。Singgih 的工作[27]使用 MiniFab 模型描述[28]在 Anylogic 中开发模型。从这个 DES 模型生成了一个数据集，用于开发机器学习分类模型。我们扩展了模拟模型的功能，以包括修复状态和晶圆生成数据集等特性。半导体制造的一个关键方面是预测给定时间点和工厂配置下制造厂的吞吐量[7]。我们对不同的工厂配置进行数据分析，并分析半导体制造过程在不同时间点的吞吐量和周转时间等参数。

The PDEVS models are used to simulate a set of experiments conducted for the single-stage and multi-stage factories using the DEVS-Suite simulator [1]. Four transducer models are used to measure and collect input, output, and state information at every simulation execution step. An eight-stage cascade model is created using the single-stage model. This model generates scenarios that exhibit higher structure and behavior complexities. Based on the logic of wafer processing of the MiniFab we formed 93 different tuples of Pa, Pb, and Tw values, each comprising different lot configurations ranging from small, medium, and large lot sizes relative to each other. In addition to the lot size and lot configurations, we can also specify how the model can process a particular batch of wafers (see Table 1). We have 372 simulation scenarios for the eight-stage cascade factory models based on wafer lot sizes, configurations, uniform and sinusoidal patterns, and repair mode. Using the fabrication model, 93 experiments are simulated, each representing a scenario for Pa, Pb, and Tw. These simulated experiments are chosen to form small, medium, and large lot configurations with unique lot sizes. (see Table 1).
PDEVS 模型用于使用 DEVS-Suite 模拟器[1]对单级和多级工厂进行一系列实验。四个传感器模型用于在每次模拟执行步骤中测量和收集输入、输出和状态信息。使用单级模型创建了一个八级级联模型。该模型生成展示更高结构和行为复杂性的场景。基于 MiniFab 晶圆加工逻辑，我们形成了 93 个不同的 Pa、Pb 和 Tw 值元组，每个元组包含不同的批次配置，从小、中到大批次大小相对于彼此。除了批次大小和批次配置外，我们还可以指定模型如何处理特定批次的晶圆（见表 1）。基于晶圆批次大小、配置、均匀和正弦模式以及维修模式，我们有 372 个八级级联工厂模型的模拟场景。使用制造模型，对 Pa、Pb 和 Tw 进行了 93 次实验模拟，每次代表具有独特批次大小的小、中和大批次配置。 （见表 1）。

The output generated from each simulation is stored in comma-separated value (CSV) files for individual atomic and coupled factory model components. Each simulated component has several rows, each representing an instance of time associated with the columns of data for every model (e.g., throughput shown in Figure 1). For example, data on loading and transportation time trajectories are essential for determining optimal factory operation under different factory configurations. Data collected for throughput and turnaround time can be used to build surrogate machine-learning models, which can be trained to mimic the simulated behavior of the factory models as closely as possible. Every of the simulation output files has a column depicting the logical time of the simulation, hence plotting different values against time can provide us with time series data. These values’ temporal nature provides insight into wafer processing at each step. Our focus of this research is to analyze the throughput of a semiconductor manufacturing factory with different configurations. Now we plot the throughput values against time with an interval of 1 minute between every successive time value as depicted in Figure 1.
每次模拟生成的输出都存储在逗号分隔值（CSV）文件中，用于单个原子和耦合工厂模型组件。每个模拟的组件都有几行，每行代表与每个模型数据列相关联的时间实例（例如，吞吐量如图 1 所示）。例如，加载和运输时间轨迹的数据对于确定在不同工厂配置下的最佳工厂运行至关重要。收集的吞吐量和周转时间数据可用于构建替代机器学习模型，这些模型可以被训练成尽可能模仿工厂模型的模拟行为。每个模拟输出文件都有一列描述模拟的逻辑时间，因此将不同值绘制在时间轴上可以为我们提供时间序列数据。这些值的时间性质可以洞察每个步骤的晶圆加工。我们研究的重点是分析具有不同配置的半导体制造工厂的吞吐量。 现在我们将吞吐量值根据时间绘制在一分钟的间隔内，每个连续的时间值如图 1 所示。

In these plots, we can see that there are some empty values at certain time instances, the empty values are based on the fact that our discrete event simulation records values at an event occurrence. The conversion of these values into a complete time series dataset would require us to have values at each time instance (Section 4.1). Even though the plots have a similar monotonically increasing trend for different configurations the values at a granular level differ from each other. The differences can be visualized in the plots based on different configurations of the factory.
在这些图中，我们可以看到在某些时间点存在一些空值，这些空值是基于我们的离散事件模拟在事件发生时记录值的事实。将这些值转换为完整的时间序列数据集需要我们在每个时间点都有值（第 4.1 节）。尽管这些图表在不同配置下具有类似的单调增长趋势，但在粒度级别上的值却有所不同。根据工厂不同配置的图表可以看出这些差异。

The simulation output is transformed from a discrete event (Figure 1) to a discrete-time time series (Figure 2) by filling the missing values with previous values. For constructing a baseline model, we have used multiple time series forecasting models: Auto-Regressive Integrated Moving Average (ARIMA), Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Temporal Convolutional Neural Network (TCN) and Temporal Fusion Transformers (TFT). The choice of selecting these models was based on the uniqueness of the dataset considering the large size of the dataset (25,000 instances) and the monotonically increasing nature with granular differences in the values of throughput for different configurations. ARIMA captures linear trends and seasonality [5], RNN and LSTM handle sequential data and dependencies [10, 12], TCN models long-range temporal dependencies [3] and TFT model considers the impact of static variables (e.g., repair state, lot size, etc.) for time series forecasting [16]. As per feature analysis in Section 4.2, some models may outperform others. Choosing the right metrics to evaluate the performance of these models on the dataset is significant. Mean-square error (MSE), is significant for evaluating forecasting performance, but since, Mean-Square Error is not a scale-free error metric and due to the small scale ($\sim 10^{-3}$) of throughput values, we get small scale values ($\sim 10^{-6}$) of MSE, hence it is convenient to use MSE as a relative measure to evaluate a model’s performance. The value of Mean Average Percentage Error (MAPE) can be used to assess the performance of a model as it is scale-free, but since MAPE cannot handle cases where the actual value of throughput can be zero, we consider non-zero values of throughput for computing MAPE. Next, we used $R^{2}$ scores to evaluate the performance, which is both scale-independent and can handle zero values in the dataset, but it can be misleading for non-linear relationships, sensitive to outliers, and does not account for overfitting or model complexity. Therefore, $R^{2}$ should be used with other metrics for a comprehensive evaluation of model performance. Finally, we use the Mean-Forecast Error (MFE) which directly compares the actual and predicted values hence it is scale-dependent, for throughput prediction it can be of the same scale as the actual values ($\sim 10^{-3}$). All the above metrics can be used as a combination to evaluate the performance of a model, since the dataset consists of multiple zero values and a small scale, relying on a single error metric can lead to ambiguous results. Additionally, with these metrics, it is also pertinent to visualize the prediction plots as there are granular differences in the overall throughput values. Performance evaluation of different time series models for an exemplary simulation configuration: Pa = 10, Pb = 90, Tw = 20, Lot Size = 120, Repair State = No Repair, Wafer Generator = Uniform results are as depicted in Figure 4 (a). Since the TFT model considers also the static covariates of a time series, the evaluation for the TFT model has been done for different configurations as seen in Figure 4 (b). The time series models predict throughput values based on a look-back window of size 10 and the error metrics for the model evaluation are as mentioned in Table 2.
模拟输出通过用先前的值填充缺失值，从离散事件（图 1）转换为离散时间序列（图 2）。为构建基准模型，我们使用了多个时间序列预测模型：自回归综合移动平均（ARIMA）、循环神经网络（RNN）、长短期记忆（LSTM）、时间卷积神经网络（TCN）和时间融合变压器（TFT）。选择这些模型是基于数据集的独特性，考虑到数据集的规模较大（25,000 个实例）以及吞吐量值在不同配置下的粒度差异呈单调增加的特性。ARIMA 捕捉线性趋势和季节性[5]，RNN 和 LSTM 处理序列数据和依赖性[10,12]，TCN 模型长期时间依赖性[3]，TFT 模型考虑静态变量（例如，维修状态、批量大小等）对时间序列预测的影响[16]。根据第 4.2 节的特征分析，一些模型可能表现优于其他模型。选择正确的指标来评估这些模型在数据集上的性能是重要的。 均方误差（MSE）对于评估预测性能至关重要，但由于均方误差不是无量纲的错误度量，并且由于吞吐量值的小尺度（），我们得到小尺度值（）的均方误差，因此使用 MSE 作为相对度量来评估模型的性能是方便的。均值百分比误差（MAPE）的值可用于评估模型的性能，因为它是无量纲的，但由于 MAPE 无法处理吞吐量实际值为零的情况，我们考虑非零吞吐量值来计算 MAPE。接下来，我们使用分数来评估性能，这既是独立于尺度的，又可以处理数据集中的零值，但对于非线性关系可能会产生误导，对异常值敏感，并且不考虑过拟合或模型复杂性。因此，应与其他指标一起用于全面评估模型性能。 最后，我们使用均值预测误差（MFE），它直接比较实际值和预测值，因此它是依赖于比例的，对于吞吐量预测，它可以与实际值（）具有相同的比例。所有上述指标可以结合使用来评估模型的性能，因为数据集包含多个零值和小规模，依赖单个错误指标可能导致模棱两可的结果。此外，使用这些指标时，还有必要可视化预测图，因为整体吞吐量值存在细微差异。对于示例仿真配置的不同时间序列模型的性能评估：Pa = 10，Pb = 90，Tw = 20，批量大小 = 120，维修状态 = 无维修，晶圆生成器 = 均匀结果如图 4（a）所示。由于 TFT 模型还考虑了时间序列的静态协变量，因此 TFT 模型的评估已针对不同配置进行，如图 4（b）所示。时间序列模型根据大小为 10 的回顾窗口预测吞吐量值，模型评估的错误指标如表 2 所述。

The ARIMA model has an MSE value of $3\times 10^{-7}$ and a MAPE value of $8\%$ but the prediction plot of the ARIMA model shows sub-optimal performance by the model. Hence the error metric along with the actual plot serves as a strong parameter when compared to error metrics alone in the case of throughput prediction.
ARIMA 模型具有 MSE 值为和 MAPE 值为，但 ARIMA 模型的预测图显示出模型性能不佳。因此，在吞吐量预测的情况下，误差度量与实际图形一起作为强参数，与仅使用误差度量相比更为重要。

Machine learning (ML) models suitable for time series can benefit from concise and accurate datasets collected from physics-based simulations. These ML models are desirable due to their significant execution speedup compared to simulating causal models. The ML datasets facilitate a wide range of metrics and measurements. Principal Component Analysis (PCA) captures key time series features and helps understand the role and impact of various semiconductor fabrication factory configurations on essential operational measures. We have developed datasets using PDEVS models and Intel’s benchmark factory description. We detailed dataset generation for multi-stage factories for uniform (with and without repair/maintenance) and sinusoidal wafer lot experimental configurations. The utility of these datasets is demonstrated by developing time series models, such as TCN and TFT. They can make predictions comparable to those obtained from PDEVS simulations, with TFT accounting for static covariates. Future work includes developing additional models and benchmark datasets for semiconductor supply-chain systems.
机器学习（ML）模型适用于时间序列，可以从基于物理模拟收集的简洁准确数据集中受益。与模拟因果模型相比，这些 ML 模型由于其显著的执行加速而备受青睐。ML 数据集促进了各种指标和测量。主成分分析（PCA）捕捉关键的时间序列特征，并有助于理解各种半导体制造工厂配置对关键运营指标的作用和影响。我们使用 PDEVS 模型和英特尔的基准工厂描述开发了数据集。我们详细介绍了多阶段工厂的数据集生成，包括均匀（带和不带维修/保养）和正弦晶圆批量实验配置。通过开发时间序列模型，如 TCN 和 TFT，展示了这些数据集的实用性。它们可以进行与从 PDEVS 模拟获得的预测相媲美的预测，TFT 考虑了静态协变量。未来工作包括为半导体供应链系统开发额外的模型和基准数据集。

This research is funded by Intel Corporation, Chandler, Arizona, USA.
这项研究由美国亚利桑那州钱德勒的英特尔公司资助。