Multi-view multi-task spatiotemporal graph convolutional network for air quality prediction
用于空气质量预测的多视图多任务时空图卷积网络

2.1. Air quality prediction
2.1. 空气质量预测

Air quality prediction has garnered considerable attention, and several efficient works have been produced. These works can be classified into statistic learning-based methods and deep learning-based methods. Statistic learning-based methods mainly include linear regression, ARIMA and SVR (Chang and Lin, 2011; Howse et al., 1997; Díaz-Robles et al., 2008). Linear regression predicts air quality by identifying relationships between air quality and relevant features. ARIMA discovers patterns in previous data over time and finally uses them to predict future air quality. (Sánchez et al., 2013) establishes a highly nonlinear urban air quality model based on SVM. However, these methods produce poor results because it is challenging to model the relationship among all affecting factors.
空气质量预测引起了相当大的关注，并产生了一些有效的工作。这些工作可以分为基于统计学习的方法和基于深度学习的方法。基于统计学习的方法主要包括线性回归、ARIMA 和 SVR（Chang 和 Lin，2011 年;Howse 等人，1997 年;Díaz-Robles等人，2008年）。线性回归通过识别空气质量与相关要素之间的关系来预测空气质量。ARIMA在以前的数据中发现一段时间内的模式，并最终使用它们来预测未来的空气质量。（Sánchez et al.， 2013）建立了一个基于支持向量机的高度非线性城市空气质量模型。然而，这些方法产生的结果很差，因为对所有影响因素之间的关系进行建模具有挑战性。

Recently, researchers attempted to utilize deep learning-based methods to predict air quality. Zheng proposed a multi-level attention-based recurrent neural network (named GeoMAN) (Liang et al., 2018) to predict the readings of a station over a couple of future hours. To further improve the performance of prediction, Multi-Group (Zhang et al., 2019) model local weather influences and fuse heterogeneous data for next-day air quality prediction. However, these models ignore the case that some distant stations could also have strong dependencies due to high similarities in other aspects. Later, ATGCN (Wang et al., 2021) model diverse inter-station relationships for air quality prediction of citywide stations. However, all of the above works still ignore the influence of air quality levels on air quality prediction tasks.
最近，研究人员试图利用基于深度学习的方法来预测空气质量。Zheng提出了一个基于多层次注意力的递归神经网络（名为GeoMAN）（Liang et al.， 2018）来预测未来几个小时内电台的读数。为了进一步提高预测性能，Multi-Group（Zhang et al.， 2019）对当地天气影响进行建模，并融合异构数据进行次日空气质量预测。然而，这些模型忽略了一些遥远的台站也可能由于在其他方面的高度相似性而具有很强的依赖性的情况。后来，ATGCN（Wang et al.， 2021）对各种站间关系进行建模，用于预测全市站的空气质量。然而，上述所有工作仍然忽略了空气质量水平对空气质量预测任务的影响。

Therefore, this paper proposes a novel model that takes air quality level as an auxiliary task to predict air quality value better.
因此，本文提出了一种以空气质量水平为辅助任务的新模型，以更好地预测空气质量值。

2.2. Graph convolutional networks in spatial-temporal prediction
2.2. 时空预测中的图卷积网络

Graph convolutional network extends a convolutional neural network to non-Euclidean data. It has been used a lot in recent years for spatial-temporal prediction tasks like traffic prediction and air quality prediction.
图卷积网络将卷积神经网络扩展到非欧几里得数据。近年来，它被大量用于交通预测和空气质量预测等时空预测任务。

Graph convolutional networks can be divided into two categories, spatial-based and spectral-based. Spectral-based works take graph convolution operation as denoise from graph signal. Spatial-based spatial-temporal works update information by designing different strategies to aggregate features of their neighbors. The diffusion GCN (Li et al., 2018) was proposed to process a graph with two different directions for traffic forecasting. (Wu et al., 2019) stacks graph convolution layers to predict spatial-temporal graph data with long-range temporal sequences. (Wang et al., 2021) encodes three types of relationships among stations into graphs and designs parallel GCN-based encoder-decoder architecture to generate multi-interval air quality predictions for all stations.
图卷积网络可分为两类，基于空间的和基于光谱的。基于频谱的工作将图卷积运算作为图信号的去噪。基于空间的时空作品通过设计不同的策略来聚合其相邻的特征来更新信息。扩散GCN（Li et al.， 2018）被提出用于处理具有两个不同方向的图以进行流量预测。（Wu et al.， 2019） 堆叠图卷积层以预测具有长程时间序列的时空图数据。（Wang et al.， 2021）将三种站点之间的关系编码为图，并设计了基于GCN的并行编码器-解码器架构，以生成所有站点的多间隔空气质量预测。

In light of the primary research, we construct multi-view graphs to reflect the similarities across stations from various semantic perspectives and then use multiple graph convolutions to capture different semantic spatial-temporal correlations of air quality patterns.
基于初步研究，我们构建了多视图图，从不同的语义角度反映了不同站点的相似性，然后使用多个图卷积来捕捉空气质量模式的不同语义时空相关性。

2.3. Multi-task learning 2.3. 多任务学习

Multi-task learning (MTL) is a common solution for multiple related tasks (Caruana, 1997). A single-task model exclusively learns limited features, ignoring and discarding information from other tasks. MTL, on the other hand, facilitates knowledge transfer. MTL has been used widely in many fields, such as natural language processing, spatial-temporal prediction (Wang et al., 2020; Liu et al., 2016) and so on. Wang et al. (2020) considers that crowd flow and the Origin-Destination location of flow trajectories and flow trajectory are highly correlated and affect each other, so both tasks are predicted simultaneously to achieve better performance. Liu et al. (2016) treat each station as a task to predict water quality.
多任务学习（MTL）是多个相关任务的常用解决方案（Caruana，1997）。单任务模型只学习有限的特征，忽略和丢弃来自其他任务的信息。另一方面，MTL促进了知识转移。MTL在自然语言处理、时空预测等多个领域得到了广泛的应用（Wang et al.， 2020;Liu et al.， 2016）等。Wang等人（2020）认为，人群流与流动轨迹和流动轨迹的始地-目的地位置高度相关并相互影响，因此同时预测两个任务以获得更好的性能。Liu et al. （2016） 将每个站点视为预测水质的任务。

In this paper, we utilize air quality level as an auxiliary task for air quality prediction.
本文利用空气质量水平作为空气质量预测的辅助任务。

4.1. Denoise block 4.1. 降噪块

Intuitively, pure or stable data can improve the model's prediction performance. To obtain purer data, the original data need to be denoised. Inspired by (Zhou et al., 2022), we utilize Fast Fourier Transform (FFT) and Inverse Fast Fourier Transform (IFFT) to filter out the irrelevant noises of each station. Given the input data $X^{S_i}\in {\mathrm{ℝ}}^{T\times m}$ of station S_i, we first perform FFT along the feature dimension to convert $X^{S_i}$ to the frequency domain:(1)${\mathbf{F}}^{S_i}=\mathrm{ℱ}\left(X^{S_i}\right)\in {\mathrm{ℂ}}^{T\times m}$where $\mathrm{ℱ}$ (·) indicates the one-dimensional FFT. Be aware that $F^{S_i}$ represents the spectrum of $X^{S_i}$ and is a complex tensor. The spectrum can then be modulated by multiplying a learnable filter $\mathbf{W}\in {\mathrm{ℂ}}^{T\times m}$:(2)${\tilde{\mathbf{F}}}^{S_i}=\mathbf{W}\odot {\mathbf{F}}^{S_i}$where $\odot$ denotes the element-wise multiplication. Finally, we utilize the IFFT to convert the modulated spectrum ${\tilde{\mathbf{F}}}^{S_i}$ back to the time domain and to update the representations:(3)${\tilde{\mathbf{X}}}^{S_i}\leftarrow {\mathrm{ℱ}}^{-1}\left({\tilde{\mathbf{F}}}^{S_i}\right)\in {\mathrm{ℂ}}^{T\times m}$where ${\mathrm{ℱ}}^{-1}$ represents the inverse 1D FFT, which transoms the complex tensor into a real number tensor. With FFT and IFFT, the noise in the raw data can be effectively reduced, allowing us to obtain more accurate feature representations. We also introduce residual connection and layer normalization operations to mitigate gradient vanishing and unstable training issues as:(4)${\tilde{X}}^{S_i}=\text{LayerNorm}\left({\tilde{\mathbf{X}}}^{S_i}+X^{S_i}\right)$
直观地说，纯数据或稳定数据可以提高模型的预测性能。为了获得更纯净的数据，需要对原始数据进行去噪。受（周等人，2022）的启发，我们利用快速傅里叶变换（FFT）和逆快速傅里叶变换（IFFT）来滤除每个站的不相关噪声。给定站点 S 的输入数据 $X^{S_i}\in {\mathrm{ℝ}}^{T\times m}$ ，我们首先沿特征维度执行 FFT 以转换为 $X^{S_i}$ 频域： (1)${\mathbf{F}}^{S_i}=\mathrm{ℱ}\left(X^{S_i}\right)\in {\mathrm{ℂ}}^{T\times m}$ 其中 $\mathrm{ℱ}$ （·） 表示一维 FFT。请注意，它 $F^{S_i}$ 表示 $X^{S_i}$ 的谱，是一个复张量。然后可以通过乘以可学习滤波器 $\mathbf{W}\in {\mathrm{ℂ}}^{T\times m}$ 来调制频谱： (2)${\tilde{\mathbf{F}}}^{S_i}=\mathbf{W}\odot {\mathbf{F}}^{S_i}$ 其中 $\odot$ 表示元素乘法。最后，我们利用 IFFT 将调制频谱 ${\tilde{\mathbf{F}}}^{S_i}$ 转换回时域并更新表示： (3)${\tilde{\mathbf{X}}}^{S_i}\leftarrow {\mathrm{ℱ}}^{-1}\left({\tilde{\mathbf{F}}}^{S_i}\right)\in {\mathrm{ℂ}}^{T\times m}$ 其中 ${\mathrm{ℱ}}^{-1}$ 表示逆一维 FFT，它将复张量转换为实数张量。通过FFT和IFFT，可以有效降低原始数据中的噪声，从而获得更准确的特征表示。我们还引入了残差连接和层归一化操作，以缓解梯度消失和不稳定的训练问题，例如： (4)${\tilde{X}}^{S_i}=\text{LayerNorm}\left({\tilde{\mathbf{X}}}^{S_i}+X^{S_i}\right)$

4.2. Geographical spatial view in encoder module
4.2. 编码器模块中的地理空间视图

The geographical spatial view model the relationship between geographic neighbors of stations according to the actual location. Motivated by the First Law of Geography (Tobler, 1970), i.e., “Everything is related to everything else, but near things are more related than distant things,” we discovered that the closer stations are to one another, the more similar their air quality is. In recent years, graph convolution has gained widespread application in various domains due to its ability to model the interaction between nodes using a non-European graph. Thus, we employ graph convolution to discover the geographical spatial relationship across stations.
地理空间视图根据实际位置对站点的地理邻居之间的关系进行建模。在地理第一定律（Tobler，1970）的启发下，即“一切都与其他事物有关，但近处的事物比远处的事物更相关”，我们发现站点之间的距离越近，它们的空气质量就越相似。近年来，图卷积因其能够使用非欧洲图对节点之间的交互进行建模而在各个领域获得了广泛的应用。因此，我们利用图卷积来发现跨站点的地理空间关系。

We construct a geographical spatial graph A_G = ($S,{\mathrm{ℰ}}_G$), which associates with the natural geographical neighbors to pick up geographical spatial information. In A_G, a node S_i represents a station and an edge $e_{\mathit{ij}}^G\in {\mathrm{ℰ}}_G$ represents that S_i and S_j are neighbors. Here, A_G also represents its adjacency matrix, where $A_G\left[i,j\right]=1$ if there exists an edge $e_{\mathit{ij}}^G$ in ${\mathrm{ℰ}}_G$; otherwise, it is 0. Following STGCN (Yu et al., 2018), the weighted adjacency matrix A_G can be formed as,(5)$A_{G_{\mathit{ij}}}=\left\{\begin{array}{ll}exp\left(-\frac{d_{\mathit{ij}}}{{\sigma }^2}\right),& d_{\mathit{ij}}<{\xi }_G\\ 0,& \text{otherwise}\end{array}\right.$where σ and ${\xi }_G$ are the thresholds to control the distribution and sparsity of A_G. $A_{G_{\mathit{ij}}}$ is the edge weight which is related to d_ij (the distance between station i and j).
构建地理空间图A _G = （ $S,{\mathrm{ℰ}}_G$ ），与自然地理邻域关联获取地理空间信息。在 A _G 中，节点 S 表示站点，边 $e_{\mathit{ij}}^G\in {\mathrm{ℰ}}_G$ 表示 S 和 S 是邻居。这里，A _G 也表示它的邻接矩阵，其中 $A_G\left[i,j\right]=1$ 如果存在 ${\mathrm{ℰ}}_G$ 一条边 $e_{\mathit{ij}}^G$ ;否则为 0。根据 STGCN （Yu et al.， 2018），加权邻接矩阵 A _G 可以形成为 ， (5)$A_{G_{\mathit{ij}}}=\left\{\begin{array}{ll}exp\left(-\frac{d_{\mathit{ij}}}{{\sigma }^2}\right),& d_{\mathit{ij}}<{\xi }_G\\ 0,& \text{otherwise}\end{array}\right.$ 其中 σ 和 ${\xi }_G$ 是控制 A _G 分布和稀疏性的阈值。 $A_{G_{\mathit{ij}}}$ 是与 D _ij （站点 I 和 J 之间的距离）相关的边重。

Given the denoised data ${\tilde{X}}_t\in {\mathrm{ℝ}}^{N\times m}$ at time interval t and geographical spatial graph A_G, the accumulated information is formulated as:(6)$s_t^G=\mathit{GConv}\left({\tilde{X}}_t,A_G\right)=\sum _{k=1}^K{A}_G^k{\tilde{X}}_t{W}_G^k$where ${\tilde{X}}_t$ denotes the input of the graph convolution. The number of graph convolution step k is set from 1 to K to respectively aggregate information from k-order neighbors with learnable feature weighted matrix $W_G^k\in {\mathrm{ℝ}}^{m\times h}$, where h is the dimension of a hidden state vector. Finally, for each time interval t, we obtain the $s_t^G\in {\mathrm{ℝ}}^{N\times h}$ as the geographical spatial view representation for all stations.

4.3. Logical spatial view in encoder module

The logical spatial view model the relationship between logical neighbors of stations according to the POIs and RNs. In addition to geographical spatial neighbor similarity, POIs and road networks also lead to the logical spatial similarity between stations. For example, stations in the industrial park would have equally poor air quality, while stations in the wetland would have relatively new air quality. Although they are geographically far apart, they are still similar. Thus, we construct a logical spatial graph A_L = ($S,{\mathrm{ℰ}}_S$) to obtain similarity between stations. The weighted adjacency matrix A_L can be formed as,(7)$A_{L_{\mathit{ij}}}=\left\{\begin{array}{ll}\mathit{cos}\left({\mathit{PR}}_i,{\mathit{PR}}_j\right),& \mathit{cos}\left({\mathit{PR}}_i,{\mathit{PR}}_j\right)>{\xi }_L\\ 0,& \text{otherwise}\end{array}\right.$where ${\xi }_L$ is the threshold to determine the sparsity of adjacency matrix A_L. We utilize cosine similarity to measure the similarity.

Given the denoised data ${\tilde{X}}_t$ at time interval t and logical spatial graphs A_L, the accumulated information is formulated as:(8)$s_t^L=\mathit{GConv}\left({\tilde{X}}_t,A_L\right)=\sum _{k=1}^K{A}_L^k{\tilde{X}}_t{W}_L^k$where ${\tilde{X}}_t$ denotes the input of the graph convolution. The interval of convolution k is set from 1 to K to respectively aggregate information from k-order neighbors with learnable feature weighted matrix $W_L^k\in {\mathrm{ℝ}}^{m\times h}$, where h is the dimension of a hidden state vector. Finally, for each time interval t, we obtain the $s_t^L\in {\mathrm{ℝ}}^{N\times h}$ as the logical spatial view representation for all stations.

4.4. Temporal view in encoder module

The temporal view models sequential relations. Geographical and logical spatial relationships have varying effects on the stations depending on the time of day. In order to differentiate the relative relevance of contextual states in different views at different moments, we design an attention fusion unit to improve the capacity to pick relative information from distinct views.(9)$\begin{array}{l}{\tilde{s}}_t={\alpha }_t^G{s}_t^G+{\alpha }_t^L{s}_t^L\\ {\alpha }_t^G=\frac{\mathit{exp}\left(s_t^G\right)}{exp\left(s_t^G\right)+exp\left(s_t^L\right)}\\ {\alpha }_t^L=\frac{\mathit{exp}\left(s_t^L\right)}{exp\left(s_t^G\right)+exp\left(s_t^L\right)}\end{array}$

We utilize GRU to capture underlying temporal correlations between stations. The GRU, an improved version of recurrent neural networks (RNNs), is chosen here because it addressed the vanishing gradient problem (Huang et al., 2019) and has an excellent performance in sequential modeling. The operation of the GRU can be expressed as follows:(10)$\begin{array}{l}{\mathbf{R}}_t=\sigma \left({\mathbf{W}}_R\left[{\mathbf{H}}_{t-1}\parallel {\tilde{\mathbf{s}}}_t\right]+{\mathbf{b}}_R\right)\\ {\mathbf{Z}}_t=\sigma \left({\mathbf{W}}_Z\left[{\mathbf{H}}_{t-1}\parallel {\tilde{\mathbf{s}}}_t\right]+{\mathbf{b}}_Z\right)\\ {\tilde{\mathbf{H}}}_t=tanh\left({\mathbf{W}}_{\tilde{H}}\left[{\mathbf{R}}_t\odot {\mathbf{H}}_{t-1}\parallel {\tilde{\mathbf{s}}}_t\right]+{\mathbf{b}}_{\tilde{H}}\right)\\ {\mathbf{H}}_t=\left(1-{\mathbf{Z}}_t\right)\odot {\mathbf{H}}_{t-1}+{\mathbf{Z}}_t\odot {\tilde{\mathbf{H}}}_t\end{array}$where R_t, Z_t denote reset gate and update gate at time interval t, W_R, W_Z, $W_{\tilde{H}}$, b_R, b_Z and $b_{\tilde{H}}$ are trainable parameters and $\parallel$, $\odot$ represents the concatenation operation and the Hadardmard product respectively. Hidden states $H^t\in {\mathrm{ℝ}}^{N\times h}$ are updated under the control of two gates, with the reset gates determining how much previous information is to be maintained, and the update gates ensuring that the candidate hidden states and memory are in balance. At each time step t, the extracted context states of station S_i are represented by the i-th state vector, which is denoted by $h_t^i\in {\mathrm{ℝ}}^h$.

4.5. Value prediction in decoder module

We consider value prediction to be the primary task and employ LSTM to make predictions for each station. In the value decoder, as shown in the upper right part of Fig. 2, we concatenate the output of the encoder $h_t^i\in {\mathrm{ℝ}}^h$ and the last output of the value decoder $c_{\tau -1}^{i,\mathit{value}}$ to update the hidden state with LSTM as below:(11)$c_{\tau }^{i,\mathit{value}}={\mathit{LSTM}}^v\left(h_t^i\parallel c_{\tau -1}^{i,\mathit{value}}\right)$

Note that the output of LSTM contains all effects of geographical spatial, temporal and logical spatial view. Then, we use a fully connected network for the final value prediction at future time interval τ and use the Rectified Linear Unit (ReLU) as the activation function:(12)$y_{\tau }^{i,\mathit{value}}=\mathit{ReLU}\left(W_v{c}_{\tau }^{i,\mathit{value}}+b_v\right)$where W_v and b_v are learnable parameters. ReLU is a typical activation function, and its output is in [0,1], as the air quality values are normalized. We later denormalize the prediction to get the actual air quality values.

4.6. Level prediction in decoder module

Similar to the value prediction task, we use LSTM to make multi-step predictions for each station individually using level prediction as an auxiliary task. In the level decoder, as shown in the lower right part of Fig. 2, we concatenate the output of the encoder $h_t^i$ and the last output of the value decoder $c_{\tau -1}^{i,\mathit{level}}$ to update the hidden state with LSTM as below:(13)$\begin{array}{l}{c}_{\tau }^{i,\mathit{level}}={\mathit{LSTM}}^l\left(h_t^i\parallel c_{\tau -1}^{i,\mathit{level}}\right)\end{array}$

Then, we use a fully connected network for the final level prediction at future time interval τ and use ReLU as the activation function:(14)$y_{\tau }^{i,\mathit{level}}=\mathit{ReLU}\left(W_l{c}_{\tau }^{i,\mathit{level}}+b_l\right)$where $W_l\in {\mathrm{ℝ}}^{h\times 6}$ and $b_l\in {\mathrm{ℝ}}^6$ are learnable parameters.

4.7. Joint training

The network can be trained through the backpropagation strategy and the Adam optimizer. To train our model, we aim to minimize the following loss function with two terms:(15)$\begin{array}{l}\mathrm{ℒ}\left(\theta \right)={\mathrm{ℒ}}_{\mathit{value}}+\lambda {\mathrm{ℒ}}_{\mathit{level}}\\ {\mathrm{ℒ}}_{\mathit{value}}=\mathit{MSELoss}\left({\widehat{Y}}^{\mathit{value}},Y^{\mathit{value}}\right)\\ {\mathrm{ℒ}}_{\mathit{level}}=\mathit{CrossEntropyLoss}\left({\widehat{Y}}^{\mathit{level}},Y^{\mathit{level}}\right)\end{array}$where λ is a trade-off between these two losses and θ are all learnable parameters in our model. MSELoss is Mean Squared Error for evaluating the errors between our prediction ${\widehat{Y}}^{\mathit{value}}$ and the corresponding ground truth $Y^{\mathit{value}}$ and CrossEntropyLoss is the cross-entropy loss between the predicted ${\widehat{Y}}^{\mathit{level}}$ and the ground-truth $Y^{\mathit{level}}$.

5.1. Datasets

In addition, similar to Zhang et al. (2019), we extract three time features from the timestamp of each data point: the hour of the day, the day of the week, and the month. And we process data that is null through linear interpolation. Moreover, we recruit the first 70 % as the training set, the next 20 % as the validation set, and the rest for the test set according to chronological order.

5.2. Evaluation metrics

In this paper, we use typical Root Mean Square Error (RMSE) and Mean Average Error (MAE) to evaluate different models. Smaller metrics indicate better performance. The detailed definitions of the two metrics are stated as below:(16)$\mathrm{MAE}=\frac{1}{N}\sum _{i=1}^N\left|{\widehat{y}}_i-y_i\right|$(17)$\text{RMSE}=\sqrt{\frac{1}{N}\sum _{i=1}^N{\left({\widehat{y}}_i-y_i\right)}^2}$

5.3. Baselines

We compare M2 with following seven baselines.

• •

  HA: we predict PM2.5 by the historical average method on each time step. For example, the average historical data observed from 01:00 to 12:00 is utilized to forecast the next 6 h (13:00–18:00).

• •

  ARIMA (Box and Pierce, 1970): the autoregressive integrated moving average is a well-known method for predicting time series.

• •

  SVR (Chang and Lin, 2011): support vector regression is a traditional time series via learning feature mapping functions.

• •

  Seq2Seq (Sutskever et al., 2014): we implement a two-layer sequence-to-sequence model for air quality prediction, where LSTM is chosen as the RNN implementation.

• •

  MGED-Net (Zhang et al., 2019): MGED-Net uses the structure with a multi-group encoder and single decoder. The grouped features are input to the multi-encoder, and their output is fused to decode for air quality prediction.

• •

  GWaveNet (Wu et al., 2019): graph WaveNet captures hidden spatial dependency with spatial-temporal graph modeling to make predictions.

• •

  GMAN (Zheng et al., 2020): GMAN proposes a spatial and temporal attention mechanism with gated fusion to simulate complex temporal and spatial correlations, and designs a switching attention mechanism to mitigate the effects of error propagation to improve long-term prediction performance.

• •

  ATGCN (Wang et al., 2021): ATGCN encodes multiple inter-station relationships into graphs and designs parallel GCN-based encoding and decoding modules to aggregate features from related stations using different graphs.

• •

  STPGCN (Zhao et al., 2022): STPGCN proposes an adaptive inference module method for spatiotemporal position-aware relations, which captures dynamic spatiotemporal relations by combining spatial and temporal position embedding and integrates these relations into the graph convolution layer to realize aggregation and updating of node features.

5.4. Training details & hyperparameters

5.5. Experimental results (RQ1)

5.6. Ablation study (RQ2)

To validate effectiveness of our proposed components, we evaluate the following variants:M2 w/o auxiliary-task level prediction(M2 w/o m), M2 w/o dual-view attention-based fusion module(M2 w/o a), M2 w/o logical spatial view(M2 w/o l), M2 w/o geographical spatial view (M2 w/o g):

• •

  M2 w/o auxiliary-task level prediction (M2 w/o m): in this variant, we remove the air quality level prediction part (auxiliary task) and only predict air quality value instead.

• •

  M2 w/o dual-view attention-based fusion module (M2 w/o a): in this variant, instead of using an attention-based fusion module in each time interval, we directly add a representation of two spatial views to demonstrate the effectiveness of our proposed module.

• •

  M2 w/o logical spatial view (M2 w/o l): we only consider the geographical and temporal views in this variant.

• •

  M2 w/o geographical spatial view (M2 w/o g): we only consider logical and temporal views in this variant.

Results of the ablation study are shown in Fig. 3, in which it can be seen that key designs all contribute to the improvement of the proposed model. Compared to M2 w/o dual-view attention-based fusion, M2 better demonstrates the importance of different views fusion at each interval. The model's performance degrades dramatically when the spatial views are eliminated, as evidenced by the stations' geographical and logical impact. Compared to M2 w/o auxiliary-task level prediction, M2 achieves better performance indicating that air quality level prediction is effective for air quality value prediction. It is worth noting that the improvement effect of auxiliary tasks in the Beijing dataset is more obvious than that in the Tianjin dataset. On the one hand, this may be because different datasets have different characteristics. For example, Beijing has more data records than Tianjin, and Beijing has more monitoring stations than Tianjin. On the other hand, auxiliary tasks may work better in datasets with more complex air quality and meteorological conditions. In these cases, the auxiliary tasks help capture the underlying air quality patterns better, thus improving the prediction accuracy of the main task. Conversely, in datasets with relatively simple and stable air quality and meteorological conditions, the contribution of the auxiliary tasks to improving prediction performance may be limited. The elimination of any component results in a significant rise in inaccuracy, which validates the relevance of each member.

1. Download : Download high-res image (243KB)
2. Download : Download full-size image

Fig. 3. Performance comparison of different variations.

5.7. Effects of hyper-parameters (RQ3)

This section analyzes the effects of hyper-parameter K which is the number of graph convolution steps and hyper-parameter α which is the weight of auxiliary task in the loss function. In our model, K is selected from $\left\{\mathrm{1,2,3,4}\right\}$ and α is from $\left\{\mathrm{0.1,0.01,0.001,0.0001,0.00001}\right\}$. Every hyper-parameter is finely tuned in the validation set, and an early stopping strategy is implemented. This strategy ensures that training is stopped after 15 iterations if there is no improvement in validation performance.

Fig. 4 illustrates the various performances when K is varied. The results show that K = 3 is the optimal value for the Beijing dataset, whereas K = 2 is optimal for the Tianjin dataset. The low graph convolution step degrades performance since little neighbor information is aggregated. As the graph convolution step increases, more neighbor information is captured, and the model's performance starts to improve. At a certain point, however, the model's performance begins to degrade again, possibly due to the introduction of unexpected noise.

1. Download : Download high-res image (259KB)
2. Download : Download full-size image

Fig. 4. Effect of the graph convolution layer K.

Next we consider the effect of different auxiliary task weight α on the prediction performance of our model. As shown in Fig. 5, distinct datasets correspond to specific optimal auxiliary task loss weight α. Excess weight and a deficiency will diminish the model's predictive performance. The results show that α = 0.001 is the best for the Beijing dataset, whereas α = 0.0001 is better for the Tianjin dataset. The optimal α on different datasets are different, which mainly depends on dataset characteristics, task correlation, training sample distribution and weight adjustment in the training process.

1. Download : Download high-res image (259KB)
2. Download : Download full-size image

Fig. 5. Effect of the loss weight α.

5.8. Seasonality prediction

5.9. Efficiency analysis

In our study, we aim to compare the performance and efficiency of our proposed model, M2, with three other baseline models, GMAN, ATGCN, and STPGCN, on the Beijing and Tianjin datasets. We evaluate the training time of these models and present the results in Fig. 6. The Tianjin dataset demonstrates faster overall running speeds than the Beijing dataset, mainly due to its smaller data volume. We find that the multi-layer ST-Attention blocks in GMAN significantly slow down its training time compared to the other models. Furthermore, M2 has a significantly slower speed than GMAN, is slower than ATGCN, and is comparable to STPGCN. However, M2 shows higher accuracy despite the slower speed than the other models. This indicates that our proposed model, M2, can provide a better trade-off between efficiency and accuracy for air quality prediction.

1. Download : Download high-res image (301KB)
2. Download : Download full-size image

Fig. 6. Running time efficiency.