Design and Implementation of Ground Meteorological Data Storage Alarm System for Other Provinces
外省地面气象数据存储报警系统设计与实施

Keywords: Meteorological Data; Data Transmission; Monitoring; Alarm; Data Services
关键词：气象数据;数据传输;监测;报警;数据服务

Due to the rapid development of the meteorological industry, there is a wide variety of meteorological products, and the timely transmission of massive meteorological data is highly demanded. Currently, ground minute data is updated every minute, and the exchange of complex data types among different systems is frequent, placing heavier tasks on operational and maintenance personnel. The Information Center of the Gansu Provincial
由于气象行业的快速发展，气象产品种类繁多，对海量气象数据的及时传输提出了很高的要求。目前，地面分钟数据每分钟更新一次，不同系统之间复杂数据类型的交换频繁，给运维人员带来了更繁重的任务。甘肃省信息中心

Meteorological Bureau receives ground meteorological data from four other provinces in the Northwestern region (Qinghai, Ningxia, and Shaanxi) through satellite broadcasts. However, issues such as computer software and hardware failures, network problems, etc., can lead to data nonexistence and data loss, which adversely affect the quality of services provided to users. Furthermore, the Northwestern region has a large amount of meteorological data, with only approximately 282 automatic stations and around 4,287 regional stations. Monitoring and tallying missing station information, using traditional manual methods, would be laborious and cumbersome. Moreover, it cannot promptly and proactively detect anomalies in the data storage process[1], leading to delayed detection and repair of faulty points, severely affecting the timeliness of ground meteorological data storage and the quality of data transmission observation. With the rapid advancement of the automation level in comprehensive meteorological observation systems, meteorological forecasting and services have increasingly higher requirements for the timeliness of data transmission processes[2]. Meteorological observation serves as a crucial foundation for accurate forecasting and precise services. It acts as the forefront of the first line of defense for meteorological disaster prevention and reduction, making it urgent to strengthen the automatic monitoring and early warning capabilities during the process of ground meteorological data transmission and processing[3]. To timely and accurately monitor meteorological data from other provinces and regions and effectively alleviate the workload pressure on operational and maintenance personnel, Xu Ning et al. [4] proposed a monitoring and transmission scheme for ground meteorological minute data.
气象局通过卫星广播接收西北地区其他四个省份（青海、宁夏和陕西）的地面气象数据。但是，计算机软硬件故障、网络问题等问题会导致数据不存在和数据丢失，从而对提供给用户的服务质量产生不利影响。此外，西北地区拥有大量的气象数据，只有大约 282 个自动站和大约 4,287 个区域站。使用传统的手动方法监控和统计缺失的站点信息将既费力又繁琐。此外，它无法及时主动地检测数据存储过程中的异常[1]，导致故障点的检测和修复延迟，严重影响地面气象数据存储的及时性和数据传输观测的质量。随着综合气象观测系统自动化水平的快速推进，气象预报和服务对数据传输过程的及时性提出了越来越高的要求[2]。气象观测是准确预报和精确服务的重要基础。它充当气象防灾减灾第一道防线的第一道防线，迫切需要加强地面气象数据传输和处理过程中的自动监测和预警能力[3]。为了及时准确地监测外省区的气象数据，有效缓解运维人员的工作量压力，徐宁等人。 [4] 提出了地面气象微小数据的监测和传输方案。

It mainly achieves the monitoring of BUFR data from ground meteorological stations and automatic retransmission of missing data. Xu Haobin et al. [5] designed a regional station data monitoring system based on practical business needs and sent missing report information to operational and maintenance personnel via email. They introduced the design of a comprehensive meteorological data flow monitoring system and proposed a method for rapid localization and monitoring of data updates. Wu Guiyi et al. [6] elaborated on the design principles and implementation methods of the comprehensive meteorological data monitoring algorithm and the construction of the data monitoring chain. Wang Shuofei et al. [7] implemented automated monitoring of the operating status of communication equipment and the transmission of meteorological data, with alarm functions enabled through a messaging platform. Currently, there is no unified real-time monitoring of the meteorological data transmission status in the four other provinces of the Northwestern region. However, the observed meteorological data from these provinces are crucial for studying the climate in the Northwestern region, such as analyzing trends in warmth and humidity and predicting future climate changes [8]. In order to further strengthen the monitoring of meteorological data storage in the Northwestern region, this study proposes a python-based data storage alarm system for data from other provinces and regions. It enables real-time monitoring of missing data from national automatic stations and regional stations in the Northwestern region. Maintenance personnel can promptly detect and locate data gaps through WeChat alarm notifications. This system ensures the orderly and timely storage of ground meteorological data from the Northwestern region into the database, thereby providing users with higher-quality data services.
主要实现对地面气象站 BUFR 数据的监测和缺失数据的自动重传。徐浩斌等[5]根据实际业务需求设计了区域站数据监测系统，并通过电子邮件将缺失的报告信息发送给运维人员。他们介绍了综合气象数据流监测系统的设计，并提出了一种快速定位和监测数据更新的方法。吴贵义等[6]详细阐述了综合气象数据监测算法的设计原则和实现方法以及数据监测链的构建。Wang Shuofei et al. [7] 实现了通信设备运行状态的自动化监测和气象数据传输，并通过消息平台启用警报功能。目前，西北地区其他四个省份还没有统一的气象数据传输状态实时监测。然而，从这些省份观测到的气象数据对于研究西北地区的气候至关重要，例如分析温暖和湿度的趋势以及预测未来的气候变化 [8]。为进一步加强西北地区气象数据存储监测，本研究提出了一种基于python的外省区数据存储告警系统。它能够实时监控来自西北地区国家自动站和区域站的缺失数据。维护人员可以通过微信告警通知及时检测和定位数据缺口。 该系统保证了西北地区地面气象数据有序、及时地存储到数据库中，从而为用户提供更高质量的数据服务。

The Meteorological Big Data Cloud Platform, known as “Tianqing”, is the most critical foundational platform for meteorological information technology. It is primarily built around big data resources and provides massive data storage, data sharing, and integrated “data-computation fusion” platform services. It integrates
气象大数据云平台，简称“天青”，是气象信息技术最关键的基础设施平台。它主要围绕大数据资源构建，提供海量数据存储、数据共享和一体化的“数算融合”平台服务。它集成了
various data resources and processing workflows of meteorological departments, eliminating data silos and information islands. By achieving the convergence and interconnection of meteorological business data resources and facilitating their coconstruction and sharing, Tianqing plays a vital role in supporting the accuracy of meteorological forecasting and the refinement of meteorological public services through its data capabilities [9]. In our province, meteorological data from the four other provinces in the Northwestern region is downloaded via satellite broadcasting. Tianqing distributes the broadcasted and downloaded meteorological data to the downstream Service-Oriented Data Storage System (SOD) through the China Telecommunication System (CTS) and Data Processing and Conditioning System (DPC) for decoding and storage processing. The Meteorological Unified Service Interface Community (MUSIC) provides data application services to users, and business units utilize the MUSIC service interface to access meteorological data. The MUSIC, based on the Meteorological Big Data Cloud Platform, is designed for meteorological operations and scientific research. It offers a nationally unified, standard, and comprehensive data access service and application programming interfaces (APIs), providing authoritative data access services for various levels of application systems. However, throughout the entire data transmission process, data loss can occur due to communication node failures, software freezes, and other malfunctions, making it impossible for users to access such meteorological data through the interface. Currently, our province has conducted in-depth research on the overall business processing flow and key components and technologies of the big data cloud platform. Real-time monitoring and data collection of the operational status of each processing component have been implemented, and the collected information has been transformed into alarm notifications. Nevertheless, the existing monitoring and alarm platform does not provide unified real-time monitoring of the ground meteorological data transmission status in the four other provinces of the
气象部门的各种数据资源和处理工作流程，消除数据孤岛和信息孤岛。通过实现气象业务数据资源的汇聚互联，促进其共建共享，天青通过其数据能力，在支撑气象预报的准确性和气象公共服务的精细化方面发挥着至关重要的作用 [9]。在我省，西北地区其他四个省份的气象数据是通过卫星广播下载的。天晴通过中国电信系统 （CTS） 和数据处理与调节系统 （DPC） 将广播和下载的气象数据分发到下游服务化数据存储系统 （SOD） 进行解码和存储处理。气象统一服务接口社区 （MUSIC） 为用户提供数据应用服务，业务部门利用 MUSIC 服务接口访问气象数据。MUSIC 基于气象大数据云平台，专为气象作业和科学研究而设计。提供全国统一、标准化、全面的数据接入服务和应用程序编程接口（API），为各级应用系统提供权威的数据接入服务。然而，在整个数据传输过程中，由于通信节点故障、软件卡顿和其他故障，可能会导致数据丢失，使用户无法通过接口访问此类气象数据。目前，我省对大数据云平台的整体业务处理流程和关键组件和技术进行了深入研究。 实现了对各处理组件运行状态的实时监控和数据采集，并将采集到的信息转化为告警通知。然而，现有的监测和报警平台并不能提供对其他四个省份的地面气象数据传输状态的统一实时监测。

Northwestern region. Therefore, it is crucial to organize and monitor the meteorological data of the Northwestern region in real time, as it can effectively enhance the overall understanding of the transmission status of ground meteorological data from the four other provinces in the Northwestern region [10].
西北地区。因此，实时组织和监测西北地区气象数据至关重要，因为它可以有效增强对西北地区其他四个省份地面气象数据传输状态的整体理解 [10]。
Based on the different functions of the system, it can be divided into three main functional modules: real-time monitoring, WeChat alarms, and logging. The general workflow of the system is depicted in Figure 1. Initially, the entire transmission process of ground meteorological data in the Northwestern region was organized, including the processes of broadcast distribution, platform downloads, data distribution, decoding, and storage. This helped establish the complete flow of data from distribution to user access through the interface. The catalog of meteorological data from national and regional automatic stations in the Northwestern region, which is disseminated through satellite broadcasting, was compiled. This catalog includes station numbers, station names, as well as information on the respective cities and counties to which the stations belong. Information on corresponding databases and data table names where such meteorological data is stored was also organized. Next, a program was developed to retrieve this type of data in real-time, validate the real-time storage status of the meteorological data, and send alarms based on different storage status information. Additionally, the existing station information table was dynamically updated in real-time based on the validation results, ensuring accurate updates of station network information. By continuously monitoring the storage status of ground meteorological data in the Northwestern region, the program generates corresponding alarms and sends them to the enterprise WeChat platform in case of any abnormal data storage situations. Timely awareness of the data storage status in the Northwestern region facilitates quick problem identification by maintenance personnel and timely resolution of ground meteorological data loss issues. This helps improve data transmission quality, reduce the workload of operation and maintenance
根据系统功能的不同，可分为三个主要功能模块：实时监控、微信告警、日志记录。该系统的一般工作流程如图 1 所示。最初，西北地区地面气象数据的整个传输过程是组织的，包括广播分发、平台下载、数据分发、解码和存储的过程。这有助于通过界面建立从分发到用户访问的完整数据流。编制了来自西北地区国家和地区自动站的气象数据目录，该目录通过卫星广播传播。该目录包括车站编号、车站名称以及车站所属的相应城市和县的信息。还组织了有关存储此类气象数据的相应数据库和数据表名称的信息。接下来，开发了一个程序来实时检索此类数据，验证气象数据的实时存储状态，并根据不同的存储状态信息发送警报。同时，根据验证结果实时动态更新已有的站信息表，确保站网信息的准确更新。通过持续监测西北地区地面气象数据的存储状态，程序生成相应的告警，并在出现异常数据存储情况时发送到企业微信平台。 及时了解西北地区的数据存储状态，有助于维护人员快速识别问题，及时解决地面气象数据丢失问题。这有助于提高数据传输质量，减少运维工作量
personnel, and establish a solid foundation for providing higher-quality data services.
人员，为提供更高质量的数据服务奠定了坚实的基础。

Figure 1. System Workflow
图 1.系统工作流程

Real-time monitoring is the foundation of normal operation in meteorological service businesses. It is also an essential tool for detecting faults in meteorological services and performance issues with meteorological equipment. The real-time monitoring program is developed using the Python language. It calls the service interface every hour to retrieve relevant meteorological data from the four other provinces in the Northwestern region and saves the output. By modifying the service interface parameter, interfaceID, it is possible to retrieve station network information within a specified time period. For example, setting ‘timeRange’ to [20230924000000, 20230925010000] allows retrieval of station network information within that time range. The output results are then processed and
实时监控是气象服务业务正常运行的基础。它也是检测气象服务故障和气象设备性能问题的重要工具。实时监控程序是使用 Python 语言开发的。它每小时调用一次服务接口，以检索西北地区其他四个省份的相关气象数据并保存输出。通过修改服务接口参数 interfaceID，可以在指定时间段内检索工作站网络信息。例如，将 'timeRange' 设置为 [20230924000000， 20230925010000] 可以检索该时间范围内的 station 网络信息。然后处理输出结果并