Study on massive data incremental processing method based on distributed parallel computing framework
CSTR:
Author:
Affiliation:

(1. Lishui Power Supply Company, State Grid Zhejiang Electric Power Co., Ltd., Lishui 323000, China;2. Pinghu Power Supply Company, State Grid Zhejiang Electric Power Co., Ltd., Pinghu 315300, China;3. Zhejiang Huayun Information Technology Co., Ltd,. Hangzhou 310000, China;4. Cixi Power Supply Company, State Grid Zhejiang Electric Power Co., Ltd., Cixi 315300, China)

Clc Number:

TM769;TK018

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    With the arrival of the era of big data, how to improve the efficiency of mass data processing is more important in the application of enterprise data warehouse. Based on the analysis of the current situation of data sharing application platform of electric, a data incremental processing method based on a distributed parallel computing framework is proposed, and the practice of massive incremental data processing is carried out based on this method. Verification results show that the problem of low data processing efficiency is solved, and the accuracy of data processing in enterprise data warehouse is enhanced.

    Reference
    Related
    Cited by
Get Citation

杨世旺,钟雨星,杨春晨,孙明铎,施焕健.基于分布式并行计算框架的海量数据增量处理方法研究[J].电力需求侧管理英文版,2019,21(S1):04-06.

Copy
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:July 24,2019
  • Revised:August 15,2019
  • Adopted:
  • Online: May 15,2023
  • Published:
Article QR Code