Study on massive data incremental processing method based on distributed parallel computing framework
CSTR:
Author:
Affiliation:

(1. Lishui Power Supply Company, State Grid Zhejiang Electric Power Co., Ltd., Lishui 323000, China;2. Pinghu Power Supply Company, State Grid Zhejiang Electric Power Co., Ltd., Pinghu 315300, China;3. Zhejiang Huayun Information Technology Co., Ltd,. Hangzhou 310000, China;4. Cixi Power Supply Company, State Grid Zhejiang Electric Power Co., Ltd., Cixi 315300, China)

Clc Number:

TM769;TK018

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    With the arrival of the era of big data, how to improve the efficiency of mass data processing is more important in the application of enterprise data warehouse. Based on the analysis of the current situation of data sharing application platform of electric, a data incremental processing method based on a distributed parallel computing framework is proposed, and the practice of massive incremental data processing is carried out based on this method. Verification results show that the problem of low data processing efficiency is solved, and the accuracy of data processing in enterprise data warehouse is enhanced.

    Reference
    Related
    Cited by
Get Citation

YANG Shiwang, ZHONG Yuxing, YANG Chunchen, SUN Mingduo, SHI Huanjian. Study on massive data incremental processing method based on distributed parallel computing framework[J].,2019,21(S1):04-06.

Copy
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:July 24,2019
  • Revised:August 15,2019
  • Adopted:
  • Online: May 15,2023
  • Published:
Article QR Code