With the increasing penetration of renewable energy and power electronics-based devices, and the hybrid operation of AC/DC power networks with heavy power transfer, the dynamics,stochastics and uncertainties of the power grid are being observed,threatening its secure operation. In order to effectively resolve security issues caused by fast variations of voltage and line flows, a reinforcement learning algorithm based on maximum entropy depth ispresented for providing online decision support in smart grid operation, which can simultaneously consider multiple control objectives.This method formulates decision derivation for grid operation as Markov decision process, which trains multi-threaded soft actor-critic and uses periodic online training mechanism to continuously improve its control performance. The developed prototype using this method has been deployed in the control center of SGCC Jiangsu electric power company, which interacts with live energy management system and learns its control policy adaptively. The well -trained agent can provide effective control actions within milliseconds to regulate voltage violation, line flow and losses.