There is a huge potential in building energy management. To solve the problem, a PID-Lagrange-SAC algorithm-based regulation method is proposed. Firstly, the problem statement of regulating building energy consumption behavior is modeled as a markov decision process（MDP）model. The state of controllable devices and external variables which introduce uncertainties are established as the state space, and the operating power of controllable devices is used as the decision variable to form the action space. Then, proper reward functions are designed to instruct the agent to learn better regulating strategies. The problem is further extended to a constrained markov decision process（CMDP）, and the Soft actor-critic algorithm is employed to train the agent while PID control and Lagrange method are applied to suppress the behavior of agents violating constraint conditions. The case study shows that the regulating strategy reduces the operating costs and carbon emissions of the building while meeting users’comfort demand, demonstrating the effectiveness and superiority of the proposed method.