Low Latency Cyberattack Detection in Smart Grids with Deep Reinforcement Learning

The algorithm is developed by using a non-linear dynamic AC system model with an extended Kalman filter (EKF) to capture power grid state transitions in real-time, while many other works in the literature use a simplified linear DC model. The DRL detection algorithm is developed by using a continuous state space deep Qnetwork (DQN) on the framework of a Markov decision process (MDP). The MDP state is designed as a sliding window of Rao-statistics of the AC dynamic state estimation residues. A new reward function is designed to allow a flexible trade-off between detection delays and detection accuracy. The delay-accuracy trade-off can be adjusted by tuning a single parameter in the reward function.

Recommended citation: Y. Li and J. Wu, “Low latency cyberattack detection in smart grids with deep reinforcement learning,” International Journal of Electrical Power & Energy Systems, vol. 142, p. 108265, 2022. https://www.sciencedirect.com/science/article/abs/pii/S0142061522002897