A Multi-Layer Reinforcement Learning and Genetic Algorithm Control Scheme for Grid Stability and Dispatch Control in PV–Fuel Cell Dominated Distribution Microgrids
DOI:
https://doi.org/10.65150/EP-gjetr/V1E4/2025-06Keywords:
Reinforcement Learning (RL); Genetic Algorithm Multi-Objective Optimization (MOGA); PV–Fuel Cell Microgrids; Dispatch Optimization; Grid Stability; Voltage and Frequency Regulation;; Hydrogen Energy Systems; Smart Microgrid Management.Abstract
The recent proliferation of photovoltaic (PV) based generation and hydrogen-based fuel cell (FC) systems in smart distribution networks pose great operational challenges, such as intermittency, voltage instability, frequency deviations, and suboptimal dispatch efficiency. Classical deterministic or rule-based control algorithms are not capable of effectively dealing with the nonlinear, stochastic and highly interconnected behaviors of PV–FC based MG systems. In this paper, a new multi-layer intelligent control framework, which combines RL for real-time scenario adaptive decision-making and GA to achieve the multi-objective optimization on energy dispatch and grid stability indices is presented. The upper RL is capable of updating the optimal control policies based on an online interaction with the microgrid environment, considering the complexities such as irradiance variations, uncertainties in loads, hydrogen storage dynamics and FC ramping constraints. There, the lower GA layer coordinates PV generation with FC output and battery state of charge trajectories, subject to optimal grid-supportive actions such as reactive power compensation and harmonic mitigation.
An extensive MATLAB/Simulink-Python co-simulation framework was implemented to study the proposed topology in the presence of fluctuating renewable profiles, sudden load changes, FC fuel starvation operation modes and islanded/connected configurations. Simulation results show that the RL–GA hybrid controller is in much better performance compared with conventional PI, model predictive control (MPC) and sole RL approaches. Significant results were found such as a reduction in the variation of 35–48%, an improvement in frequency stability of 28%, up to 22% reduction is hydrogen consumption, and an increase in dispatch efficiency ranging from 30 to 55% subject to operational constraints and extension of FC Stack life. Furthermore, the OC system is robust to prediction errors, and to communication delays as well as component uncertainties. Results demonstrate the promising prospects of scalable hierarchical AI-based control and operation, for stable, efficient, and autonomous performance in renewable-driven microgrids. This study paves the way for the development of intelligent controllers to use in distributed energy systems with potential high penetration levels of renewables and hydrogen-integrated configurations.
References
1) S. Abdelshafy, A. Eldessouky, and M. Mohamed, “Hybrid PV–Fuel Cell Microgrid: Optimal Energy Management and Grid Interaction under Uncertainty,” Energy Convers. Manag., vol. 235, pp. 1–15, 2021.
2) M. A. Khan et al., “Hydrogen-Based Hybrid Microgrids: A Comprehensive Review,” Renew. Sustain. Energy Rev., vol. 146, 111177, 2021.
3) M. Zamani-Dehkordi et al., “Robust Control of PV–Fuel Cell Systems for Distribution Networks,” IEEE Trans. Smart Grid, vol. 12, no. 4, pp. 3104–3115, 2021.
4) M. Nazi, P. Mandal, and A. K. Srivastava, “Hydrogen-Assisted Microgrids for Future Power Systems,” IEEE Trans. Ind. Appl., vol. 59, no. 2, pp. 2148–2159, 2023.
5) R. Alasseri et al., “Optimal Sizing and Energy Management of PV–Electrolyzer–Fuel Cell Microgrids,” Energy, vol. 239, 122337, 2022.
6) Y. Zhang et al., “Deep Reinforcement Learning-Based Frequency Regulation in Islanded Microgrids,” IEEE Trans. Smart Grid, vol. 12, no. 3, pp. 2091–2103, 2021.
7) X. Han, J. Zhao, and H. Liang, “A Multi-Agent Deep RL Framework for Distributed Voltage Control,” IEEE Trans. Smart Grid, vol. 13, no. 1, pp. 504–516, 2022.
8) H. Liu et al., “Hierarchical Reinforcement Learning for Multi-Timescale Microgrid Management,” Appl. Energy, vol. 328, 120054, 2022.
9) T. Kim and W. Saad, “Q-Learning and Actor–Critic Methods for Grid Stabilization under High Renewable Penetration,” IEEE Trans. Power Syst., vol. 36, no. 5, pp. 4420–4431, 2021.
10) M. Sadeghi, A. Safdarian, and M. Fotuhi-Firuzabad, “AI-Based Microgrid Energy Management: A Reinforcement Learning Perspective,” IEEE Syst. J., vol. 17, no. 1, pp. 617–628, 2023.
11) X. Yu, C. Zhang, and H. Sun, “Multi-Objective Genetic Algorithm Optimization for Microgrid Dispatch with Renewable Uncertainties,” IEEE Trans. Sustain. Energy, vol. 12, no. 4, pp. 2024–2036, 2021.
12) M. Shams, A. Kazemi, and M. Ghasemi, “Improved NSGA-II for Multi-Objective Microgrid Scheduling,” Energy, vol. 235, 121372, 2021.
13) S. Chandra et al., “GA-Based Optimization of Hybrid Renewable Systems Integrating Hydrogen Storage,” Int. J. Hydrogen Energy, vol. 47, no. 67, pp. 28645–28659, 2022.
14) A. A. Al-Saedi et al., “GA–RL Hybrid Optimization for Smart Grid Energy Scheduling,” Energy AI, vol. 7, 100125, 2022.
15) L. Li et al., “Evolutionary Optimization for PV–FC Hybrid Systems with Degradation Constraints,” Energy Convers. Manag., vol. 270, 116252, 2022.
16) N. Zamel et al., “Advancements in Proton Exchange Membrane Fuel Cell Models,” Renew. Sustain. Energy Rev., vol. 146, 111185, 2021.
17) M. Verma et al., “Dynamic Modeling of Electrolyzer–Fuel Cell Systems in Microgrids,” IEEE Access, vol. 9, pp. 106543–106556, 2021.
18) M. Bagheri, S. M. Muyeen, “Optimal Hydrogen Production Scheduling in Smart Grids,” Int. J. Hydrogen Energy, vol. 47, no. 76, pp. 32543–32558, 2022.
19) A. Mehrizi-Sani et al., “Stability Assessment in Modern Hybrid Microgrids,” IEEE Trans. Power Syst., vol. 36, no. 5, pp. 4471–4483, 2021.
20) G. Lou, Y. Xu, “Adaptive Control for Renewable-Rich Microgrids,” IEEE Trans. Smart Grid, vol. 12, no. 4, pp. 3131–3143, 2021.
21) N. Gupta and M. Singh, “Advanced EMS for PV–Hydrogen Microgrids,” Renew. Energy, vol. 185, pp. 991–1004, 2022.
22) M. Salim and R. Ali, “Learning-Based Stability Enhancement for Future Distribution Systems,” IEEE Access, vol. 10, pp. 12890–12905, 2022.
23) L. Peng et al., “Deep Learning Enabled Dispatch Optimization for Smart Microgrids,” Appl. Energy, vol. 336, 120808, 2023.
24) X. Wang et al., “Hybrid Genetic Algorithm and Deep RL for Coordinated Control of Renewable Microgrids,” Energy Convers. Manag., vol. 268, 116016, 2022.
25) A. Ullah et al., “Deep Reinforcement Learning for Net-Zero Microgrids,” IEEE Trans. Smart Grid, vol. 15, no. 1, pp. 231–245, 2025.
26) T. Nguyen and M. Muyeen, "Adaptive VSM using reinforcement learning," IEEE Access, vol. 8, pp. 129580–129590, 2020.
27) Z. Yang and L. Xu, "Reinforcement learning for microgrid frequency control," IEEE Trans. Smart Grid, vol. 10, no. 2, pp. 1972–1982, 2019.
28) H. Zhang, S. Zhao, "Deep reinforcement learning for energy management," IEEE Internet Things J., vol. 6, no. 6, pp. 10637–10651, 2019.
29) S. Chakraborty, M. Rahman, and A. Saha, "Hybrid renewable energy-based microgrid stability analysis," IEEE Trans. Sustain. Energy, vol. 11, no. 1, pp. 62–73, 2020.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Adel Elgammal (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
