Intelligent Self-Evolving Neural Control Using Multi-Objective Swarm Optimization and Reinforcement Learning for Dynamic Energy Management in PV-Driven Smart Cities
DOI:
https://doi.org/10.65150/EP-gjetr/V1E3/2025-03Keywords:
self-evolving neural network, reinforcement learning, multi-objective particle swarm optimization, photovoltaic smart grid, dynamic energy management, urban sustainability.Abstract
In this paper, an adaptive self-evolving neural control strategy based on the integration of Reinforcement Learning (RL) and Multi-Objective Particle Swarm Optimization (MOPSO) is proposed for dynamic energy management in Photovoltaic PV-based smart cities. The motivation behind this system is to cope with complexity of modern energy networks, uncertain large-scale solar generation and stochastic urban demand in combination with dynamic grid conditions. We adopt a self-evolving neural network (SENN), being the adaptive decision-making core, that dynamically tunes its parameters by reward-driven learning and Pareto-based optimization. The RL agent continuously controls power flow, energy storage distribution and load scheduling and MOPSO adjusts the multi-criterion objective indices including PQ improvement, voltage stability index, total cost of electrical system and carbon dioxide emission reduction. A hierarchical control architecture is proposed to integrate on-line operation optimisation of the microgrid while considering long-term city-scale prediction. The potential of the framework is demonstrated on a realistic urban PV-grid model including residential, commercial, and EV charging nodes with dynamic weather and demand. Simulation results show that the proposed SENN–MOPSO–RL controller outperforms the traditional Model Predictive Control (MPC), Proportional-Integral (PI) and Fuzzy Logic controllers. In particular, the total power loss is mitigated by 18.6%, voltage deviations by 27.4%, and the REs consumption improved about 22.9% respectively. Additionally, the controller kept grid stability for partial shading, demand surge and communication time delay, with good robustness and self-adaptation. The results validate that integrating swarm intelligence and reinforcement-based neural learning can result in a scalable and robust control framework for autonomous energy systems (AESs) found in PV-dominate smart cities. Also, hardware-in-the-loop validation and multi-agent interaction involving inter-microgrid collaboration with peer-to-peer trading will be investigated in the future.
References
1) M. A. Hossain et al., “Power Quality Improvement in PV Integrated Grids: A Review,” Renewable and Sustainable Energy Reviews, vol. 153, pp. 111–128, 2021.
2) S. A. Arefifar and Y. Mohamed, “Energy Management in Smart Cities: Challenges and Opportunities,” IEEE Access, vol. 9, pp. 116–134, 2021.
3) H. S. Patel and K. Bhattacharya, “Hybrid Energy Management Systems for Smart Urban Networks,” IEEE Transactions on Smart Grid, vol. 12, no. 6, pp. 4821–4833, 2021.
4) M. F. Hussain et al., “AI-Driven Optimization in Renewable Energy Systems,” Applied Energy, vol. 335, 120770, 2023.
5) R. Sutton and A. Barto, Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018.
6) N. Xu et al., “A Review of Smart Grid Evolution and Reinforcement Learning: Applications and Challenges,” Energies, vol. 18, no. 7, 1837, 2025.
7) A. V. Waghmare et al., “A Systematic Review of Reinforcement Learning in Energy Management for Microgrids,” SN Applied Sciences, 2025.
8) A. Pigott et al., “GridLearn: Multi-Agent Reinforcement Learning for Grid-Aware Building Energy Management,” IEEE Access, vol. 11, pp. 89350–89364, 2023.
9) A. Shojaeighadikolaei et al., “Distributed Energy Management Using Deep Reinforcement Learning,” Energy AI, vol. 14, 100325, 2022.
10) Z. Zhang et al., “Hybrid Reinforcement Learning and Evolutionary Optimization for Renewable Grid Control,” IEEE Transactions on Sustainable Energy, vol. 14, no. 3, pp. 1759–1771, 2023.
11) Y. Zhang and H. Y. Wang, “Neural-Network-Based Optimal Control in Smart Grids,” Electric Power Systems Research, vol. 209, 107901, 2022.
12) K. Parhi and S. Jha, “Self-Evolving Neural Architectures: A Survey,” Neurocomputing, vol. 550, pp. 127–143, 2023.
13) S. M. Thajeel and D. Ç. Atilla, “Reinforcement Neural Network-Based Grid-Integrated PV Control and Battery Management,” Energies, vol. 18, no. 3, 637, 2025.
14) D. Zhang et al., “Evolving Connectionist Systems for Fault Detection in Smart Grids,” IEEE Access, vol. 10, pp. 119305–119318, 2022.
15) J. Kennedy and R. Eberhart, “Particle Swarm Optimization,” Proc. IEEE Int. Conf. Neural Networks, 1995.
16) A. Kerboua, “PSO-Based Optimization for Microgrid Energy Cost Minimization,” Int. J. Smart Grid and Clean Energy, vol. 9, pp. 62–70, 2020.
17) Y. Huang et al., “Multi-Objective Particle Swarm Optimization for Household Microgrids,” Frontiers in Energy Research, vol. 12, 1354869, 2024.
18) A. Rochd et al., “Swarm Intelligence-Driven Multi-Objective Energy Trading Optimization,” Sustainable Energy Technologies and Assessments, vol. 74, 103326, 2025.
19) S. Aslam et al., “Optimization Algorithms in Smart Grids: A Systematic Review,” arXiv:2301.07512, 2023.
20) T. R. Razak et al., “Artificial Intelligence in Renewable Energy: A Comprehensive Review,” Sustainability, vol. 17, no. 5, 1123, 2025.
21) A. Alkuhayli et al., “Designing a Multi-Objective Energy Management System in Multiple Microgrids,” Energy Reports, vol. 10, pp. 1253–1266, 2024.
22) Y. Li et al., “Advanced Intelligent Optimization Algorithms for Multi-Objective Optimal Power Flow,” Energy AI, vol. 20, 100678, 2024.
23) F. Long et al., “Evolutionary Multi-Objective Optimization in Smart Grid Dispatch,” Electrica Journal, vol. 72, no. 4, pp. 293–304, 2022.
24) M. S. Hossain et al., “Harmonic Reduction Using Adaptive AI-Controllers,” IEEE Transactions on Power Electronics, vol. 38, no. 8, pp. 9567–9579, 2023.
25) K. Paul et al., “Quantum PSO for Sustainable Microgrid Optimization,” Scientific Reports, vol. 15, 5843, 2025.
26) J. C. Liao and L. Lin, “Federated Multi-Agent Reinforcement Learning for Smart Grids,” IEEE Access, vol. 12, pp. 14345–14357, 2024.
27) M. Rezazadeh et al., “Federated Deep RL for Distributed DER Control,” arXiv:2211.03430, 2022.
28) G. Nunes et al., “Self-Evolving Controllers: A Review of Lifelong Learning in Engineering Systems,” Engineering Applications of Artificial Intelligence, vol. 137, 107876, 2025.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Adel Elgammal (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
