Please use this identifier to cite or link to this item: http://theses.ncl.ac.uk/jspui/handle/10443/4933
Title: Reinforcement Learning based Adaptive Model Predictive Power Pinch Analysis Systems Level Energy Management Approach to Uncertainty in Isolated Hybrid Energy Storage Systems
Authors: Nyong-Bassey, Bassey Etim
Issue Date: 2020
Publisher: Newcastle University
Abstract: Hybrid energy storage systems (HESS) involves the integration of multiple energy storage technologies with different complementary characteristics which are significantly advantageous compared to a single energy storage system, and can greatly improve the reliability of intermittent renewable energy sources (RES). Aside from the advantages HESS offer, the control and coordination of the multiple energy storages and the vital elements of the system via an optimised energy management strategy (EMS) involves increased computational time. Nevertheless, a systems-level graphical EMS based on Power Pinch Analysis (PoPA) which is a low burden computational tool was recently proposed for HESS. In this respect, the EMS which effectively resolved deficit and excess energy objectives was effected via the graphical PoPA tool, the power grand composite curve (PGCC). PGCC is basically a plot of integrated energy demands and sources in the system as a function of time. Although of proven success, accounting for uncertainty with PoPA is a cogent research question due to the assumption of an ideal day ahead (DA) generation and load profiles forecast. Therefore, the proposition of several graphical and reinforcement learning based ‘adaptive’ PoPA EMSs in order to address the issue of uncertainty with PoPA, has been the major contribution of this thesis. Firstly, to counteract the combined effect of uncertainty with PoPA, an Adaptive PoPA EMS for a standalone HESS has been proposed. In the Adaptive PoPA, the PGCC was implemented within a receding horizon model predictive framework with the current output state of the energy storage (in this case the battery) used as control feedback to derive an updated sequence of EMS, inferred via PGCC shaping. Additionally, during the control and operation of the HESS, re-computation of the PGCC only occurs if a forecast uncertainty occurs such that the error between the real and estimated battery’s state of charge becomes greater than an arbitrarily chosen threshold value of 5%. Secondly a Kalman filter for the optimal estimation of uncertainty distributed as a normal Gaussian is integrated into the Adaptive PoPA in order to recursively predict the State of Charge of the battery based on the likelihood of uncertainty. Thus, the Kalman filter Adaptive PoPA by anticipating the effect of uncertainty offers an improved approach to the Adaptive PoPA particularly when the uncertainty is of a Gaussian distribution. The algorithm is therefore more sophisticated than the Adaptive PoPA but nevertheless computationally efficient and offers a preventive measure as an improvement. Furthermore, Tabular Dyna Q-learning algorithm, a subset of reinforcement learning which employs a learning agent to solve a discrete Markov Decision Process by maximising an expected reward in accordance with the Bellman optimality, is integrated within the Power Pinch Analysis. Thereafter, a deep neural network is used to approximate the Q-Learning Table. These aforementioned methods which have been highlighted in order of computational time can be deployed with only a minimal level of historical data requirements such as the average load profile or base load data and solar irradiance forecast to produce a deterministic solution. Nevertheless, this thesis proposed a probabilistic adaptive PoPA strategy based on a (recursive least square) Monte Carlo simulation chance constrained framework, in the event where there is sufficient amount of historical data such as the probability distribution of the uncertain model parameters. The probabilistic approach is no doubt more computationally intensive than the deterministic methods presented though it proffers a much more realistic solution to the problem of uncertainty. In order to enhance the probabilistic adaptive PoPA, an actor-critic deep neural network reinforcement learning agent is incorporated. The six methods are evaluated against the DA PoPA on an actual isolated HESS microgrid built in Greece with respect to the violation of the energy storage operating constraints and plummeting carbon emission footprint.
Description: Ph. D. Thesis
URI: http://theses.ncl.ac.uk/jspui/handle/10443/4933
Appears in Collections:School of Engineering

Files in This Item:
File Description SizeFormat 
Nyong-Bassey B E 2020.pdfThesis17.25 MBAdobe PDFView/Open
dspacelicence.pdfLicence43.82 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.