Chapter 1: Background, Problem Statement and Research Aims
Background
The Australian Federal and State Governmental Policies increasingly recognise the need to grow the capacity for renewable energy generation in response to climate change, energy security and regional economic development [1]. The mix of energy generation in Australia is changing, a number of coal fired generators are reaching the end of their period for operation and there are indications that new investment in coal fired generation is less likely than investment in renewable energy generation capacity [2]. Energy generation contributed approximately 32.9% of Australia’s national annual emissions in 2019, a decrease of 2.9% due to increases in energy supplied from solar generation [3]. Renewable energy generation has great potential to significantly contribute to the Australian Government’s commitment towards the Paris agreement in reducing greenhouse gas emissions by 26 to 28 percent on 2005 levels by 2030 [2]. In Queensland, a number of large scale renewable energy projects have been initiated with support from the Queensland Government’s “Powering Queensland Plan” which aims for 50% renewable energy by 2030 [4]. The diverse mixture of renewable energy sources within Queensland is a deliberate strategy to ensure security of energy supply and competitive market pricing. Upon completion, the plan is estimated to deliver 1200 MW to the state energy network [1]. Nationally the increasing capacity for renewable energy generation (with a 22% increase in solar generation in 2019) plays an important role in decreasing Carbon dioxide (\(CO_2\)) emissions produced in the National Energy Market [3].
Changes in solar availability affect the capacity of energy production for major projects. Identifying potential shifts in seasonal trends of solar availability will assist in projecting annual capacity for such projects over longer time scales and identify regional variations of solar availability under future climate warming scenarios. An assessment of the longer-term impacts of climate change on the availability of renewable energy resources can identify stability of resources to target best investments in large scale renewable energy projects.
This Master of Science Dissertation is focused on the development of CNN-LSTM-based local and universal models (i.e., CNN-LSTM\(_L\) and CNN-LSTM\(_U\)) to downscale an ensemble of three GCM model simulations against ground station observations for the global solar radiation (GSR) as a target variable. The CNN-LSTM\(_L\) and CNN-LSTM\(_U\) models are applied for downscaling of GSR from the coarse resolution scale of 1.25°×1.875° grid spacing to the higher resolution of 0.05°×0.05° grid spacing for 17 solar generation sites in Queensland Australia.
Statement of the Problem
Longer term assessment of the availability of renewable energy resources largely depend on the projections produced from Global Climate Models (GCMs) that have been made available through initiatives such as the Inter-governmental Panel on Climate Change (IPCC) Coupled Model Inter-comparison Phase 5 Project (CMIP5). However, solar energy generation by means of Photovoltaic (PV) and Concentrated Solar Power (CSP) systems are vulnerable to long term changes in climate [5]. Energy systems thus require the use of a variety of models in order to evaluate the impact of climate change on energy systems operations, scheduling and forward planning [5]. General Circulation Models (GCM) are coarse resolution models which require a method of mapping outputs to a regional scale. Statistical methods have traditionally provided a means of learning such a mapping. Such approaches have limited capability in capturing the non-stationary spatial and time dependent character of the signal for local observations and are challenged in their ability to provide a generalisation over multiple geographic sites. Feature selection, provided by techniques such as genetic algorithms, is required in order to determine which combination of inputs should be selected for training accurate predictive models. The resulting downscaled mapping will inherit the uncertainties and bias of the source GCM model, hence the use of an ensemble of GCMs is recommended [6]. The downscaling of global solar radiation is less common than other climate variables such as precipitation and temperature, and there are relatively few studies investigating the downscaling of GCM outputs for the long-term projection of global solar radiation within the Australian region.
The Deep learning methods, used in this field of research, have demonstrated a broad applicability in being able to learn abstract representations for complex time dependent and non-linear data [7]. Such methods are gaining adoption for use within short term forecasting of climate signals such as precipitation and global solar radiation [8], [9]. However, there is a lack of application of deep learning methods to the downscaling of GCM outputs for the projection of global solar radiation.
Deep learning architectures are composed out of smaller non-linear modules which are combined to form a representation learning pipeline (successive modules automatically extract features and relationships within the data that is fed through the network) [7]. However, most model architectures appear to be designed by means of heuristic guidelines and previous empirical studies. There is need for a new methodology supporting the evaluation of the configuration of these algorithms within the respective network architectures and also to produce a network configuration that is suitable for the application (e.g., the downscaling task performed in this thesis).
Research Aims and Objectives
This Master of Science Dissertation focusses on developing deep learning methodologies for renewable energy applications and aims to contribute towards the following high-level research aims.
The first aim is to build a deep learning-based CNN-LSTM model that integrates Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM) Networks and test its capability to downscale the global solar radiation. For this purpose, an ensemble of three Coupled Model Inter-comparison Phase CMIP5 GCM data sets at seventeen key solar energy sites within Queensland, Australia are used to train and evaluate the proposed CNN-LSTM model. By applying the proposed CNN-LSTM model, the thesis also provides a feasible projection of the downscaled global solar radiation datasets for selected study sites, and the projected future changes are evaluated for two global warming scenarios based on RCP4.5 and RCP8.5 datasets produced by the CMIP5 GCM model. This thesis will provide a comprehensive analysis of the results produced by the CNN-LSTM model, via the quantification of the uncertainty in downscaling global solar radiation datasets in reference to their observations at each study site.
The efficacy of the deep learning-based CNN-LSTM model is evaluated in comparison to the baseline models previously used in downscaling global solar radiation for both the local (i.e., single grid point) and the universal (i.e., multiple grid points within the vicinity of each study site) cases. The trade-offs in the choice of network configurations in designing the optimal network architecture are to be evaluated within the context of the downscaling tasks.
In this Master of Science Dissertation, the research methodology is based on deep learning algorithms that have good ability to automatically extract non-linear, spatio-temporal relationships between many variables [8],[10] [11]. The methods such as super-resolution (SR) downscaling consisting of 2-dimensional Convolutional modules (CNN), have been applied in downscaling of reanalysis products for modelling climate variables including precipitation [10] and atmospheric optical depths [12]. In this project, the Convolutional (CNN) and Long Short-Term Memory (LSTM) models have been integrated following earlier studies such as the use of these models in autoregressive time series prediction of precipitation [8] and global solar radiation [9]. However, the application of such modelling architectures is yet to be applied for direct downscaling of GCM model outputs to simulate global solar radiation.
The primary aim of this Master of Science Dissertation is to build the CNN-LSTM model using an ensemble of three selected CMIP5 GCM model outputs in order to downscale the projected changes in global solar radiation for selected sites with a high capacity for solar energy within Queensland. This new modelling procedure utilises historical outputs from the GCM model ensembles, matched with the closest ground-based station observations of global solar radiation as well as the gridded datasets sourced from the Scientific Information for Land-owners (SILO) climate database operated by the Queensland Government [13]. Two variants of the modelling task are produced, a local CNN-LSTM model (CNN-LSTM\(_L\)) which is trained on \(1 \times 1\) grid point derived from all GCM model outputs for each site so as to learn a mapping to a single SILO global solar radiation observation at the study site grid point. Two kinds of local CNN-LSTM models are constructed, the CNN-LSTM\(_L\) produces a single grid point, the second CNN-LSTM\(_L\) DENSE operates in a manner similar to an encoder-decoder network and mirrors the input as well as generating simulations for the single point of global solar radiation. The second downscaling task, the universal CNN-LSTM model (CNN-LSTM\(_U\)), is trained on \(3\ \times 3\) GCM output grid point set around the study site in order to learn a mapping to a \(9\ \times 9\) SILO high resolution grid point set for the historical global solar radiation observations around each study site. The use of multiple training locations in the dataset ensures the model is exposed to broader climate variability over the region and improves the robustness of the model [14]. Deo, et al. apply a similar methodology in the development of a universal ELM for the forecasting of long-term solar radiation, training on a subset of the study sites and validating on the remaining sites [14]. This thesis differs in method where the training and validation datasets are sampled across all sites in the study region.
In the next phase after the initial model design, the future projections are then explored for two climate warming scenarios. The study firstly considers Representative Concentration Pathway RCP4.5 future simulation datasets where a global response to climate change is made based on the declining reliance on fossil fuels and an emissions trading scheme to stabilise long term greenhouse gas emissions by the year 2100 [15]. This scenario is associated with an estimated rise in average global temperature between 1.1 °C and 2.6 °C [16]. The second datasets representing future simulations is obtained from RCP8.5 scenarios where there is less response to climate change with high population growth and has the highest corresponding greenhouse gas emissions [17]. Under this warming scenario, the average global temperature is expected to increase between 2.6 °C and 4.8 °C [16].
The utilisation of these projections is expected to provide a good indication of the likely impact of longer-term changes in global climate affecting the generation of solar energy for the Queensland region. This research project aims to implement a newly designed CNN-LSTM models for its capability to simulate CMIP5 GCM datasets, both for the historical and future period based on RCP4.5 and RCP8.5 global warming scenarios.
The secondary aim seeks to quantify the uncertainty of the CNN-LSTM model projections against the distribution of observations from which they are derived. By doing so, a level of confidence can be determined for use of the localised future projections. The downscaling CNN-LSTM model is required exhibit the ability to generalise with suitable accuracy across multiple sites within the region of interest. This is a challenge for the traditional statistical models which require a separate model for each individual site due to variation in local climate. Hence the model must be able to capture such variation well. The resulting CNN-LSTM architecture if successful will provide a method by which the downscaling of GCM outputs may be performed for other study areas. There are trade-offs in the choices of how specific deep learning modules should be composed into the overall network architecture, and this thesis aims to make an evaluation of several network configurations and to determine recommended configurations for network training.
Significance of the Study
The downscaling of global solar radiation is of interest in order to assess the suitability of different locations for renewable energy generation, from the perspective of allocating investment in proposed projects as well as in terms of longer-term energy security. Changing climate introduces further uncertainty for decision making in planning such projects, and mechanisms for modelling the availability of global solar radiation under different climate pathways are necessary in order to support such decision making. Global Climate Models are the primary source of climate variables that support longer term simulation of large scale climate variables. These models produce outputs at a course scale grid resolution. In order to evaluate longer term availability for global solar radiation, at a local scale, it is necessary to generate models capable of downscaling such climate variables under future climate scenarios. Yet much of the literature focuses on downscaling from reanalysis products rather from GCM models themselves and focus largely on precipitation and temperature with very few studies focusing on solar radiation.
This Master of Science Dissertation develops a CNN-LSTM model to downscale output variables from three large scale Global Climate Models to a local scale at selected key solar project sites within the Queensland region. It compares the proposed model with three baseline statistical models traditionally used for the downscaling task and produces projections under two representative concentration pathways for the next century up to 2100 in order to identify future differences in solar availability within the Queensland region. The use of the CNN-LSTM has previously been applied in the forecasting domain where there is an auto-regressive setting, but not to the GCM downscaling setting, where there is no such relationship, and the model must learn the mapping between variables produced by the course scale GCM and fine scale observed global solar radiation.
Organisation of this Thesis
This thesis is organised such that Chapter 1 introduces the motivation for downscaling Global Climate Models with respect to solar radiation and renewable energy. It briefly identifies the short comings of traditional methods for downscaling and introduces the potential of Deep Learning methods in developing end to end pipelines for the purposes of downscaling. The research aims are outlined where the primary objective is to develop a CNN-LSTM model for the downscaling of a GCM ensemble for a selection of major solar generation projects within the Queensland region.
Chapter 2 describes current research in the area of Climate Model downscaling as well as in the field of Deep Learning and its application to climate forecasting and climate model downscaling. It identifies the challenges involved with statistical downscaling and the approaches taken in statistical and machine learning methods. A number of approaches for deep learning methods are also identified along with the advantages identified in research of leveraging these approaches. A feature of deep learning is the use of CNN modules for feature extraction and LSTM modules for modelling dynamic systems, as such the hybrid architecture of the CNN-LSTM is identified as having performed well in the forecasting task but not applied to the downscaling context, leaving the opportunity to evaluate the efficacy of such an approach in the task of downscaling global solar radiation.
Chapter 3 describes the methodology of this thesis and introduces the approaches to model development for the local and universal CNN-LSTM and baseline models as well as describing the evaluation metrics and quantification of uncertainty used in this thesis. Details as to the region of interest and the treatment of GCM data in preparation for modelling are outlined. The end to end search procedure for the development of the CNN-LSTM model architecture is described in detail. The resulting parameterisation of the CNN-LSTM\(_L\) and CNN-LSTM\(_U\) as well as the baseline models are reported at the end of the chapter.
The evaluation of the CNN-LSTM\(_L\) and CNN-LSTM\(_U\) models are contained within Chapters 4, 5 and 6. Detailed model evaluation and comparison are carried out in Chapter 4 for both the local (single grid point) and universal (\(9 \times 9\) grid point) downscaling tasks. Comparisons of evaluation metrics are made between each CNN-LSTM model variants and baseline models for the historical test set period from 1988 to 2005 as well as for the two climate profiles RCP4.5 and RCP8.5 between the period 2006 and 2020. Where the CNN-LSTM\(_L\) is shown to outperform the baseline models at a majority of sites on each of the evaluation metrics, while the CNN-LSTM\(_U\) is shown to have lower KGE results yet outperforms the baseline model on all other metrics. In Chapter 5, model uncertainty is evaluated for the historical test set as well as the two climate profiles RCP4.5 and RCP8.5. Comparisons are made between the variations of the CNN-LSTM model and the selected baseline. Bias is examined for each month of the year and months exhibiting larger bias are identified.
Chapter 6 examines the projections for predicted global solar radiation under each of the climate profiles RCP4.5 and RCP8.5. Differences between the profiles are examined for both the local and universal models.
Chapter 7 provides the summary of findings and examines the limitations of the modelling procedure and the resulting CNN-LSTM models as well as offering some views on why these limitations occur. The thesis is completed with a number of recommendations on the directions for future work.
References
Downscaling Global Climate Models with Convolutional and Long-Short-Term Memory Networks for Solar Energy Applications by C.P. Davey is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.