Welcome to our data-driven journey through the ebbs and flows of electricity generation in the times of COVID-19. As the world hit the pause button and streets emptied, power plants continued humming—but the tune was a little different. From the bustling cities of France to the serene landscapes of Finland, we saw changes that made us rethink our relationship with energy. Think of this report not just as a collection of stats and graphs but as a story of resilience, adaptation, and the occasional surprise. So grab a cup of your favorite beverage, get comfy, and let’s dive into the electrifying narrative of how life’s unexpected turns can light up the grid in ways we never imagined.
This data story delves into the fascinating interplay between the COVID-19 pandemic and the transition to green energy in Finland, France, Germany Netherlands, Serbia and Spain. By analyzing various datasets, we uncover how the pandemic impacted electricity generation, consumption, and awareness of renewable energy sources. Our exploration also includes machine learning predictions based on historical data and pandemic interventions. Key questions we address include:
Before we analysis further and answer our questions, we first need have a glimpse of our datas.
Actual Electricity Generation per Production Type in Europe Countries (2019~2022): This dataset contains changes in the electricity generated by different energy sources(eg. biomass, nuclear, marine.) in europe countries, which is recorded hourly or every 15-minutes.
The figure above provides a comprehensive overview of the composition and trends in electricity generation over a period of approximately three and a half years in selected countries. We can discover that:
This is a dataset of the evolution of views aggregated by topic, across a variety of languages. The most relevant topic for our analysis was the Stem Earth and Environment. In an overall idea, the increase of visiting a certain topic means the increase of awareness for that topic. However, when analyzing this, we will have to take the increase and decrease of the background visit. In other words, if the whole website has an increasing visit, it is hard to say that which topic is gaining more awareness. As a result, it is necessary to consider the overall visit frequency when trying to understand the topic visit frequency.
This is the Wikipedia pageviews in different languages.
These additional datasets are pageviews of:
This dataset shows the time of six countries to take actions towards COVID, including the time of reporting the first case and the first death due to COVID, the time that school is closed and public events banned and the time that the country is in lockdown. Also, it shows the time of the country taking normalcy. Some countries didn’t announce the national lockdown or national normalcy, so some time in this table is missing. When analyzing the trend of energy transition, the time listed in the table will be an important reference to when COVID started attacking and when COVID stopped.
The Oxford COVID-19 Government Response Tracker (OxCGRT) database : This dataset contains the strictness of lockdown policies for nearly all countries. We focus on eight containment indicators (C1~C8) in the OxCGRT database:
According to the guidelines of this data set, we calculated the Containment Index of six countries using C1 to C8. It ranges from 0 to 100, with 100 representing the strictest lockdown intensity. The evolution of lockdown intensity in selected countries is shown in the chart below.
Thanks to OxCGRT, which furnishes extensive and scientific data about governments’ diverse policy responses in 2020, 2021, and 2022. This data assesses the degree of governmental action on a scale. The resulting scores were combined to construct a series of policy indices, summarizing the collective impact of these measures.
| Contries | CI (mean) |
|---|---|
| Finland | 29.221927 |
| France | 35.373300 |
| Germany | 40.314003 |
| Netherlands | 38.458093 |
| Serbia | 33.245152 |
| Spain | 44.206023 |
The average CI in different countries are listed in the table above, we can discover that:
This data set contains Google Mobility Report and Apple Mobility Trends, these two datasets are extended to 2022.
This figure comes from Google Mobility Report. It divides mobility to some different categories according to their goal and shows the trend of mobility. The higher the curve come, the more mobility there is at the time. The overall trend of COVID is that it strikes a lot at winter, but in summer the power of COVID will be reduced. As a result, we can see from the figure that the mobility gets the highest during summer (mainly from late May to early September), but gets the lowest during winter (mainly late November to early March). For different activities, the mobility also varies a lot. The activity that is necessary has less fluctuation from time to time, such as grocery & pharmacy and residential. The mobility of entertaining places and workplaces, on the opposite, has a big fluctuation from time to time. We also find that the mobility of workplaces comes low during Janurary every year.
In our exploration of the impact of COVID on the green transition, we first started by the analysis of the aggregated_timeseries dataset. The most relevant topic for our analysis was the Stem Earth and Environment and we focused on it’s distributions of pageview counts pre and post COVID periods. The histogram help vizualizing the pre and post distributions which doesn’t match our expected conclusion, as we see that the mean count of the pre-COVID is higher that the post-COVID one.
To statistically validate the observed difference, we use the z-score test. By converting our pageview counts into z-scores, we standardize the data, allowing us to assess the number of standard deviations each data point lies from the mean. This normalization is crucial for unbiased comparisons.
We have obtained a p-value of 0.016 < significance level of 0.05 , which suggests that the difference is statistically significant, the z-score quantify this difference and shows that the post-COVID mean is 2.407 standard deviations below the pre-COVID mean. This is a substantial deviation and suggests that there is a significant decrease in pageviews for Stem Earth and Environment content post-COVID compared to pre-COVID.
Here, as we have pageview counts for different languages, a Simpson’s Paradox can occur, where a trend appears in several different groups of data but disappears or reverses when these groups are combined, suggesting that the observed overall decrease in pageviews post-COVID might not apply uniformly across all languages.
Therefore, to fully understand the dynamics and to avoid potentially misleading conclusions, we will proceed with a language-wise analysis.
Initial visual assessment of differences between pageview counts accross languages
Our first visual conclusions are for the English (en) and Danish (da) language where the histograms show a pronounced shift towards the left in the post-COVID period. The density of the distributions in the Italian (it) and Finnish (fi) segments, however, shifts right, suggesting an increase in pageviews post-COVID. The histogram for Turkish (tr) content shows an extreme skew towards the post-COVID period, with a very high peak.
Again, we need to statistically validate the observed difference:
Upon analyzing the z-scores and p-values, we find that:
This detailed breakdown demonstrates the varied impact of COVID-19 on online engagement with the STEM Earth and Environment content across different linguistic audiences.
In the second part of our analysis, we expanded the dataset to include pageview counts from a range of topics directly relevant to our investigation. These additional datasets are:
We start by the visualization of the pre and post distribution of page views count for each dataset:
As we can face again the Simpson Paradox, we will the analysis language wise, we have chosen another set of languages for this analysis that covers a wider range of native speakers. The languages considered in these datasets are:
The heatmap illustrates the z-scores associated with changes in pageviews for various environmental topics post-COVID, categorized by language. The shades of red and green represent the magnitude of decrease or increase in engagement, respectively.
For the upper half, particularly, the topics of “Electricity Generation” and “Energy Storage” exhibit some of the most pronounced decreases across multiple languages.
Conversely, the lower half of the heatmap shows significant increases post-COVID. It’s especially apparent for “Renewable Energy,” “Photovoltaics,” and “Electric Vehicle” topics, where certain languages like Portuguese (pt) and Vietnamese (vi) display substantial growth in interest.
While some areas have seen a surge in public attention, others have experienced a decline. Some of this conclusions can only be validated by other cross analysis. For example, we cannot access that the COVID is responsible of the increase in terms of search in the topic Electric Vehicle. To assess this with certainty, we need to perform a causal analysis taking into account the trend of the later years on this topic, the EU reglementations and the exploding prices of gas.
In the real world, there are obvious cyclical patterns in power generation and consumption.
The first figure presents a box plot distribution of electricity generation measured in gigawatt-hours (GWh) across the days of the week. A discernible trend is observed where weekdays exhibit a higher median generation compared to weekends, reflecting the conventional demand cycle influenced by industrial and commercial activities. The presence of outliers, particularly on weekdays, could indicate exceptional demand or supply events.
The second figure shifts the focus to a monthly overview, revealing the seasonality in electricity generation. The box plots indicate a significant variation between months, with the winter months showing elevated levels of generation, which could be attributed to increased heating demand and shorter daylight hours. Notably, December stands out with the highest median generation and the largest interquartile range, suggesting a potential increase in both baseline and peak demand during the holiday season.
These figures underscore the variability in electricity generation that aligns with human activity patterns and natural cycles. By understanding these trends, we further clarified the need to filter out these differences (cofounders) in subsequent analyses.
Before we move on, one thing we need to mention is that nuclear,coal, oil, and gas are non-renewable energy while others(hydro, biomass, wind, and waste) are considered renewable energy sources.
In order to overcome the impact of differences between weekdays and weekends, in our analysis, the total electricity generation and renewable electricity generation are taken as a 7-day average before and after all events.
It can be seen that in most countries, with the advent of the epidemic, total power generation has declined, and a slowing down trend can be seen after Normalcy.
In the figure below, figure shows the trend of energy generated and renewable electricity generated in different countries, trying to find out what impact it will have to different energy generation methods. We can see in most countries, with the advent of the epidemic, total power generation has shown a decline pattern, with a slowing down trend after Normalcy.
Then, A t-test was performed on the average total electricity generation and renewable electricity generation in the 7 days before and after important events.
The heatmap for total electricity generation depicts a general pattern of significant change in response to the onset of COVID-19 and the ensuing governmental measures. Most countries registered statistically significant shifts in electricity generation patterns during key events. However, the imposition of mobility restrictions did not correspond with significant changes in electricity generation in countries like France and Spain, as evidenced by p-values of 0.994 and 0.679, respectively.
Similarly, in the heatmap for renewable electricity generation, it appears that renewable sources were impacted by the pandemic measures in most countries. Notable exceptions include Finland and the Netherlands, where ‘1st case’ and ‘Normalcy’ led to a larger p-value, indicating there is no significant change in renewable electricity generation. This could reflect the inherent variability in renewable sources or potential shifts in energy policy or market dynamics in response to the pandemic.
First, let’s take a look at the electricity generation structure amongs six countries before, during and after the pandemic respectively.
Before we show you the visualization, let me introduce how we define before, during and after.
In our analysis, we define the “epidemic period” as the time frame beginning with the first confirmed case and extending until the point of return to normalcy((Spanish normalcy is missing, we took the average to fill it in)). This period encompasses the entire duration of the epidemic’s impact, starting from its initial emergence and concluding when conditions have stabilized and daily life has resumed its usual rhythm. This definition allows for a comprehensive understanding of the epidemic’s full scope and duration, capturing both the onset and the eventual subsiding of its effects.
To mitigate the impact of natural rhythms or seasonal fluctuations, we adopt a comparative approach in our data analysis. Specifically, for the periods before and after our focal “during” phase, we align the data based on the same months and days, but adjust the years, either by decreasing or increasing them by one year. This method allows us to closely mirror the temporal context of the “during” phase while effectively isolating and minimizing seasonal or cyclical variations. By comparing these analogous time periods across different years, we aim to gain a clearer understanding of any changes or trends that are not merely attributable to regular seasonal patterns.
Based on pie chart, we can get some basic observations:
These observations reflect the dynamic nature of energy consumption and production patterns in response to global events such as an epidemic. They also highlight the varying degrees of adaptability and resilience in different countries’ energy sectors.
Let’s conduct an analysis of variance (ANOVA) for these energy sources to assess whether there are statistically significant differences between them in terms of their effects or contributions. This analysis will help us determine if there are notable variations among these sources and which sources might significantly differ from the rest in their impact or significance.
The bar chart visualizes the -log10 transformed p-values from the ANOVA results for each energy source. Red Dashed Line: This line represents the -log10 transformed significance level (typically p = 0.05). Bars that extend above this line are considered statistically significant.
Well, it is obvious that the key sources we mentioned previously are all in greater significance. So there might be some main factors that affect the trend of the key sources.
Interestingly, for most countries (except France), the p-value of nuclear power is not very high, which is consistent with the fact that nuclear power acts as base load in reality.
In order to answer these questions, we need to explore the mobility data: apple_mobility which contains the intensity score of the region in three transportation types(driving, walking and transit) of each day from 2020-01-13 to 2022-04-12. Please notice that the transit data of Serbia is missing.
The intensity score refer to relative mobility strength based on first day(2020-01-13, which intensity score is 100). The higher intensity score, the more people choosing the transportation types.
Review the data we all have now, mobility, intervention and energy generation. It is time to combine then together!
At the onset of lockdowns, a sharp decline in mobility was observed across all countries. The once-bustling avenues of movement—captured in metrics of driving, walking, and transit—saw a sudden plunge. This drop in human activity was mirrored by a corresponding decrease in energy consumption. The graphs show a synchrony between the fall in mobility and a dip in energy usage, suggesting a direct correlation between the two.
While the reduction in mobility was somewhat uniform, the impact on energy consumption varied. In Finland, energy generation closely tracked reductions in mobility, indicating a direct dependency of energy usage on population movement. In contrast, countries like Germany displayed resilience in energy consumption despite decreased mobility, hinting at a robust energy infrastructure possibly catering to essential services that remained operational.
Gradually, as restrictions lifted, there was a cautious uptick in mobility. This return to movement, however, was not always matched by a proportional increase in energy consumption, suggesting a lasting change in energy use patterns, perhaps due to a sustained shift towards remote work and more conscious energy utilization.
While we’ve established a foundational perspective on how interventions influence mobility and energy consumption, to truly grasp the depth of these impacts, we require a more systematic assessment of the intervention strictness implemented by these countries. By quantifying the degrees of lockdown measures, we can disentangle the nuanced effects they have had on the day-to-day activities and energy usage patterns. This will enable us to paint a clearer picture of the relationship between policy strictness and its real-world repercussions
Acknowledging the myriad of elements that could potentially affect energy consumption and its distribution, it becomes crucial to dissect and understand these intricacies. To achieve this, we are set to embark on a correlation analysis, complemented by an examination of p-values. This statistical approach will assist us in uncovering the degree to which these factors are related to energy consumption—whether they move in tandem or in opposition to each other. By deciphering the significance, directionality of these correlations, and the p-values, we can gauge the statistical significance of our findings. The inclusion of p-value analysis ensures that our conclusions are not just based on observed trends but are also statistically robust. This comprehensive analysis will illuminate how various influences interplay to shape energy usage patterns, particularly during the unique circumstances brought by the pandemic.
Let’s analyze the result one by one:
Finland: A notable observation is the strong negative correlation between walking and total energy generation (-0.611). However, the p-value (0.935) suggests that this correlation is not statistically significant, which means we cannot confidently attribute changes in energy generation to changes in walking activity.
France: The negative correlation between renewable energy percentage and total energy generation (-0.351) is not statistically significant either, given its p-value (0.306). Thus, while there is an observed association, we cannot reliably infer a causal link between these variables.
Germany: The correlation between parks and total energy generation is slightly negative (-0.507), and the p-value (0.040) indicates that this correlation could be statistically significant. This suggests a potential inverse relationship between leisure activity and energy production, possibly worth further investigation.
The Netherlands: There’s a strong negative correlation with the day of the week (-0.431), with a significant p-value (0.030), implying a consistent pattern where the day within the week might be influencing energy generation significantly, potentially reflecting operational cycles in energy usage.
Serbia: While there is a moderate negative correlation between grocery/pharmacy activity and energy generation (-0.362), the p-value (0.689) suggests that this correlation is not statistically significant.
Spain: A positive correlation with the year (0.379) suggests an increasing trend in energy generation over time, but the p-value (0.349) indicates that this trend is not statistically significant.
Across all countries, correlations vary in strength, but many lack statistical significance based on their p-values. This suggests that while some factors may appear to influence energy generation, these influences are not consistently strong enough to exclude the possibility that they are due to random chance.
With the insights from the correlation and significance analysis, we can now transition to predictive modeling using machine learning. The goal is to use the identified relationships that are both strong and statistically significant as features in our model.
From the Spearman correlation analysis above, we excluded the features of ‘DayofMonth’ , ‘transit’ and ‘Workplaces’, which shows a relatively high p-value in most countries.
Before we start, let us first introduce the datasets we use to do analysis and forecasting. Based on the correlation and p-value analysis
Total representing the total electricity consumption as the label.
Based on the correlation and p-value analysis, we select out the following features:
DayOfWeek If this factor has consistently low p-values across multiple countries, it indicates a significant weekly pattern in energy generation that could be predictive regardless of the country. Energy consumption patterns can significantly differ between weekdays and weekends.
Renewable_percent: Even though the p-values vary, if the correlation is consistently strong across countries, this feature may capture the influence of renewable energy production on the total energy generation, which is a crucial aspect of modern energy systems.
Parks: If the recreational activity, as represented by the usage of parks, shows consistent correlations with energy generation, it might reflect broader societal behavior patterns that affect energy usage, such as reduced industrial activity on days with higher recreational activity.
Transit Stations: The use of transit stations could be a proxy for overall economic activity, including commuting patterns and industrial activity. If it correlates with energy generation across several countries, it could be a relevant predictor.
CI (Containment Index): This could be a particularly relevant feature if the lockdown measures significantly impacted energy consumption patterns. The CI might capture the effects of reduced industrial, commercial, and personal activity on energy demand.
Walking: The activity level of the population, as indicated by walking, if correlated with energy generation, might reflect changes in daily routines that could impact energy consumption, such as increased remote work or shifts in commercial activity.
Month, Year, driving, Retail & Recreation, Residential are included as well.
To ensure our model’s reliability, we’ll split the dataset into training and test sets. This division enables us to train the model on historical data and then assess its performance on unseen data, a crucial step before proceeding with analysis and forecasting.
Specifically, we select all the data from 2020 to 2021 as the training set while the test set contains data from ‘2022’ onwards.
Now that we’ve completed the preparatory work, let’s proceed to train the model
Utilizing the XGBoost library, we construct an XGBoostRegressor model, a highly regarded gradient boosting algorithm recognized for its proficiency in time series forecasting. Fitting the model involves training it on the provided dataset. Subsequently, we assess its performance on both the training and test sets by employing the root mean squared error (RMSE) metric.
Now, it is time to utilize our trained model to make predictions on the test set and then create visualizations to represent the results.
Well, from a direct visual standpoint, it hard to point out which country the model fits well since the y-axis scale is different, we need further information.
As data analysts, we can not rely solely on intuition. So let’s take a look at their root mean squared error scores(RMSE) and relative error(RE). We can find that the model performs better in France and Spain while the model doesn’t fit Serbia well.
| Countries | RMSE [GWh] | RE |
|---|---|---|
| Finland | 15.81 | 0.081 |
| France | 113.36 | 0.076 |
| Germany | 125.05 | 0.081 |
| Netherlands | 31.78 | 0.097 |
| Serbia | 11.52 | 0.121 |
| Spain | 57.00 | 0.079 |
As we see, their RE are about 10% , which indicates the model is performing good.
Within this section, we conducted an analysis of total energy consumption utilizing the integrated dataset. This comprehensive analysis not only offers valuable insights into the patterns of total energy consumption but also facilitates precise forecasting. By comprehending the trends in total energy consumption, informed decisions can be made concerning energy management and resource allocation in the future public health issues.
Our findings shed light on the pandemic’s impact on green energy transition:
In summary, this data story underscores the pandemic’s complex implications for green energy transition and calls for further research and action in these critical areas.
As we wrap up our storytelling session, let’s flick the switch off with some reflections. We’ve surfed through peaks and troughs of watts and volts, uncovering how each country danced to the pandemic’s peculiar rhythm. While traditional power generation often took a siesta, renewables kept the party going, showing us that they’re not just about green living but also about keeping the lights on when life throws a curveball.
It’s been quite the ride—like a rollercoaster for electrons. As we move forward, this tale of energy in the time of a pandemic leaves us with a few nuggets of wisdom: diversity is key in our energy mix, and a bit of green can help keep the grid serene. So let’s toast to the power of data in shedding light on our world and helping us chart a course to a brighter, more sustainable future. Here’s to more stories, more data, and more insights.
It’s NOW, Power offffffff !