Skip directly to search

Skip directly to content

 

Hand in Hand with Artificial Intelligence in the Energy Sector

 
 

AI | Jasmina Ovcak |
07 December 2021

Artificial Intelligence (AI) is everywhere. It recognises our faces to unlock our smartphones, it helps us to find our favourite product in an online shop, and it even is in our washing machines! It is no surprise then that AI has also found its way into the energy sector – it can forecast electricity demand, consumption, and cost, detect problems in energy systems, and much more.

AI seems like a magical creature, but in reality, it is a product of human intelligence and lots and lots of data. AI experts combine mathematic, statistic, programming, analytical, and many other skills to create AI models which are able to perform intelligent tasks. This is a complex process, where two key ingredients are necessary for an AI model to be able to forecast phenomena of interest.

The model needs to:

  1. gain experience – through data and interactions,
  2. know how to “think” and make decisions – which is defined by an algorithm.

At Endava, we have developed a solution for forecasting electricity demand, including getting the necessary data, developing an AI model that uses this data, and finally testing the whole solution.

DATA – OBTAIN IT, PREPARE IT

AI systems gain knowledge through data, and therefore, the data should be as good as possible in terms of volume, diversity, and quality. Simply put: the better the data, the better the prediction capability of the model. However, before arriving at the point where we can feed the model with the data, we have to take care of the following aspects:

Identify data sources: first, we need to identify the data sources which contain the data relevant to the problem we are trying to solve. Besides identifying obviously important sources, such as historical values of the variable we want to predict, in this phase we also create hypotheses and make assumptions about additional data sources that could contain valuable information. The practical usefulness of that additional data is evaluated later when we perform additional analyses and observe our AI model in action.

Obtain and process the data: next, we need to obtain access to the data sources, collect the data, parse, validate, transform, and store it. Since, in our case, the data was obtained from various sources, it was necessary to normalise it by transforming the data into a common format in terms of time zones, data frequency, etc. Moreover, since the real-life data is rarely ideal, we also took care of missing and anomalous data points. There are a lot of steps in data processing, and one needs to be very careful when carrying them out. Even the smallest mistake can shift or change the data in such a way that it stops representing a realistic scenario. Consequently, this can influence the forecasting power of the model trained on such data.

AI in Energy

Understand your data and the domain: when working with data, it is important to explore and understand the specifics of our data and to know how the transformations affect it. Also, it is essential to learn about the particular domain we are working on and to apply this valuable domain knowledge as much as possible. For instance, weather can influence electricity demand, and this information should be taken into account when selecting data sources, for example. Only if we truly understand our data, the problem we are trying to solve, and the specifics of the domain, can we apply the right techniques and prepare the data for the next step: the algorithm.

THE ALGORITHM – THE BRAINS OF THE MODEL

Besides the data, the algorithm plays a crucial role in creating a predictive solution. The algorithm defines how the model processes data and makes predictions. From our experience, the implementation of an algorithm roughly incorporates the following aspects:

Select a machine learning (ML) algorithm: first, one needs to select a suitable ML algorithm. A lot of different algorithms exist, but it takes profound knowledge and experience to choose the best one (or several) for the problem at hand. The decision is made based on the type of problem we are trying to solve, the data characteristics, our resources, and previous experience. In our project, we aimed to forecast the electricity demand in a market, and for the basic ML algorithm, we chose the one called Random Forest due to its various advantages.

Implement the custom code: although the basic implementation of Random Forest exists in ML libraries, it was not sufficient to just call up a function from the library, pass the data as a parameter, and observe the final predictions. It was necessary to implement additional, carefully designed custom functions for the specific problem at hand. These combine various models for predicting demand in advance for different timeframes, process and aggregate the outputs, predict peaks, etc.

Again, understand your work: also in this phase, the domain knowledge and problem understanding are essential to creating a solution for predicting phenomena of interest in the best possible way.

DON'T FORGET TO TEST

Even if we carefully implement the code, patiently work with the data, incorporate expert knowledge into AI models, and truly understand every step on our way, one other key ingredient is necessary: testing. We wrote unit tests, component tests, integration tests, and estimated the predictive power of the developed models through carefully selected and designed AI validation procedures. Quality is important to us, and our goal is to deliver high-quality and robust AI solutions.

ON THE WHOLE...

Our journey hand in hand with artificial intelligence in the energy sector has been fantastic and inspiring – from collecting and processing the data to designing and integrating our predictive solution. When I think about it, I remember hard work, awesome ideas, and discussions which resulted in a whiteboard full of text and diagrams, but also all the time spent with a great team and their happy faces when, after all the hard work, we had achieved our goals.

Jasmina Ovcak

Data Science Consultant

Jasmina is a data scientist and machine learning engineer with a background in software development. She has more than 10 years of experience and a PhD in the domain of machine learning and artificial intelligence. Jasmina is passionate about knowledge and constantly making progress in her career path. Besides working on AI projects and leading teams, she has also gained valuable experience in market research, pre-sales, and preparing project proposals. In her free time, Jasmina enjoys travelling, reading books, dancing salsa, and doing yoga.

 

Related Articles

  • 07 December 2021

    Hand in Hand with Artificial Intelligence in the Energy Sector

  • 02 November 2021

    Leveraging ESG Data to Grow Your Business

  • 05 October 2021

    How to Improve Intelligent Energy Storage Systems Using AI

  • 28 September 2021

    Data-Driven Insurance

  • 05 May 2021

    Artificial Intelligence: Where Does The Real Value Lie?

  • 11 February 2021

    Mapping the Future Applications of Artificial Intelligence

  • 16 June 2020

    Automation in the Age of Digital Necessity

  • 27 August 2019

    Taming AI in a Cognitive Driven Business World

Most Popular Articles

hello! I’m Paul Willoughby
 

Meet the SME | Paul Willoughby | 12 January 2022

hello! I’m Paul Willoughby

An Introduction to Mobility as a Service in the US
 

Mobility | Brian Estep | 18 January 2022

An Introduction to Mobility as a Service in the US

Insurance Insights: Customer Retention & Cross-Selling
 

Insurance Insights | Bradley Howard | 16 March 2021

Insurance Insights: Customer Retention & Cross-Selling

How the Board Game Catan Conquered the Digital World
 

Innovation | Moritz Hampel | 27 July 2021

How the Board Game Catan Conquered the Digital World

Payment Service Providers 2.0
 

Payments | Peter Theunis | 11 January 2022

Payment Service Providers 2.0

How to Improve Intelligent Energy Storage Systems Using AI
 

AI | Uros Bajec | 05 October 2021

How to Improve Intelligent Energy Storage Systems Using AI

Insurance Insights: Intelligent Underwriting Workbench
 

Insurance Insights | Gareth Miller | 30 March 2021

Insurance Insights: Intelligent Underwriting Workbench

How to Create a Company Culture that Encourages and Withstands Failure
 

Innovation | Teodora Chetan | 14 January 2020

How to Create a Company Culture that Encourages and Withstands Failure

Operating Responsibly for Future Success
 

The Endava Experience | Rohit Bhoothalingam | 03 November 2021

Operating Responsibly for Future Success

 

Archive

  • 19 January 2022

    The 3 Big Ps in Modern Insurance: Personalisation, Prediction and Prevention

  • 18 January 2022

    An Introduction to Mobility as a Service in the US

  • 12 January 2022

    Buy or Build? A Game-Changing Question in Insurance

  • 12 January 2022

    hello! I’m Paul Willoughby

  • 11 January 2022

    Payment Service Providers 2.0

We are listening

How would you rate your experience with Endava so far?

We would appreciate talking to you about your feedback. Could you share with us your contact details?