Skip to main content

Webcams for the energy transition

How we use public webcams to make solar power forecasts cheaper and more accurate.

#ai #deeplearning #sustainability
Webcams für die Energiewende St. Gallen

Our Impact

  • More accurate forecasts and good quality with public webcams

  • Forecast accuracy: approx. 4.3% nRMSE (10 min ahead)

  • Deep learning prototype developed in just 6 months

The Challenge

Global electricity consumption? Rising. By 2024, the increase will be 4.3%. The highest figure in years. And who is driving this growth? 75% of it comes from solar energy – within the newly built renewable energy sources. In the EU alone, it is now the leading source of electricity. However, the more solar power plants there are, the more important it becomes to control them precisely. The problem is that spontaneous weather changes, especially locally occurring clouds, make short-term forecasts difficult. Traditional meteorology reaches its limits here.

Research is dominated by so-called SkyCams – special hemispheric cameras that look vertically into the sky. They deliver good results, but are expensive and rare. So we asked ourselves: Why not use what's already there? Public webcams.

The Solution

At the heart of our research is a deep learning prototype that generates electricity forecasts based on publicly available webcam image sequences. Why is this special? Because these cameras were not built for this purpose—unlike classic SkyCams, which look vertically into the sky and are considered the gold standard in the literature.

We are changing the perspective:

Instead of relying on special hardware, we are using what is already available – horizontally aligned webcams within a radius of 12–24 km around St. Gallen. The challenge here is that these cameras do not always provide the perfect image and are sometimes located kilometers away from the PV systems.

Our answer to this is technical in nature: we combine convolutional neural networks (CNN) for image processing with long short-term memory networks (LSTM) for temporal modeling. The model is supplemented by contextual information – such as the position of the sun or past PV production. The result is a multi-horizon prediction model that simultaneously forecasts six points in time at 10-minute intervals.

This has several advantages:

  • Less computing power required

  • More consistent results

  • Greater robustness against outliers

Technically, we work with:

  • Python, PyTorch & PyTorch Lightning

  • Smart meter data (15-minute resolution, interpolated to 10 minutes)

  • Webcam images synchronized every 10 minutes 

  • Evaluation methods such as nRMSE, MAE, MSE, WAPE

This creates a data pipeline that automatically processes and prepares image data and production values and feeds them into the model – a cleanly integrated process from the camera to the prediction.

Another focus was on the spatial combination of the different camera sources. This is because it is not the distance to the PV system that is decisive, but its position relative to the wind direction. Those who see the weather coming win. In later model versions, we are therefore also evaluating strategies such as attention mechanisms and camera embeddings in order to weight relevant viewing angles even more specifically.

The Result

A prototype that not only works – but impresses. Our best model achieves a prediction accuracy of around 4.3% nRMSE (10min ahead) – with up to a 3.25% improvement over our own baseline – and thus outperforms established benchmarks from the technical literature.

All central model variants were evaluated with significant improvements (p < 0.01). Our results show that even horizontally aligned, non-specially installed webcams are capable of providing reliable performance forecasts for photovoltaic systems – on a par with SkyCams.

What this means for the energy sector:

  • Cost efficiency: No special cameras, no additional installations

  • Scalability: Webcams are available in almost every city – the potential is huge

  • Access to data: The key to accurate predictions lies in the right data. It was only possible to train and evaluate the model to this level of quality thanks to the freely accessible production data from Stadtwerke St. Gallen's Open Data Portal. This shows that transparent data provision is not a nice-to-have, but a real enabler for innovation in the energy transition.

We are also closing gaps in research: Previous studies with public webcams were limited to individual images or extreme proximity to the PV system. Our work shows that with the right model architecture, intelligent feature combinations, and good preparation, even more widely distributed camera locations can be highly effective.

Ready for the next step: Whether attention mechanisms, transfer learning with domain-specific CNN backbones, or the integration of additional weather factors such as wind, temperature, or snow cover—our model is the basis for a new generation of smart, accessible energy forecasts.

What the experts say

  • "The energy transition is not just happening at the political level—it is taking place in cities like ours in very concrete ways. For us, it is essential to test technologies that not only work but are also widely applicable.“

  • "If we are serious about the energy transition, we also need new ways to control its dynamics. Our research shows that it can be done more cost-effectively, more scalably, and still with precision."

Let's connect!

Want to learn more?