Tag: optimization

The many flavors of the “stochastic optimization sandwich”

Whenever I discuss with colleagues the topic of stochastic optimization, there always has been an image coming to my mind: that of a sandwich (or more of a burger, for culinary precision).

As odd as it sounds, I had the feeling that this may help illustrate some of the difficult notions with regard to optimality in optimization under uncertainty. In particular the gap that often lies between the optima with and without perfect foresight (perfect knowlegde of futur events, which are otherwise supposed uncertain in stochastic optimization). This gap is known as EVPI (Expected Value of Perfect Information) in the Stochastic Programming literature.

Although I already have a slide deck for my short introductory course on the topic (EVPI comes at p. 52/85), I realized I had never materialized this “sandwich illustration”, except transiently on a white board. Today, as I have so many other things to finish, I finally opened Inkscape to make a first version. I’m not sure it’s that meaningful without an explanation, but I find it decent. It should find its place in next year’s version of the slides.

Illustration of stochastic optimization solution concepts and the gap between then, called the “stochastic optimization sandwich”

Some explanations

The illustration encodes two ideas:

  • Sandwich: Stochastic Programming theory states that the values of three typical solutions of the stochastic optimization problem are ordered, thus they can be stacked as in a sandwich. Here there are, sorted from the best to the worst (sandwich bottom, middle and top)
    • the “best ever solution”, which assumes perfect foresight, which is cheating on the stochastic aspect of the problem
    • the genuine best solution, coming from Stochastic Programming, which truly respect the fact that foresight is limited (only a probabilistic knowledge of the future is assumed)
    • the “deterministic forecast heuristic” solution, where the problem is solved by replacing each uncertain quantity by its forecasted (expected) value
  • Different flavors: theory further states that the performance of these three variants are ranked, but only with non-strict inequalities (≥, not >), so that it may be the case, on a particular problem, that the perfect foresight solution is no better than the imperfect foresight solution (in the domain jargon: EVPI = 0). This implies it is not worth investing resources on finding better forecasts of uncertain quantities. Same, when comparing the deterministic heuristic solution with the stochastic solution: the gap between the two (called VSS, for Value of the Stochastic Solution) can be zero in some cases, so that the application the fancy stochastic programming method brings no benefit. The fact that the EVPI and VSS layers (certainly tasty ingredients, although I have never tasted them...) in the sandwich can disappear makes up for the different flavors.

Electricity market: What does it take to reproduce long strokes of negative prices?

This week, I was trying to get a deep understanding of the necessary conditions for the appearance of contiguous periods ("long strokes") of negative prices in electricity spot markets. Such strokes appeared for example last weekend (5 Apr 2026).

For this study, I've created a toy economic dispatch, with as few parameters as possible. Indeed the goal is certainly not to recreate a complicated twin of the real power markets but instead only summon the simplest elements which are needed to cause the phenomenon.

My core assumption, waiting to be refuted, is that negative system prices can appear even when the marginal costs of all plants in the system are positive or zero. I suppose that inflexibility (e.g. block orders) should suffice. Is this indeed the case?

As of now, with only two power plants (base: cheap but inflexible and peak: flexible but expensive), the dispatch model can reproduce isolated negative marginal price events (see video capture), which occur at the single instant(s) supporting the curtailment of the base plant due to its inflexibility. However, this model cannot reproduce a sequence of consecutive negative price instants.

Adding free solar electricity makes up for more colorful graphs and generates long strokes of zero marginal price, but not strictly negative.

So my question/challenge is: what does it take to reproduce long strokes of negative prices?

  • A full-blown day-ahead power market model? Hopefully not!
  • Introducing binary (ON/OFF) decisions or other non-convexities? Perhaps, is it necessary to reproduce single negative price instants?
  • A larger, more diverse, fleet of power plants? Perhaps even a number as large the number of consecutive negative instants to be reproduced?

Answer not found yet...

Code available (Python notebook with CVXPY and jupyter widgets): https://github.com/pierre-haessig/electricity-dispatch-negative-price

Présentation « Optimisation des microréseaux » @ENS Rennes

(Post in French, since it’s about a presentation in French...)

Ma présentation « Optimisation des microréseaux » faite à l’École normale supérieure de Rennes pour les Rencontres Mécatroniques est disponible en ligne. À destination des étudiant.e.s en mécatronique, elle se voulait assez pédagogique pour expliquer les enjeux de dimensionnement et gestion d’énergie des systèmes énergétiques.

Présentation « Optimisation des microréseaux » @ENS Rennes (27’ présentation + questions).
Pas facile de se ré-entendre avec tout ses tics de parole !

Beaucoup d’idée développées dans le cadre de la thèse d’Elsy El Sayegh (soutenue mars 2024) avec mon collègue Nabil Sadou et qu'on continue à explorer avec Jean NIKIEMA, nouveau doctorant de l’équipe AUT !

An open benchmark for energy management under uncertainty

Now that I'm back from SGE 2018 conference, I've put online the manuscript of my article and the slides of my presentation (in French).

“Gestion d'énergie avec entrées incertaines :
quel algorithme choisir ?
Benchmark open source sur une maison solaire”

The title in English (translation of the whole article in progress...) is:

“Energy management with uncertain inputs:
which algorithms ?
Open source benchmark based on a solar home”

Here is the model of the solar home (power flows)

solar home control bench (power flows model )

I've also a first translation of the abstract:

“Optimal management of energy systems requires strategies based on optimization algorithms. The range of tools is wide, and each tool calls on various theories (convex, dynamic, stochastic optimization...) which each require a period of appropriation ranging from a few days to several months.

It is therefore difficult for the novice energy management practitioner to
understand the main characteristics of each approach so we can compare them objectively and finally find the method or methods best suited to a given problem.

To facilitate an objective and transparent comparison, we propose an exemplary and simple energy management problem: a solar house with photovoltaic production and storage. After justifying the sizing of the system, we illustrate the benchmark by a first comparison of some energy management methods (heuristic rule, MPC and anticipatory optimization). In particular, we highlight the effect of the uncertainty of solar production on performance.

This benchmark, including the management methods described,
is open source, accessible online and multi-language (Python, Julia and Matlab).”

Access to the benchmark

The entire source code and the data (an extract from the Solar home electricity dataset by Ausgrid) is available on GitHub:

https://github.com/pierre-haessig/solarhome-control-bench/

As of now, only rather simple energy management methods are implemented, but I'd like to add some kind of stochastic MPC (once I've clarified what this really means), and later Stochastic Dynamic Programming.