Disabling syntax highlighting with the Crayon plugin

Since the beginning of this WordPress blog/webpage, I've been using the Crayon syntax Highlighter plugin to enhance the display of small code extracts. Recently, a few days after publishing my small post on upgrading Julia, I discovered the page display was unfortunately broken. Only the post title was rendered and I had no idea why…

Screenshot of the blog post on Upgrading Julia, broken due to the Crayon syntax hightlighter plugin
Screenshot of the blog post on Upgrading Julia, broken due to the Crayon syntax hightlighter plugin

Debugging a Worpress website

I had no experience with debugging a Worpress website. Fortunately, it turned out to be not that complicated. Searching for “Wordpress debug” quicky yield useful pages like How to Enable WordPress Debug or Debugging in WordPress. Setting two variables in the wp-config.php file was enough. I quickly saw logs full of mentions to the Crayon plugin. In particular one PHP Fatal error. Even without being a specialist, this didn't sound ok!

[16-Oct-2019 12:57:53 UTC] PHP Fatal error:  Uncaught Error: Call to a member function id() on array in […]/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_formatter.class.php:36
 Stack trace:
 0 […]/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_formatter.class.php(538): CrayonFormatter::format_code('', Array, Object(CrayonHighlighter))
 1 [internal function]: CrayonFormatter::delim_to_internal(Array)
 2 […]/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_formatter.class.php(516): preg_replace_callback('#()#msi', 'CrayonFormatter…', 'export PATH="/h…')
 3 […]/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_highlighter.class.php(166): CrayonFormatter::format_mixed_code('export PATH="/h…', Object(CrayonLang), Object(CrayonHighlighter))
 4 […]/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_highlighter.class.php(186): CrayonHighlighter->process()
 5 […]/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_wp.class.php( in […]/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_formatter.class.php on line 36

After that diagnostic, it was just a matter of disabling the Crayon plugin and the website got back on track!

It happens that Crayon is unfortunately unmaintained these days. It is only testing against Worpress 4.2, while we are now in the 5.x series, and issues are piling up on GitHub. For the moment, I don't have searched for a replacement. All code extracts throughout the website display without syntax hightlighting, but at least they do display!

Upgrading Julia and IJulia on Linux

This is a short note on how to swiftly update Julia and IJulia (the kernel to work with Jupyter notebooks) when a new minor version is released. The following instructions are for Linux, with the example of upgrading from version 1.1 to version 1.2.

(remark: the existing StackOverflow question “How to upgrade Julia to a new release?” is very outdated, as of September 2019, and doesn’t cover Jupyter kernels)

Step 1: dowload

Download the latest binary release: https://julialang.org/downloads/ and unpack it. For me: ~/Programmes/others/julia-1.2.0

Step 2: update $PATH

Update path to the Julia binary which is prepended to the $PATH variable in the ~/.bashrc file:

export PATH="/home/pierre/Programmes/others/julia-1.2.0/bin:$PATH"

Test in the shell command line: the julia command should now launch the new Julia REPL (at least after a shell restart).

Step 3: reinstall (and update) packages

Objective here is to reuse all the previously installed packages without the need to remember their names.

In the ~/.julia/environments directory, copy the Project.toml file from the previous version directory (ex: v1.1) and paste it in a directory named as the new version (ex: v1.2).

Now, in the Julia REPL, enter the Pkg REPL (pressing ]) and run update. If there is no new package version, there is no actual download happening. However, compiled packages like IJulia should be rebuilt.

Test: assuming that the IJulia packaged was included in Project.toml, the new Julia version should be available as a Jupyter kernel in the lab/notebook interface.

Step 4: clean up the old Jupyter kernel

In the ~/.local/share/jupyter/kernels directory, there should be at least two directories: the one for the old Jupyter kernel (ex: julia-1.1) and the one for the new kernel (ex: julia-1.2). If the old kernel is not needed, it can be removed.

Notice: the kernel directories can be found with the $jupyter kernelspec list command.

(The directory of the old Julia binary can be deleted as well.)

An open benchmark for energy management under uncertainty

Now that I'm back from SGE 2018 conference, I've put online the manuscript of my article and the slides of my presentation (in French).

“Gestion d'énergie avec entrées incertaines :
quel algorithme choisir ?
Benchmark open source sur une maison solaire”

The title in English (translation of the whole article in progress...) is:

“Energy management with uncertain inputs:
which algorithms ?
Open source benchmark based on a solar home”

Here is the model of the solar home (power flows)

solar home control bench (power flows model )

I've also a first translation of the abstract:

“Optimal management of energy systems requires strategies based on optimization algorithms. The range of tools is wide, and each tool calls on various theories (convex, dynamic, stochastic optimization...) which each require a period of appropriation ranging from a few days to several months.

It is therefore difficult for the novice energy management practitioner to
understand the main characteristics of each approach so we can compare them objectively and finally find the method or methods best suited to a given problem.

To facilitate an objective and transparent comparison, we propose an exemplary and simple energy management problem: a solar house with photovoltaic production and storage. After justifying the sizing of the system, we illustrate the benchmark by a first comparison of some energy management methods (heuristic rule, MPC and anticipatory optimization). In particular, we highlight the effect of the uncertainty of solar production on performance.

This benchmark, including the management methods described,
is open source, accessible online and multi-language (Python, Julia and Matlab).”

Access to the benchmark

The entire source code and the data (an extract from the Solar home electricity dataset by Ausgrid) is available on GitHub:

https://github.com/pierre-haessig/solarhome-control-bench/

As of now, only rather simple energy management methods are implemented, but I'd like to add some kind of stochastic MPC (once I've clarified what this really means), and later Stochastic Dynamic Programming.

Solar home electricity dataset by Ausgrid

I've started exploring an open dataset that should be very useful for research on residential solar energy (for example studying energy self-consumption with a PV-battery system).

My Python code to manipulate this dataset and some preliminary studies (simple statistics) is freely available on Github: https://github.com/pierre-haessig/ausgrid-solar-data. This data exploration code is extensively using pandas (for slicing, grouping, resampling, etc.).

About the dataset

The “Solar home electricity dataset” is made available by Ausgrid, an Australian electric utility which operates the distribution grid in Sidney and nearby areas.

This dataset contains a pretty rich record:

  • electricity consumption and the PV production
  • of 300 customers (in Sidney and its area),
  • over three years (July 2010 to June 2013),
  • with a 30 minutes timestep.

Here is a small extract of the PV production for 3 randomly chosen customers, over 3 days in July 2011:

https://raw.githubusercontent.com/pierre-haessig/ausgrid-solar-data/master/PV%20production%202011-07%2001-03.png

Many more plots and statistics are given in the Jupyter notebooks available on the Github repository.

Locating postcodes (geocoding)

Also, the exploration of this dataset was an occasion to discover the Google Maps Geocoding API. Indeed, the location of each anonymous customer is given by a postcode only. To enable quantitative study of the spatiotemporal pattern in PV production, I've tried to locate this postcodes. The Python code for this is in the Postcodes location.ipynb Notebook. Map plotting is done with cartopy.

Extracted from this notebook, here is an overview of the locations of the postcodes present in the dataset (in Australia, NSW). Red rectangles are the boundaries of each postcode, as returned by Google maps (small in urban areas along the coast, gigantic otherwise).

Installing JModelica on Ubuntu 16.04

This a copy of  the post I've just placed  JModelica forum: http://jmodelica.org/27775

I just wanted to share my experience building JModelica from source (trunk version) on Ubuntu 16.04. After several attempts, I think I've a quite streamlined procedure. Both commands "make install" and "make casadi_interface" are successful. In the end, I was also able to successfully run "make test" (although it took 58 min on the virtual machine).

For most aspects, I've followed the user guide, but using updated versions of the dependencies (see after), in particular using Java 8 instead of 6 (not available anymore).

My main departure from user guide is that I used Ipopt from Ubuntu instead of compiling from source, as recommended. This works well enough (functionally speaking, since I've no ideas about the performance). In the end, I think this approach is simpler/quicker (especially since it also saves from recompiling blas). However, I had to manually patch one Ipopt header for the compilation to work (but then this modification may not be needed when just using the JModelica binary). See below.

It would be also interesting to try with Ipopt compiled from source, but for the moment I've no incentive to replace something that I just got to work!

Dependency versions

Here are the list of versions of the main dependencies (some packages, like g++, come already pre-installed with a fresh Ubuntu install):

g++: 5.4.0
subversion: 1.9.3
gfortran: 5.4.0
cmake: 3.5.1
swig: 3.0.8
ant: 1.9.6
openjdk-8-jdk (8u91) , instead of openjdk-6
python-dev: 2.7.11
python-numpy: 1.11
python-scipy: 0.17
python-matplotlib: 1.5.1
cython: 0.23.4
python-lxml: 3.5.0
python-nose: 1.3.7
python-jpype: 0.5.4.2 (alternative: there is a fork JPype1, on PyP, which seems more up to dateI. not tested)
zlib1g-dev: 1.2.8
libboost-dev: 1.58
jcc: 2.21 (from Ubuntu rather than from PyPI, as suggested in the user guide, since both versions are the same)

extra : python-pip : 8.1.1

For ipython, I'm NOT using 2.4.1 from Ubuntu, but rather 5.1 from PyPI (pip install)

Also, for using Ipopt from Ubuntu, ipopt + some extra headers are needed:

coinor-libipopt1v5: 3.11.9
coinor-libipopt-dev: 3.11.9
libblas-dev: 3.6.0
liblapack-dev: 3.6.0

Patching Ubuntu's Ipopt header

As I said, I had to patch one Ipopt header. The starting point is this compilation error in the "make install" before the patch:

libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../../../JMI/src -I../.. -Wall -I/usr/include/coin -DJMI_AD=JMI_AD_NONE -g -I/home/modelica/JModelica_trunk/build/sundials_install/include -fPIC -g -O2 -MT libjmi_solver_la-jmi_opt_coll_ipopt.lo -MD -MP -MF .deps/libjmi_solver_la-jmi_opt_coll_ipopt.Tpo -c ../../../JMI/src/jmi_opt_coll_ipopt.cpp  -fPIC -DPIC -o .libs/libjmi_solver_la-jmi_opt_coll_ipopt.o
In file included from /usr/include/coin/IpJournalist.hpp:15:0,
                 from /usr/include/coin/IpIpoptApplication.hpp:26,
                 from ../../../JMI/src/jmi_opt_coll_ipopt.cpp:21:
/usr/include/coin/IpSmartPtr.hpp:18:4: error: #error "don't have header file for stddef"
 #  error "don't have header file for stddef"
    ^
Makefile:1240: recipe for target 'libjmi_solver_la-jmi_opt_coll_ipopt.lo' failed
make[1]: *** [libjmi_solver_la-jmi_opt_coll_ipopt.lo] Error 1
make[1]: Leaving directory '/home/modelica/JModelica_trunk/build/JMI/src'
Makefile:417: recipe for target 'install-recursive' failed
make: *** [install-recursive] Error 1

So the error comes from line 18 in IpSmartPtr.hpp, in the /usr/include/coin/ directory. I've modified this line by taking inspiration from similar line in IpJournalist.hpp (I don't remember the source of this idea, because I did this back in April. Sorry if I forgot to credit some other source). So I changed line 18 of IpSmartPtr.hpp to remove the error and instead force inclusion of <cstdarg>, as done in IpJournalist.hpp. This is the line after modification (with my initials to remember that this is a dirty patched line, but the rest of the comment really comes from IpJournalist.hpp )

#  include <cstdarg>  // if this header is included by someone who does not define HAVE_CSTDARG or HAVE_STDARG, let's hope that cstdarg is available. PH 2016-09

Now with this modification, I think somebody else should be able to reproduce the build.

Summary of the commands

This list of commands are just adapted from the user guide. Only the include path for Ipopt is specific (using Ubuntu package).

Install directory:

$ mkdir /home/modelica/Programmes/JModelica

in the source tree:

$ mkdir build
$ cd build/
$ ../configure --prefix=/home/modelica/Programmes/JModelica --with-ipopt=/usr
$ make install

then casadi interface

$ make casadi_interface

→ should just work!

Enjeux énergétiques par le prisme d'objets du quotidien

Dans le cadre de la Journée Nouvelles Technologies du CCMO, j'ai fait une présentation intitulée "Enjeux énergétiques par le prisme d'objets du quotidien". Voici les diapos (pdf).

Cette présentation était à destination d'enseignants de science en lycée. L'idée était d'utiliser des objets de la vie courante (bouilloire, frigo) pour rendre plus concrètes les notions d'énergie et de puissance. Exemple : une batterie de téléphone portable ≈ 5 Wh. En matérialisant ces notions, on peut aider à construire un regard actif/critique sur l'enjeu citoyen qu'est l'énergie.

La présentation comprend aussi une comparaison entre 2 moyens de production autonome d'électricité : groupe électrogène et panneaux photovoltaïques. Coût d'investissement faible pour le groupe électrogène, mais coût élevé du carburant. C'est bien sûr l'inverse pour le solaire : investissement onéreux, mais "carburant" gratuit.

Pour l'analyse de la ressource solaire, j'ai utilisé les graphiques et les données de PVGIS (Photovoltaic Geographical Information System).

Ressource solaire à Rennes d'après PVGIS
Ressource solaire à Rennes d'après PVGIS

Au final en moyenne annuelle, on peut espérer 4 kWh/m²/jour soit 1450 kWh/m²/an. Par rapport au sud de la France, Rennes n'est donc pas si défavorisée, puisqu'on n'atteint en Côte d'Azur "que" 2000 kWh/m²/an (cf. la jolie carte du potentiel de solaire de la France).

Par ailleurs, en préparant cette présentation, je suis tombé sur le blog richement alimenté "Do the Math" de Tom Murphy, professeur à University of California, San Diego, et qui creuse justement la question des enjeux de l'énergie.

PyCOZIR

I've started to play with the pretty nice Cozir CO2 sensor from GSS. This relates to research projects on air quality control.

For testing purpose, the sensor is connected to my computer through a USB-serial converter cable. In order to communicate  with the sensor (e.g. grab the CO2 concentration data), I've written a bit of Python code to wrap the low-level ASCII communication protocol into a higher level, more compact API.

For example, instead of exchanging byte codes, reading the temperature becomes:

>>> from cozir import Cozir
>>> c = Cozir('/dev/ttyUSB0')
>>> c.read_temperature()
20.5

For people that might be interested, all the details and the code are on the Github repository: https://github.com/pierre-haessig/pycozir

Also, the repo contains a basic logging program that can be customized to anyone's needs. Tested for now with Python 2.7, on Linux and Windows.

 

PowerTech 2015 article online

The article I submitted to PowerTech 2015 (Eindhoven, June 2015) has been accepted. I've put online the pdf manuscript. It is an author version, since the copyright will be transfered to IEEE.

Title: "Energy Storage Control with Aging Limitation"

[Update July 7, 2015] presentation slides are now also available.

Abstract

Energy Storage Systems (ESS) are often proposed to mitigate the fluctuations of renewable power sources like wind turbines. In such a context, the main objective for the ESS control (its energy management) is to maximize the performance of the mitigation scheme.

However, most ESS, and electrochemical batteries in particular, can only perform a limited number of charge/discharge cycles over their lifetime. This limitation is rarely taken into account in the optimization of the energy management, because of the lack of an appropriate formalization of cycling aging.

We present a method to explicitly embed a limitation of cycling aging, as a constraint, in the control optimization. We model cycling aging with the usual ``exchanged energy'' counting method. We demonstrate the effectiveness of our aging-constrained energy management using a publicly available wind power time series. Day-ahead forecast error is minimized while keeping storage cycling just under an acceptable target level.