The problem: a certain data is publicly available only for the last ten years in the form of data table.
I'd like to complete it with the largest time frame's data.

But: there are many charts with "all time" data on the Web.

The solution: use Recurve!

This is done in 3 easy steps (OK, after 3 preliminary steps 😉):

-2) Download and install GNAT from here.

-1) Download GID (Generic Image Decoder) from here or here, or get it as a crate from Alire (crate description here)

0) Build the GID tools (Windows: double-click "build.cmd" or command-line: "gprbuild -p -P gid" or Alire: "alr get gid", then "alr run" (details in the doc)).

1) Open the "test" directory.

2) Windows and probably others: drag-&-drop your chart image on recurve[.exe], or command-line: "[./]recurve pic.ext"

Now, we can see the various data reconstruction steps. Starting point:

• Turn x values from pixels into dates, pick correct curves at the ends (the moving average 20 days in the chart happened to have the same colour as the values).
  
  
  
  
• De-logarithm-ize the y values, have index points instead of pixels.
  
  
  
• Switch to the actual data since March 2011.
  
  
  

Done!

Yet another use of the Recurve tool available with GID @ https://gen-img-dec.sourceforge.io/ or https://github.com/zertovitch/gid : the long-term reconstruction of the Harpex index, which tracks worldwide container shipping prices.

The Harpex chart since 2001 is available for free on the Harper Petersen & Co Web site, but the data is freely available only for the last 3 years.

Solution: use the Recurve tool for digitizing the long-term chart till 2018.

All in all (two digitized charts, plus data since 2018):

We've caught the "all RGB" virus!

The allRGB site shows many pictures with a very special characteristic: each colour (of the standard 8-bit-per-channel rendering) appears in one and only one pixel.
With 8-bit-per-channel rendering, you have 2^8 = 256 values of red, 256 values of green, 256 values of blue.
As a result, you have 256 * 256 * 256 = 16,777,216 different colors.

Of course, we could not resist making our own "all RGB" demo around GID (the Generic Image Decoder).
You can find now in the test directory in the GID project (available on SourceForge here and GitHub here), a new all_rbg.adb program that transforms an image into a similar image in which each pixel has a different colour and all 8-bit-per-channel colours are displayed.

Actually, our algorithm goes differently: it starts with a trivial, deterministic "all RGB" image...

...makes it uniform by swapping pixels randomly...

...then modifies it progressively in order to make it look like the image you have given.
This is done, at each step, by choosing two pixels randomly, and swapping them, or not, depending on how the pair resembles the given image.
The invariant of the algorithm is that the image, during the whole process, stays "all RGB".

Here is an example (all images are reduced for this blog post). Original photography (pictured: a village called Ottrott in Alsace):

Here, a detail:

QOI (the Quite OK Image Format, home page here) is a very simple raster image format with a decent lossless compression and an extremely good performance - a direct consequence of its simplicity and its compression features.

It was clear from the first sight on that format that it was urgent to add it to GID, the Generic Image Decoder 😎. GID is free, open-source, available on SourceForge here and GitHub here.

Here, a few examples of QOI test images decoded by GID (with its default background for transparency):

The cool ideas in the QOI format are

• a "moving palette" - a list of recently shown colours, that is updated during the encoding or decoding of the pixels; the indexing is done with a hash function
  
• a shortcut encoding for slightly different colours from a pixel to the next
• a shortcut encoding for slightly different brightness from a pixel to the next.

Transparency, in the form of levels from 0 to 255, (the alpha channel) is supported.

Add to it a run-length encoding that fits well surfaces with identical colours and transparency, and the format remains so simple that you can squeeze its detailed specification on a single, readable, A4 page!

Click to enlarge