❌ About FreshRSS

Normal view

There are new articles available, click to refresh the page.
Before yesterdayNews from the Ada programming language world

Unlocking the Power of OpenAI in Ada programs

1 October 2023 at 16:33
[Ada/openai-img.jpg](Ada/openai-img.jpg)

The OpenAI(https://openai.com/) provides a service that can be queried using REST requests. The service API can be classified as follows:

  • OpenAI's GPT (generative pre-trained transformer) is the well-known text generation based on text inputs.
 It gives access to several AI models that have been trained to understand natural language and code.
  • Image generation gives access to the DALL.E(https://platform.openai.com/docs/models/dall-e) for both
  •  generating images based on a text description, or, modify an existing image based on a text description.
    
  • Speech to text gives access to the Whisper model(https://openai.com/research/whisper) that converts
  •  audio records into text (several languages are supported).
    
  • OpenAIҀ™s text embeddings measure the relatedness of text strings.
  • For a detailed description of these API have a look at the OpenAI API Introduction(https://platform.openai.com/docs/introduction) document. To use these APIs, you'll first need to register on their service and get an API key that will grant access to the operations.

    The library was generated by the OpenAPI(https://github.com/OpenAPITools/openapi-generator) code generator from the OpenAPI description of the OpenAI service and it uses the OpenAPI Ada(https://github.com/stcarrez/swagger-ada) library to make HTTP requests and perform JSON serialization and de-serialization. Each OpenAI operation is made available through an Ada procedure which takes care of building the request that was filled within some Ada record, submitting it to the OpenAI server and extract the response in another Ada record. Every request submitted to the OpenAI server is therefore strongly typed!

      1. Setup

    To use the library, you should use Alire to setup your project and use:

    ``` alr index add git+https://gitlab.com/stcarrez/awa-alire-index.git name awa alr with openai ```

    For the HTTP connection, you can either use AWS(https://github.com/AdaCore/aws) or curl(https://curl.se/) and run one of the following commands:

    ``` alr with utilada_curl alr with utilada_aws ```

      1. Initialization

    First, make sure you import at least the following Ada packages:

    ``` with Util.Http.Clients.Curl; -- replace Curl with AWS if needed with OpenAPI.Credentials.OAuth; with OpenAI.Clients; with OpenAI.Models; ```

    If you want to use curl(https://curl.se/), the initialization should use the following:

    ``` Util.Http.Clients.Curl.Register; ```

    But if you want to use AWS(https://github.com/AdaCore/aws), you will initialize with:

    ``` Util.Http.Clients.AWS.Register; ```

    After the initialization is done, you will declare the `OpenAI` client instance to access the API operations. The OpenAI(https://openai.com/) service uses an OAuth bearer API key to authenticate requests made on the server. We will need an `OAuth2_Credential_Type` instance represented by `Cred` below.

    ``` Cred : aliased OpenAPI.Credentials.OAuth.OAuth2_Credential_Type; Client : OpenAI.Clients.Client_Type; ```

      1. Credential setup

    For the credential setup you will first need to get your access key from your account. Once you have your key as a `String`, you can configure the `Cred` object and tell the client connection entry point which credentials to use:

    ``` Api_Key : constant String := ...;

      Cred.Bearer_Token (Api_Key);
      Client.Set_Credentials (Cred'Unchecked_Access);
    

    ```

      1. OpenAPI client setup

    The last step necessary before you can make requests, is to setup the server base URL to connect for the REST requests:

    ```

     Client.Set_Server ("https://api.openai.com/v1");
    

    ```

      1. API for chat

    The `Create_Chat` is the main operation for the conversation chat generation. The request is represented by the `ChatRequest_Type` type which describes the OpenAI chat model that must be used and the query parameters. The request can be filled with a complete conversation chat which means it is possible to call it several times with previous queries and responses to proceed in the chat conversation.

    ``` C : OpenAI.Clients.Client_Type; Req : OpenAI.Models.ChatRequest_Type; Reply : OpenAI.Models.ChatResponse_Type; ...

      Req.Model := OpenAPI.To_UString ("gpt-3.5-turbo");
      Req.Messages.Append ((Role => Openapi.To_Ustring ("user"),
                            Content => Prompt,
                            others => <>));
      Req.Temperature := 0.3;
      Req.Max_Tokens := (Is_Null => False, Value => 1000);
      Req.Top_P := 1.0;
      Req.Frequency_Penalty := 0.0;
      Req.Presence_Penalty := 0.0;
      C.Create_Chat (Req, Reply);
    

    ```

    Upon successful completion, we get a list of choices in the reply that contains the text of the conversation. You can iterate over the list with the following code extract. Beware that the returned string is using UTF-8 encoding and it may need a conversion to a `Wide_Wide_String` if necessary.

    ```

      for Choice of Reply.Choices loop
         declare
            Utf8 : constant String := OpenAPI.To_String (Choice.Message.Content);
         begin
            Put_Line (Ada.Strings.UTF_Encoding.Wide_Wide_Strings.Decode (Utf8));
         end;
      end loop;
    

    ```

    The complete chat example is available at: OpenAI Chat(https://gitlab.com/stcarrez/openai-chat) .

      1. Generating an image

    The image generation is based on the DALL.E(https://platform.openai.com/docs/models/dall-e) model generator. The creation is made by populating a request object, making the call (which is an HTTP POST) and getting the result in a response. Both request and response are represented by full Ada types and are strongly typed:

    The request contains a prompt string which must be provided and is the textual description of the image to create. An optional parameter allows to control the number of images which are created (from 1 to 10). Another optional parameter controls the dimension of the final image. The OpenAI API limits the possible values to: `256x256`, `512x512` and `1024x1024`. The image creation is represented by the `Create_Image` procedure and we can call it with our request instance:

    ``` Req : OpenAI.Models.CreateImagesRequest_Type; Reply : OpenAI.Models.ImagesResponse_Type; ...

      Req.Prompt := Prompt;
      Req.N := (Is_Null => False, Value => 3);
      Req.Size := (Is_Null => False, Value => "512x512");
      C.Create_Image (Req, Reply);
    

    ```

    Once it has finished, it produces a response which basically contains a list of URLs for each generated image.

    ```

      for Url of Reply.Data loop
         if not Url.Url.Is_Null then
            Ada.Text_IO.Put_Line (OpenAPI.To_String (Url.Url.Value));
         end if;
      end loop;
    

    ```

    The complete image generation example is available at: OpenAI Image Generation(https://gitlab.com/stcarrez/openai-image) .

    For more information about the image creation, have a look at the OpenAI Images API reference(https://platform.openai.com/docs/api-reference/images).

      1. Conclusion

    The Ada OpenAI(https://gitlab.com/stcarrez/ada-openai) library is eagerly awaiting your Ada programming creativity. You can try using the chat generation example to ask the AI to generate some Ada code, but you'll likely be disappointed by the poor quality of Ada programs that OpenAI(https://openai.com/) generates. However, you may still find some small benefits in using it across different domains. I encourage you to give it a try for your enjoyment and pleasure.

    Ada BFD 1.3.0

    20 August 2023 at 14:14
    [Ada/ada-bfd-1.3.jpg](Ada/ada-bfd-1.3.jpg)
      1. Integration with Alire

    For Linux users only, the Ada BFD(https://github.com/stcarrez/ada-bfd) has an associated Alire crate which allows you to use it easily. To get access to the Alire crate, you should add the AWA Alire index(https://github.com/stcarrez/awa-alire-index) in your Alire configuration as follows:

    ``` alr index add=https://github.com/stcarrez/awa-alire-index.git name awa ```

    Then, you can get access to the crate by using

    ``` alr with bfdada ```

    Let's see how to use this library...

      1. Declarations

    The Ada BFD(https://github.com/stcarrez/ada-bfd) library provides a set of Ada bindings that give access to the BFD library. A binary file such as an object file, an executable or an archive is represented by the `Bfd.Files.File_Type` limited type. The symbol table is represented by the `Bfd.Symbols.Symbol_Table` limited type. These two types hold internal data used and managed by the BFD library.

    ```ada with Bfd.Files; with Bfd.Sections; with Bfd.Symbols; ...

     File    : Bfd.Files.File_Type;
     Symbols : Bfd.Symbols.Symbol_Table;
    

    ```

      1. Opening the BFD file

    The first step is to use the `Open` procedure to read the object or executable file whose path is given as argument. The `File_Type` parameter will be initialized to get access to the binary file information. The `Check_Format` function must then be called to let the BFD library gather the file format information and verify that it is an object file or an executable.

    ```ada Bfd.Files.Open (File, Path, ""); if Bfd.Files.Check_Format (File, Bfd.Files.OBJECT) then

       ...
    

    end if; ```

    The `File_Type` uses finalization so that it will close and reclaim resources automatically.

      1. Loading the symbol table

    The symbol table is loaded by using the `Read_Symbols` procedure.

    ```ada

      Bfd.Symbols.Read_Symbols (File, Symbols);
    

    ```

    The resources used by the symbol table will be freed when the symbol table instance is finalized.

      1. Find nearest line

    Once the symbol table is loaded, we can use the `Find_Nearest_Line` function to find the nearest line of a function knowing some address. This is almost a part of that function that the addr2line (1)(https://www.man7.org/linux/man-pages/man1/addr2line.1.html) command is using.

    ```ada File_Name, Func_Name : Ada.Strings.Unbounded.Unbounded_String; Text_Section : Bfd.Sections.Section; Line : Natural; Pc : constant Bfd.Vma_Type := ...; ...

      Text_Section := Bfd.Sections.Find_Section (File, ".text");
      Bfd.Symbols.Find_Nearest_Line (File    => File,
                                     Sec     => Text_Section,
                                     Symbols => Symbols,
                                     Addr    => Pc,
                                     Name    => File_Name,
                                     Func    => Func_Name,
                                     Line    => Line);
    

    ```

    One tricky aspect of using `Find_Nearest_Line` is the fact that the address we are giving must **sometimes** be converted to an offset within the text region. With Address space layout randomization (ASLR)(https://en.wikipedia.org/wiki/Address_space_layout_randomization) a program is mapped at a random address when it executes. Before calling `Find_Nearest_Line`, we must subtract the base address of the memory region. We must now find the virtual address of the start of the text region that is mapped in memory. While the program is running, you can find the base address of the program by looking at the `/proc/self/maps` file. This special file indicates the list of memory regions used by the process with the addresses, flags and other information. Without ASLR, the program is almost always loaded at the `0x00400000` address.

    ``` 00400000-007f9000 r-xp 00000000 fd:01 12067645 /home/... 009f8000-009fa000 r--p 003f8000 fd:01 12067645 /home/... 009fa000-00a01000 rw-p 003fa000 fd:01 12067645 /home/... ```

    But when it is mapped at a random address, we get a different address each time the program is launched:

    ``` 55d5983d9000-55d598592000 r--p 00000000 fc:02 1573554 /... 55d598592000-55d599376000 r-xp 001b9000 fc:02 1573554 /... 55d599376000-55d5997ed000 r--p 00f9d000 fc:02 1573554 /... 55d5997ee000-55d5998bb000 r--p 01414000 fc:02 1573554 /... 55d5998bb000-55d5998c6000 rw-p 014e1000 fc:02 1573554 /... ```

    In that case, the value to use it the first address of first `r--p` region associated with the program (here `0x55d5983d9000`).

    Another method to know the virtual base address is to use the dl_iterate_phdr (3)(https://man7.org/linux/man-pages/man3/dl_iterate_phdr.3.html) function and look at the shared objects which are loaded. This function must be executed by the program itself: it gets as parameter a callback function which is called for each loaded shared object and a data parameter that will be passed to the callback.

    ```

    1. include <dlfcn.h>

    static int dl_callback (struct dl_phdr_info* info, size_t size, void* data) {

     /* VM base address is: info->dlpi_addr */
     return 0;
    

    } ...

      dl_iterate_phdr (dl_callback, 0);
    

    ```

    When the callback is called, you can get the name of the shared object by looking at `info->dlpi_name` and the virtual base address by looking at `info->dlpi_addr`.

    Ada BFD(https://github.com/stcarrez/ada-bfd) is a very specific library that is not always easy to use due to the complexity of binary program representation (ELF, DWARF, ...) and program execution. It is however used in very specific contexts such as the Muen Separation Kernel(https://muen.codelabs.ch/) and the Memory Analysis Tool(https://github.com/stcarrez/mat).

    Reentrant scanner and parser with Aflex and Ayacc

    14 May 2023 at 19:02
    [Aflex and Ayacc](Ada/Aflex-Ayacc-code.jpg)
      1. What's new in Aflex 1.6

    - Support the flex options `%option output`, `%option nooutput`, `%option yywrap`, `%option noinput`,

     `%option noyywrap`, `%option unput`, `%option nounput`, `%option bufsize=NNN` to better control the
     generated `_IO` package.
    
    - Aflex(https://github.com/Ada-France/aflex) templates provide more control for tuning the code generation and
     they are embedded with [Advanced Resource Embedder](https://gitlab.com/stcarrez/resource-embedder)
    
    - Support to define Ada code block in the scanner that is inserted in the generated scanner - New option -P to generate a private Ada package for DFA and IO - New directive `%option reentrant` and `%yyvar` to generate a recursive scanner - New directive `%yydecl` to allow passing parameters to `YYLex`
     or change the default function name
    

    Example of `%option` directives to tell Aflex(https://github.com/Ada-France/aflex) to avoid generating several function or procedures and customize the buffer size.

    ```Ada %option nounput %option noinput %option nooutput %option noyywrap %option bufsize=1024 ```

    The tool supports some code block injection at various places in the generated scanner. The code block has the following syntax where `<block-name>` is the name of the code block:

    ```Ada %<block-name> {

     -- Put Ada code here
    

    } ```

    The `%yytype` code block can contain type declaration, function and procedure declarations. It is injected within the `YYLex` function in the declaration part. The `%yyinit` code block can contain statements that are executed at beginning of the `YYLex` function. The `%yyaction` code block can contain statements that are executed before running any action. The `%yywrap` code block can contain statements which are executed when the end of current file is reached to start parsing a next input.

      1. What's new in Ayacc 1.4

    - Support the Bison `%define variable value` option to configure the parser generator - Support the Bison `%code name { ... }` directive to insert code verbatim into the output parser - Recognize some Bison variables `api.pure`, `api.private`, `parse.error`, `parse.stacksize`,

     `parse.name`, `parse.params`, `parse.yyclearin`, `parse.yyerrok`, `parse.error`
    
    - New option `-S skeleton` to allow using an external skeleton file for the parser generator - Ayacc(https://github.com/Ada-France/ayacc) templates provide more control for tuning the code generation and
     they are embedded with [Advanced Resource Embedder](https://gitlab.com/stcarrez/resource-embedder)
    
    - New option `-P` to generate a private Ada package for the tokens package - Improvement to allow passing parameters to `YYParse` for the grammar rules - New `%lex` directive to control the call of `YYLex` function - Fix #6: ayacc gets stuck creating an infinitely large file after encountering a comment in an action

    The generator supports two code block injections, the first one `decl` is injected in the `YYParse` procedure declaration and the `init` is injected as first statements to be executed only once when the procedure is called. The syntax is borrowed from the Bison parser:

    ```Ada %code decl {

      -- Put Ada declarations
    

    } %code init {

      -- Put Ada statements
    

    } ```

    Some other Bison like improvements have been introduced to control the generation of the parser code.

    ``` %define parse.error true %define parse.stacksize 256 %define parse.yyclearin false %define parse.yyerrok false %define parse.name MyParser ```

      1. How to use

    The easiest way to use Ayacc(https://github.com/Ada-France/ayacc) and Aflex(https://github.com/Ada-France/aflex) is to use Alire(https://github.com/alire-project/alire), get the sources, build them and install them. You can do this as follows:

    ``` alr get aflex cd aflex_1.6.0_b3c21d99 alr build alr install alr get ayacc cd ayacc_1.4.0_c06f997f alr build alr install ```

    • UPDATE*: the `alr install` command is available only with Alire(https://github.com/alire-project/alire) 2.0.

    Using these tools is done in two steps:

    1. a first step to call `aflex` or `ayacc` command with the scanner file or grammar file, 2. a second step to call `gnatchop` to split the generated file in separate Ada files

    For example, with a `calc_lex.l` scanner file, you would use:

    ``` aflex calc_lex.l gnatchop -w calc_lex.ada ```

    And with a `calc.y` grammar file:

    ``` ayacc calc.y gnatchop -w calc.ada ```

    To know more about how to write a scanner file or grammar file, have a look at Aflex 1.5 and Ayacc 1.3.0(https://blog.vacs.fr/vacs/blogs/post.html?post=2021/12/18/Aflex-1.5-and-Ayacc-1.3.0) which explains more into details some of these aspects.

      1. Highlight on reentrancy

    By default Aflex(https://github.com/Ada-France/aflex) and Ayacc(https://github.com/Ada-France/ayacc) generate a scanner and a parser which use global variables declared in a generated Ada package. These global variables contain some state about the scanner such as the current file being scanned. The Ayacc(https://github.com/Ada-France/ayacc) parser generates on its side two global variables `YYLVal` and `YYVal`.

    Using global variables creates some strong restrictions when using the generated scanner and parser: we can scan and parse only one file at a time. It cannot be used in a multi-thread environment unless the scan and parse is protected from concurrent access. We cannot use easily some grammars that need to recurse and parse another file such as an included file.

        1. Reentrant scanner

    The reentrant scanner is created by using the `-R` option or the `%option reentrant` directive. The scanner will then need a specific declaration with a context parameter that will hold the scanner state and variables. The context parameter has its type generated in the `Lexer_IO` package. The `%yydecl` directive in the scanner file must be used to declare the `YYLex` function with its parameters. By default the name of the context variable is `Context` but you can decide to customize and change it to another name by using the `%yyvar` directive.

    ``` %option reentrant %yyvar Context %yydecl function YYLex (Context : in out Lexer_IO.Context_Type) return Token ```

    When the `reentrant` option is activated, Aflex(https://github.com/Ada-France/aflex) will generate a first `Context_Type` limited type in the `Lexer_DFA` package and another one in the `Lexer_IO` package. The generator can probably be improved in the future to provide a single package with a single type declaration. The `Lexer_DFA` package contains the internal data structures for the scanner to maintain its state and the `Lexer_IO` package holds the input file as well as the `YYLVal` and `YYVal` values.

        1. Reentrant parser

    On its side, Ayacc(https://github.com/Ada-France/ayacc) uses the `YYLVal` and `YYVal` variables. By default, it generates them in the `_tokens` package that contains the list of parser symbols. It must not generate them and it must now use the scanner `Context_Type` to hold them as well as the scanner internal state. The setup requires several steps:

    1. The reentrant parser is activated by using the `%define api.pure`

      directive similar to the [bison %define](https://www.gnu.org/software/bison/manual/html_node/_0025define-Summary.html).
    

    2. The `%lex` directive must be used to define how the `YYLex` function must be called since it now has some

      context parameter.
    

    3. The scanner context variable must be declared somewhere, either as parameter to the `YYParse`

      procedure or as a local variable to `YYParse`.  This is done using the new `%code decl` directive
      and allows to customize the local declaration part of the `YYParse` generated procedure.
    

    4. We must give visibility of the `YYLVal` and `YYVal` values defined in the scanner context variable.

      Again, we can do this within the `%code decl` directive.
    

    A simple reentrant parser could be defined by using:

    ```Ada %define api.pure true %lex YYLex (Scanner) %code decl {

         Scanner : Lexer_IO.Context_Type;
         YYLVal  : YYSType renames Scanner.YYLVal;
         YYVal   : YYSType renames Scanner.YYVal;
    

    } ```

    However, this simple form is not really useful as you may need to open the file and setup the scanner to read from it. It is probably better to pass the scanner context as parameter to the `YYParse` procedure. For this, we can use the `%define parse.params` directive to control the procedure parameters. The reentrant parser is declared as follows:

    ```Ada %lex YYLex (Scanner) %define api.pure true %define parse.params "Scanner : in out Lexer_IO.Context_Type" %code decl {

         YYLVal : YYSType renames Scanner.YYLVal;
         YYVal  : YYSType renames Scanner.YYVal;
    

    } ```

    To use the reentrant parser and scanner, we only need to declare the scanner context, open the file by using the `Lexer_IO.Open_Input` procedure and call the `YYParse` procedure as follows:

    ```Ada

     Scanner : Lexer_IO.Context_Type;
     ...
       Lexer_IO.Open_Input (Scanner, "file-to-scan");
       YYParse (Scanner);
    

    ```

        1. Grammar examples:

    To have a more complete example of a reentrant parser, you may have a look at the following files:

    AWA 2.4.0

    7 August 2022 at 21:12

    The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.

    AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.

    A typical architecture of an AWA application is represented by the picture below:

    [Ada Web Application architecture](Ada/awa_architecture_overview.png)
      1. Using AWA with Alire

    To use AWA with Alire, you should first register a new index to get access to the AWA crates and its dependencies.

    ``` alr index add git+https://github.com/stcarrez/awa-alire-index name awa ```

    After this command, the `alr` command should give you access to the AWA crates and you can check that with the command:

    ``` alr search awa ```

    Once you have setup the Alire index, you can import the `awa` framework and the `dynamo` associated tool by using the following commands:

    ``` alr init demo cd demo alr with awa alr with dynamo alr with servletada_aws alr build ```

    Note, if you are using FreeBSD, you should probably use the following command to build (because the MariaDB shared libraries are installed in `/usr/local/lib/mariadb` directory):

    ``` alr build -- -largs -L/usr/local/lib/mariadb ```

    Once the `dynamo` tool is built, you can use it to setup your project by using the `create-project` command. To run the `dynamo` tool, you must run it by using the `alr exec` command because Alire will setup some environment variables and `PATH` that gives access to Dynamo and AWA configuration files.

    ``` alr exec dynamo create-project -l apache web demo [email protected] ```

    Change the `demo.gpr` GNAT project that was generated by `alr` and replace `demo.adb` by `demo-server.adb`. That later source has been generated by `dynamo` and it contains everything to setup, prepare and run the Web server. Once this is done, build the application:

    ``` alr build ```

    The Atlas AWA Demonstrator(https://github.com/stcarrez/atlas) is also available as Alire crates. Two versions are available, the `atlas` crate is using the Ada Web Server and the `atlas_ews` is using the Embedded Web Server. To play to Atlas, you can try one of the following commands:

    ``` alr get atlas

    1. or

    alr get altas_ews ```

      1. Debian packages

    Debian packages are also available for Ubuntu 20, Ubuntu 22 and Debian 11. The repository also includes a Debian package for Alire 1.2. Choose **one** of the following configuration and add it to your `/etc/apt/sources.list` configuration.

    ``` deb https://apt.vacs.fr/ubuntu-focal focal main

    1. or

    deb https://apt.vacs.fr/ubuntu-jammy jammy main

    1. or

    deb https://apt.vacs.fr/debian-bullseye bullseye main ```

    And you can run the following command to accept the signed packages:

    ``` wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add - ```

        1. AWA 2.4.0(https://github.com/stcarrez/ada-awa/releases/tag/2.4.0) Download: awa-2.4.0.tar.gz(http://download.vacs.fr/ada-awa/awa-all-2.4.0.tar.gz)
     - Add support for SQL queries embedded in applications with ARE
     - Fix #20: Do not use git:// protocol
     - New EasyMDE plugin to integrate the Easy Markdown Editor
     - Update AWA blog and AWA wiki to use the EasyMDE editor for Markdown
     - Use Dynamo 1.3.0, Ada Server Faces 1.5.0, Ada Servlet 1.6.0, OpenAPI Ada 0.6.0
     - Use Ada Wiki 1.4.0, Ada Database Objects 2.3.0
     - Use Ada Keystore 1.3.3, Ada EL 1.8.5, Ada Utility Library 2.5.0
    
        1. Dynamo 1.3.0(https://github.com/stcarrez/dynamo/releases/tag/1.3.0) Download: dynamo-1.3.0.tar.gz(http://download.vacs.fr/dynamo/dynamo-1.3.0.tar.gz)
     - Fix #5: Generated body does not compile when an enumeration of another UML package is used
     - Fix #7: No default type for SQL generation of a column that uses an enumeration
     - Fix #9: Option or configuration to disable some SQL generation
     - Fix #10: Definition of an UML datatype with a tagged value raises an exception
     - Fix #12: Avoid emitting a full qualified type name for types declared in the current package
     - Fix #16: Improvement in Markdown documentation generator
     - Fix #17: YAML parser: accessibility check failure
     - Fix #18: Generate database operation to reload an object
     - Fix #19: Add dynamo configuration through environment support
     - Fix #20: Give access to the man page from alire
     - Fix $21: Generated procedure Create is missing overriding keyword
    
        1. OpenAPI Ada 0.6.0(https://github.com/stcarrez/swagger-ada/releases/tag/0.6.0) Download: openapi-ada-0.6.0.tar.gz(http://download.vacs.fr/openapi-ada/openapi-ada-0.6.0.tar.gz)
     - Rename Swagger package into OpenAPI and provide a Swagger package for compatibility
     - Update the openapi generator to version 6.0.0
     - Add support for text/plain response
     - Add support to use external Ada types in OpenAPI definitions
     - Add support for multiple response types
     - Add support for binary responses
     - Add support for Ada enum generation for OpenAPI enums
     - Integrate Swagger UI v4.13.0
    
        1. Ada Server Faces 1.5.0(https://github.com/stcarrez/ada-asf/releases/tag/1.5.0) Download: ada-asf-1.5.0.tar.gz(http://download.vacs.fr/ada-asf/ada-asf-1.5.0.tar.gz)

    - New widget <w:progress> to display horizontal/vertical progress bars

        1. Ada Servlet 1.6.0(https://github.com/stcarrez/ada-servlet/releases/tag/1.6.0) Download: ada-servlet-1.6.0.tar.gz(http://download.vacs.fr/ada-servlet/ada-servlet-1.6.0.tar.gz)
        2.  - Fix #4: Alire servletada_aws GNAT project fails due to missing Naming rule
           - Fix #5: The Input_Line_Size_Limit parameter is not taken into account
           - Fix #6: GNAT configuration project is not correct to build with debugging
           - Fix #7: Constraint error raised when matching empty path routes
           - Fix #11: Support for Embedded Web Server
           - Fix #12: Support for multiple response types in REST operations
          
        1. Ada Security 1.4.1(https://github.com/stcarrez/ada-security/releases/tag/1.4.1) Download: ada-security-1.4.1.tar.gz(http://download.vacs.fr/ada-security/ada-security-1.4.1.tar.gz)
     - Fix Alire GNAT project to build in debug mode
     - Fix Security.Random that generates shorter random string
    
        1. Ada Database Objects 2.3.0(https://github.com/stcarrez/ada-ado/releases/tag/2.3.0) Download: ada-ado-2.3.0.tar.gz(http://download.vacs.fr/ada-ado/ada-ado-2.3.0.tar.gz)
     - Fix #4: Is_Loaded predicate operation is false when an object is not yet inserted in the database
     - Fix #5: Exception raised when SQLite Query_Statement is finalized if the SQL query was invalid
     - Fix #7: Update SQLite support to 3.31.1
     - Fix #8: Add SQlite busy handler to handle the SQLITE_BUSY error
     - Fix #9: Better handling of SQLITE_BUSY error
     - Fix #10: Error 'XML query file does not exist' when the query is loaded from a static embedded loader
    
        1. Ada Wiki Engine 1.5.0(https://github.com/stcarrez/ada-wiki/releases/tag/1.5.0) Download: ada-wiki-1.5.0.tar.gz(http://download.vacs.fr/ada-wiki/ada-wiki-1.5.0.tar.gz)
        2.  - Add support for Textile markup language
           - Rewrote the Markdown parser to better follow the Common Mark Specification
          
        1. Ada Utility Library 2.5.0(https://github.com/stcarrez/ada-util/releases/tag/2.5.0) Download: ada-util-2.5.0.tar.gz(http://download.vacs.fr/ada-util/ada-util-2.5.0.tar.gz)
     - New examples to illustrate the IO stream composition
     - New examples for JSON parser and Util.Beans.Objects
     - Add support to set environment variables when launching a process (without changing the current process environment!)
     - Add support to indent XML output streams
     - New package Util.Files.Rolling to provide a rolling file manager
     - New package Util.Beans.Objects.Iterators to easily iterate over objects
     - Add a new log appender to support rolling log files with size and time based policies
     - New operation Util.Files.Delete_Tree to delete a directory tree and work arround
       for GNAT bug gcc/63222 and gcc/56055
     - New operation Util.Files.Realpath to find the canonicalized absolute path of a file
     - New package Util.Blobs to hold binary content with reference counting
     - New package Util.Http.Headers to provide some HTTP helper operations
     - Add support for Blob in bean objects
     - Fix compilation on NetBSD 9.2
     - Fix compilation with AWS >= 22.0
    

    Ada development on FreeBSD 13.1

    5 August 2022 at 20:22
    • Warning:* this is full of hacks and I don't pretend to provide any complete detailed and completely reproducible steps for getting a new Ada compiler.

    Before proceeding, make sure you have `gmake` installed because the BSD make uses an old Makefile syntax and is not able to handle GNU specific Makefiles.

    ``` pkg install gmake ```

      1. Get gcc6-aux from an old FreeBSD installation

    First step is to get the `gcc6-aux` port as is from an old FreeBSD installation. Basically, a tar-gz of the `/usr/local/gcc6-aux` directory tree is enougth. Basically, what I did is to run a `tar cf /tmp/gcc.tar /usr/local/gcc6-aux` on an old FreeBSD and then extract that tar as is on the FreeBSD 13.1 system (Of course, it must be the same architecture).

      1. Build gcc 12

    Get the GCC 12 sources from https://gcc.gnu.org and extract them:

    ``` tar xzf gcc-12.1.0.tar.xz ```

    Building GCC must be made in another directory and we must also setup the `PATH` to give access to the old Ada compiler. You must setup and run the configure script as follows:

    ``` export PATH=/usr/local/gcc6-aux/bin:$PATH mkdir build cd build ../gcc-12.1.0/configure disable-nls enable-gnu-indirect-function enable-host-shared with-as=/usr/local/bin/as with-gmp=/usr/local with-ld=/usr/local/bin/ld with-system-zlib without-zstd enable-libada localstatedir=/var prefix=/build/gcc-12.1 build=x86_64-portbld-freebsd13.0 --enable-languages=c,ada,c++ ```

    The `with-as` and `with-ld` are important to use the correct `as` and `ld` commands. A subset of the above configure command is taken from the configure command that FreeBSD ports is using to build GCC 12.

    After the configure, run the `gmake` command:

    ``` gmake ```

    or to run several compilations in parallel use:

    ``` gmake -j4 ```

    While building, the compiler is built a first time with `gcc6-aux` and another time with itself. Building the Ada libraries failed for me with:

    ``` /home/ciceron/src/build/./gcc/xgcc -B/home/ciceron/src/build/./gcc/ -B/build/gcc-12.1/x86_64-portbld-freebsd13.0/bin/ -B/build/gcc-12.1/x86_64-portbld-freebsd13.0/lib/ -isystem /build/gcc-12.1/x86_64-portbld-freebsd13.0/include -isystem /build/gcc-12.1/x86_64-portbld-freebsd13.0/sys-include -fchecking=1 -c -g -O2 -m32 -fpic -W -Wall -gnatpg -nostdinc -m32 s-exnllf.adb -o s-exnllf.o s-exnllf.ads:38:04: warning: in instantiation at s-dorepr.adb:82 enabled by default s-exnllf.ads:38:04: warning: in instantiation at s-exponr.adb:54 enabled by default s-exnllf.ads:38:04: warning: "Temp" overlays smaller object enabled by default s-exnllf.ads:38:04: warning: program execution may be erroneous enabled by default s-exnllf.ads:38:04: warning: size of "Temp" is 96 enabled by default s-exnllf.ads:38:04: warning: size of "Rep64" is 64 enabled by default gmake9: *** ../gcc-interface/Makefile:301: s-exnllf.o Error 1 ```

    This error is caused by some inconsistency between the floating point mantissa size returned by `Machine_Mantissa'` Ada predicate and the size of the floating point number. After hacking the `gcc/ada/libgnat/s-dorepr.adb` I've managed to build the Ada libraries.

    After a successful build, installation in `/build/gcc-12.1` is done with:

    ``` gmake install ```

      1. Build gprbuild

    Yet, another challenge is building gprbuild(https://github.com/AdaCore/gprbuild). First, get the following sources:

    ``` git clone https://github.com/AdaCore/xmlada.git git clone https://github.com/AdaCore/gprconfig_kb git clone https://github.com/AdaCore/gprbuild.git ```

    Then run the boostrap script from `gprbuild` and make sure to setup your PATH to use the new GCC Ada compiler:

    ``` export PATH=/build/gcc-12.1/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin cd gprbuild ./bootstrap.sh with-xmlada=../xmlada with-kb=../gprconfig_kb --prefix=./bootstrap ```

    Then, build again and install in the same location as GCC:

    ``` gmake prefix=/build/gcc-12.1 SOURCE_DIR=`pwd` setup gmake all gmake install ```

    Don't stop at building only `gprbuild` because you'll probably need the `libgpr` library (if you want to build Ada Web Server(https://github.com/AdaCore/aws) which now depends on GNAT libcoll(https://github.com/AdaCore/gnatcoll-core) which needs libgpr).

    ``` gmake libgpr.build gmake libgpr.install ```

      1. Installation of vscode

    It is possible to use vscode and the Ada plugin for development under FreeBSD. The vscode Ada plugin contains the `ada_language_server` binary which is executed as a language server for vscode. No binary is provided for FreeBSD but since the binary is provided for Linux, we can use it by using the Linux support in FreeBSD. First, install vscode with:

    ``` pkg install vscode ```

    then, launch it and install the `Language Support for Ada` extension. When the extension is activated, you will see a message indicating that the language server cannot be started. There are two things to do:

    • create a `freebsd` directory with the `ada_language_server` binary,
    • activate and setup the Linux support in FreeBSD

    For the first part, copy

    ``` cd ~/.vscode-oss/extensions/adacore.ada-23.0.8 cp -rp linux freebsd ```

    Now we must setup the Linux support in FreeBSD, and follow the FreeBSD Chapter 10. Linux Binary Compatibility(https://docs.freebsd.org/en/books/handbook/linuxemu/).

    Add in `/etc/rc.conf`:

    ``` linux_enable="YES" ```

    and start with:

    ``` sudo service linux start ```

    Install the `debootsrap`:

    ``` sudo pkg install debootstrap sudo debootstrap focal /compat/ubuntu. ```

    Add the line in `/etc/sysctl.conf`:

    ``` compat.linux.emul_path="/compat/ubuntu" ```

    and configure with:

    ``` sudo sysctl -f /etc/sysctl.conf ```

    Mark the binary as Linux:

    ``` brandelf -t Linux ~/.vscode-oss/extensions/adacore.ada-23.0.8/freebsd/ada_language_server ```

    Replace the symbolic link `/compat/ubuntu/lib64/ld-linux-x86-64.so.2` to a real file otherwise the Linux linker is not found by the kernel.

    Now, restart vscode and the Ada language server should be started.

    The description on how to build `ada_language_server` for FreeBSD is left as an exercise to the reader :-)

    IO stream composition and serialization with Ada Utility Library

    5 March 2022 at 22:48

    To be able to provide this IO stream combination, the Ada Utility Library(https://github.com/stcarrez/ada-util) defines two Ada types: the `Input_Stream` and the `Output_Stream` limited interfaces. The `Input_Stream` interface only defines a `Read` procedure and the `Output_Stream` interface defines the `Write`, `Flush` and `Close` procedures. By implementing these interfaces, it is possible to provide stream objects that can be combined together.

    [IO Stream Composition and Serialization](Ada/ada-util-streams.png)

    The Ada Utility Library(https://github.com/stcarrez/ada-util) provides stream types that implement these interfaces so that it is possible to read or write files, sockets and system pipes. In many cases, the concrete type implements both interfaces so that reading or writing is possible. This is the case for `File_Stream` which allows to read or write on a file, the `Socket_Stream` which handles sockets by using GNAT sockets. The `Pipe_Stream` on its side allows to launch an external program and either read its output, available through the `Input_Stream`, or feed the external program with some input by using the `Output_Stream`.

    The Ada Utility Library(https://github.com/stcarrez/ada-util) also provides stream objects that make transformation on the data through various data encoders. The Ada library supports the following encoders:

    • Base 16, Base 64,
    • AES encryption or decryption,
    • LZMA compression or decompression

    Other encoders could be added and it is always possible to provide custom transformations by implementing the `Input_Stream` and `Output_Stream` interfaces.

    The last part that completes the IO stream framework is the serialization framework. That framework defines and provides interface and types to read or write a CSV, XML, JSON or HTTP form stream. The serialization framework uses either the `Input_Stream` or the `Output_Stream` interfaces to either read or write the content. The serialization framework defines operations in a way that allows to read or write these streams independently of their representation format.

      1. LZMA Compression

    Let's have a look at compressing a file by using the `Util.Streams` framework. First we need a `File_Stream` that is configured to read the file to compress and we need another `File_Stream` configured for writing to save in another file. The first file is opened by using the `Open` procedure and the `In_File` mode while the second one is using `Create` and the `Out_File` mode. The `File_Stream` is using the Ada `Stream_IO` standard package to access files.

    ```Ada with Util.Streams.Files;

      In_Stream  : aliased Util.Streams.Files.File_Stream;
      Out_Stream : aliased Util.Streams.Files.File_Stream;
    
      In_Stream.Open (Mode => Ada.Streams.Stream_IO.In_File, Name => Source);
      Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
    

    ```

    In the middle of these two streams, we are going to use a `Compress_Stream` whose job is to compress the data and write the compressed result to a target stream. The compression stream is configured by using the `Initialize` procedure and it is configured to write on the `Out_Stream` file stream. The compression stream needs a buffer and its size is configured with the `Size` parameter.

    ```Ada with Util.Streams.Lzma;

      Compressor : aliased Util.Streams.Lzma.Compress_Stream;
    
      Compressor.Initialize (Output => Out_Stream'Unchecked_Access, Size => 32768);
    

    ```

    To feed the compressor stream with the input file, we are going to use the `Copy` procedure. This procedure reads the content from the `In_Stream` and writes what is read to the `Compressor` stream.

    ```Ada

      Util.Streams.Copy (From => In_Stream, Into => Compressor);
    

    ```

    Flushing and closing the files is automatically handled by a `Finalize` procedure on the `File_Stream` type.

    Complete source example: compress.adb(https://github.com/stcarrez/ada-util/tree/master/samples/compress.adb)

      1. LZMA Decompression

    The LZMA decompression is very close to the LZMA compression but instead it uses the `Decompress_Stream`. The complete decompression method is the following:

    ```Ada procedure Decompress_File (Source : in String;

                              Destination : in String) is
      In_Stream    : aliased Util.Streams.Files.File_Stream;
      Out_Stream   : aliased Util.Streams.Files.File_Stream;
      Decompressor : aliased Util.Streams.Lzma.Decompress_Stream;
    

    begin

      In_Stream.Open (Mode => Ada.Streams.Stream_IO.In_File, Name => Source);
      Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
      Decompressor.Initialize (Input => In_Stream'Unchecked_Access, Size => 32768);
      Util.Streams.Copy (From => Decompressor, Into => Out_Stream);
    

    end Decompress_File; ```

    Complete source example: decompress.adb(https://github.com/stcarrez/ada-util/tree/master/samples/decompress.adb)

      1. AES Encryption

    Encryption is a little bit more complex due to the encryption key that must be configured. The encryption is provided by the `Encoding_Stream` and it uses a `Secret_Key` to configure the encryption key. The `Secret_Key` is a limited type and it cannot be copied. To build the encryption key, one method consists in using the PBKDF2 algorithm described in RFC 8018(https://tools.ietf.org/html/rfc8018). The user password is passed to the PBKDF2 algorithm configured to use the HMAC-256 hashing. The hash method is called on itself 20000 times in this example to produce the final encryption key.

    ```Ada with Util.Streams.AES; with Util.Encoders.AES; with Util.Encoders.KDF.PBKDF2_HMAC_SHA256;

      Cipher       : aliased Util.Streams.AES.Encoding_Stream;
      Password_Key : constant Util.Encoders.Secret_Key := Util.Encoders.Create (Password);
      Salt         : constant Util.Encoders.Secret_Key := Util.Encoders.Create ("fake-salt");
      Key          : Util.Encoders.Secret_Key (Length => Util.Encoders.AES.AES_256_Length);
      ...
         PBKDF2_HMAC_SHA256 (Password => Password_Key,
                             Salt     => Salt,
                             Counter  => 20000,
                             Result   => Key);
    

    ```

    The encoding stream is able to produce or consume another stream. For the encryption, we are going to use the first mode and use the `Produces` procedure to configure the encryption to write on the `Out_Stream` file. Once configured, the `Set_Key` procedure must be called with the encryption key and the encryption method. The initial encryption `IV` vector can be configured by using the `Set_IV` procedure (not used by the example). As soon as the encryption key is configured, the encryption can start and the `Cipher` encoding stream can be used as an `Output_Stream`: we can write on it and it will encrypt the content before passing the result to the next stream. This means that we can use the same `Copy` procedure to read the input file and pass it through the encryption encoder.

    ```Ada

      Cipher.Produces (Output => Out_Stream'Unchecked_Access, Size => 32768);
      Cipher.Set_Key (Secret => Key, Mode => Util.Encoders.AES.ECB);
      Util.Streams.Copy (From => In_Stream, Into => Cipher);
    

    ```

    Complete source example: encrypt.adb(https://github.com/stcarrez/ada-util/tree/master/samples/encrypt.adb)

      1. AES Decryption

    Decryption is similar but it uses the `Decoding_Stream` type. Below is the complete example to decrypt the file:

    ```Ada procedure Decrypt_File (Source : in String;

                           Destination : in String;
                           Password    : in String) is
      In_Stream    : aliased Util.Streams.Files.File_Stream;
      Out_Stream   : aliased Util.Streams.Files.File_Stream;
      Decipher     : aliased Util.Streams.AES.Decoding_Stream;
      Password_Key : constant Util.Encoders.Secret_Key := Util.Encoders.Create (Password);
      Salt         : constant Util.Encoders.Secret_Key := Util.Encoders.Create ("fake-salt");
      Key          : Util.Encoders.Secret_Key (Length => Util.Encoders.AES.AES_256_Length);
    

    begin

      --  Generate a derived key from the password.
      PBKDF2_HMAC_SHA256 (Password => Password_Key,
                          Salt     => Salt,
                          Counter  => 20000,
                          Result   => Key);
    
      --  Setup file -> input and cipher -> output file streams.
      In_Stream.Open (Ada.Streams.Stream_IO.In_File, Source);
      Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
      Decipher.Produces (Output => Out_Stream'Access, Size => 32768);
      Decipher.Set_Key (Secret => Key, Mode => Util.Encoders.AES.ECB);
    
      --  Copy input to output through the cipher.
      Util.Streams.Copy (From => In_Stream, Into => Decipher);
    

    end Decrypt_File; ```

    Complete source example: decrypt.adb(https://github.com/stcarrez/ada-util/tree/master/samples/decrypt.adb)

      1. Stream composition: LZMA > AES

    Now, if we want to compress the stream before encryption, we can do this by connecting the `Compressor` to the `Cipher` stream and we only have to use the `Compressor` instead of the `Cipher` in the call to `Copy`.

    ```Ada

      In_Stream.Open (Ada.Streams.Stream_IO.In_File, Source);
      Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
      Cipher.Produces (Output => Out_Stream'Unchecked_Access, Size => 32768);
      Cipher.Set_Key (Secret => Key, Mode => Util.Encoders.AES.ECB);
      Compressor.Initialize (Output => Cipher'Unchecked_Access, Size => 4096);
    
      Util.Streams.Copy (From => In_Stream, Into => Compressor);
    

    ```

    When `Copy` is called, the following will happen:

    • first, it reads the `In_Stream` source file,
    • the data is written to the `Compress` stream,
    • the `Compressor` stream runs the LZMA compression and writes on the `Cipher` stream,
    • the `Cipher` stream encrypts the data and writes on the `Out_Stream`,
    • the `Out_Stream` writes on the destination file.

    Complete source example: lzma_encrypt.adb(https://github.com/stcarrez/ada-util/tree/master/samples/lzma_encrypt.adb)

      1. More stream composition: LZMA > AES > Base64

    We can easily change the stream composition to encode in Base64 after the encryption. We only have to declare an instance of the Base64 `Encoding_Stream` and configure the encryption stream to write on the Base64 stream instead of the output file. The Base64 stream is configured to write on the output stream.

    ```Ada In_Stream : aliased Util.Streams.Files.File_Stream; Out_Stream : aliased Util.Streams.Files.File_Stream; Base64 : aliased Util.Streams.Base64.Encoding_Stream; Cipher : aliased Util.Streams.AES.Encoding_Stream; Compressor : aliased Util.Streams.Lzma.Compress_Stream;

      In_Stream.Open (Ada.Streams.Stream_IO.In_File, Source);
      Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
      Base64.Produces (Output => Out_Stream'Unchecked_Access, Size => 32768);
      Cipher.Produces (Output => Base64'Unchecked_Access, Size => 32768);
      Cipher.Set_Key (Secret => Key, Mode => Util.Encoders.AES.ECB);
      Compressor.Initialize (Output => Cipher'Unchecked_Access, Size => 4096);
    

    ```

    Complete source example: lzma_encrypt_b64.adb(https://github.com/stcarrez/ada-util/tree/master/samples/lzma_encrypt_b64.adb)

      1. Serialization

    Serialization is achieved by using the `Util.Serialize.IO` packages and child packages and their specific types. The parent package defines the limited `Output_Stream` interface which inherit from the `Util.Streams.Output_Stream` interface. This allows to define specific operations to write various Ada types but also it provides common set of abstractions that allow to write either a JSON, XML, CSV and FORM (`x-www-form-urlencoded`) formats.

    The target format is supported by a child package so that you only have to use the `Output_Stream` type declared in one of the `JSON`, `XML`, `CSV` or `Form` child package and use it transparently. There are some constraint if you want to switch from one output format to another while keeping the same code. These constraints comes from the nature of the different formats: `XML` has a notion of entity and attribute but other formats don't differentiate entities from attributes.

    • A `Start_Document` procedure must be called first. Not all serialization method need it but it is required for JSON to produce a correct output.
    • A `Write_Entity` procedure writes an XML entity of the given name. When used in JSON, it writes a JSON attribute.
    • A `Start_Entity` procedure prepares the start of an XML entity or a JSON structure with a given name.
    • A `Write_Attribute` procedure writes an XML attribute after a `Start_Entity`. When used in JSON, it writes a JSON attribute.
    • A `End_Entity` procedure terminates an XML entity or a JSON structure that was opened by `Start_Entity`.
    • At the end, the `End_Document` procedure must be called to finish correctly the output and terminate the JSON or XML content.

    ```Ada procedure Write (Stream : in out Util.Serialize.IO.Output_Stream'Class) is begin

      Stream.Start_Document;
      Stream.Start_Entity ("person");
      Stream.Write_Entity ("name", "Harry Potter");
      Stream.Write_Entity ("gender", "male");
      Stream.Write_Entity ("age", 17);
      Stream.End_Entity ("person");
      Stream.End_Document;
    

    end Write; ```

        1. JSON Serialization

    With the above `Write` procedure, if we want to produce a JSON stream, we only have to setup a JSON serializer. The JSON serializer is connected to a `Print_Stream` which provides a buffer and helper operations to write some text content. An instance of the `Print_Stream` is declared in `Output` and configured with a buffer size. The JSON serializer is then connected to it by calling the `Initialize` procedure and giving the `Output` parameter.

    After writing the content, the JSON is stored in the `Output` print stream and it can be retrieved by using the `To_String` function.

    
    

    ```Ada with Ada.Text_IO; with Util.Serialize.IO.JSON; with Util.Streams.Texts; procedure Serialize is

      Output : aliased Util.Streams.Texts.Print_Stream;
      Stream : Util.Serialize.IO.JSON.Output_Stream;
    

    begin

      Output.Initialize (Size => 10000);
      Stream.Initialize (Output => Output'Unchecked_Access);
      Write (Stream);
      Ada.Text_IO.Put_Line (Util.Streams.Texts.To_String (Output));
    

    end Serialize; ```

    The `Write` procedure described above produces the following JSON content:

    ```C {"person":{"name":"Harry Potter","gender":"male","age": 17}} ```

    Complete source example: serialize.adb(https://github.com/stcarrez/ada-util/tree/master/samples/serialize.adb)

        1. XML Serialization

    Switching to an XML serialization is easy: replace `JSON` by `XML` in the package to use the XML serializer instead.

    ```Ada with Ada.Text_IO; with Util.Serialize.IO.XML; with Util.Streams.Texts; procedure Serialize is

      Output : aliased Util.Streams.Texts.Print_Stream;
      Stream : Util.Serialize.IO.XML.Output_Stream;
    

    begin

      Output.Initialize (Size => 10000);
      Stream.Initialize (Output => Output'Unchecked_Access);
      Write (Stream);
      Ada.Text_IO.Put_Line (Util.Streams.Texts.To_String (Output));
    

    end Serialize; ```

    This time, the same `Write` procedure produces the following XML content:

    ```C <person><name>Harry Potter</name><gender>male</gender><age>17</age></person> ```

    Complete source example: serialize_xml.adb(https://github.com/stcarrez/ada-util/tree/master/samples/serialize_xml.adb)

    Advanced Resource Embedder 1.2.0

    23 January 2022 at 09:04
    [Embedding SQL in binary](Ada/resource-embedder-bin.png)

    The new release contains the following fixes:

    - Fix Ada generator to be able to use other binary content types

     such as `System.Storage_Elements.Storage_Array`
    
    - Fix Ada generator to escape special characters in strings

    Since the previous version, Fabien Chouteau asked to be able to use an Ada system type to represent a binary content. This is now possible by using the following XML extract for the description to tell the resource embedder how to integrate the file and generate the Ada source code:

    ```XML <package>

     <resource name='Resources.Help'
               format='binary'
               type='access constant System.Storage_Elements.Storage_Array'>
       <install mode='copy'>
         <fileset dir="help">
           <include name="**/*.txt"/>
         </fileset>
       </install>
     </resource>
     ...
    

    </package> ```

    With the above description, the Ada code generator produces the following package specification:

    ```Ada with System.Storage_Elements; package Resources.Help is

      type Content_Access is access constant System.Storage_Elements.Storage_Array;
      type Name_Access is access constant String;
      type Name_Array is array (Natural range <>) of Name_Access;
      Names : constant Name_Array;
      function Get_Content (Name : String)
         access constant System.Storage_Elements.Storage_Array;
    

    private

      ...
    

    end Resources.Help; ```

    This example is available in Embedding help and documentation in Ada (binary)(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-help-binary). If you prefer to use `String` instead of a `Storage_Array`, have a look at the Embedding help and documentation in Ada(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-help). Both examples are similar but they are exposing the file using different Ada types.

    To install the tool, follow the instructions given in the initial announcement: Advanced Resource Embedder for Ada, C and Go(https://blog.vacs.fr/vacs/blogs/post.html?post=2021/06/11/Advanced-Resource-Embedder).

    If you want to know more about the tool, have a look at its documentation:

    • Resource Embedder Guide(https://resource-embedder.readthedocs.io/en/latest/) PDF(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are-book.pdf)
    • Man page: are (1)(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are.md)

    and if you have ideas for improvements, fork the project and submit a pull request!

    AWA 2.3.0

    31 July 2021 at 08:53

    The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.

    AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.

    A typical architecture of an AWA application is represented by the picture below:

    ((Ada/awa_architecture_overview.png|awa_architecture_overview.png|C|Ada Web Application architecture)

    This version of AWA integrates smoothly with Ada Keystore in order to protect the server sensitive configuration.

    AWA, Version 2.3.0

    • Update Trumbowyg editor to version 2.23.0
    • Fix generation of og:image meta for blog articles written in Markdown
    • Fix wiki preview with latest xkhtmltoimage 0.12.6
    • Use Dynamo 1.2.2, Ada Server Faces 1.4.3, Ada Servlet 1.5.2, OpenAPI Ada 0.5.0
    • Use Ada Wiki 1.3.2, Ada Database Objects 2.2.0
    • Use Ada Keystore 1.3.2, Ada EL 1.8.3, Ada Utility Library 2.4.1

    Links: Download GitHub GitLab

    Dynamo, Version 1.2.2

    • Fix the SQL type definition for double on PostgreSQL
    • Fix double support and nullable entity_type
    • Fix SQL generation for a foreign key with variable length

    Links: Download GitHub GitLab

    Swagger Ada, Version 0.5.0

    • Fix for GNAT 2021
    • Update the openapi generator to version 5.2.0

    Links: Download GitHub GitLab

    Ada Servlet, Version 1.5.2

    • Fix for GNAT 2021

    Links: Download GitHub GitLab

    Ada Wiki Library, Version 1.3.2

    • Fix <hr> and <br> generation to follow HTML5 convention.
    • Add option -H to the render example
    • Fix for GNAT 2021

    Links: Download GitHub GitLab

    Ada Server Faces, Version 1.4.3

    • Add jQuery 3.6.0
    • Add a programmer's guide
    • Remove very old jQuery 1.11.3, jQuery UI 1.11.4, jQuery Chosen 1.4.2

    Links: Download GitHub GitLab

    Ada Database Objects, Version 2.2.0

    • Improvement of query loaders
    • Fix reading database schema with float/double values
    • Rename Get_Double into Get_Long_Float

    Links: Download GitHub GitLab

    Ada EL Library, Version 1.8.3

    • Fix compilation warning with GNAT 2021

    Links: Download GitHub GitLab

    Ada Utility Library, Version 2.4.1

    • Fix compilation issues with GNAT 2021
    • Fix serialization of Util.Beans.Objects holding arrays or maps

    Links: Download GitHub GitLab

    Ada Keystore, Version 1.3.2

    • Minor compilation warning fixes

    Links: Download GitHub GitLab

    All these Ada projects can be downloaded individually but they are also packaged together to help in their download and build process. You can also download everything at http://download.vacs.fr/ada-awa/awa-all-2.3.0.tar.gz

    After downloading the awa-all-2.3.0.tar.gz package, have a look at the Ada Web Application Programmer's Guide to learn how to build, install and start using all this.

    If you don't have time to build all this, a docker container is available: https://hub.docker.com/r/ciceron/ada-awa/

    Debian packages

    I've created and setup a Debian repository to give access to several Debian packages for several Ada projects that I manage. The goal is to provide some easy and ready to use packages to simplify and help in the installation of various Ada libraries.

    Access to the repository

    The repository packages are signed with GPG. To get the verification key and setup the apt-get tool, you should run the following command:

    wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
    

    Ubuntu 18.04 Bionic Beaver

    A first repository provides Debian packages targeted at Ubuntu 18.04 bionic. They are built with the gnat-7 package and depend on libgpr1-dev, libaws3.3.2.2-dev and libxmlada-sax7-dev. Add the following line to your /etc/apt/sources.list configuration:

    deb https://apt.vacs.fr/ubuntu-bionic bionic main
    

    Ubuntu 20.04 Focal Fossa

    A second repository provides Debian packages targeted at Ubuntu 20.04 focal. They are built with the gnat-9 package and depend on libgnatprj7-dev, libaws19-dev and libxmlada-sax9-dev. Add the following line to your /etc/apt/sources.list configuration:

    deb https://apt.vacs.fr/ubuntu-focal focal main
    

    Debian 10 Buster

    A third repository provides Debian packages targeted at Debian 10 buster. They are built with the gnat-8 package and depend on libgpr2-dev, libaws18-dev and libxmlada-sax8-dev. Add the following line to your /etc/apt/sources.list configuration:

    deb https://apt.vacs.fr/debian-buster buster main
    

    Installation

    Once you've added the configuration line, you can install the packages:

    sudo apt-get update
    sudo apt-get install -y dynamo akt libawa2-dev libawa-unit2-dev
    

    The Ada Web Application framework contains several plugins which are provided by separate Debian packages.

    sudo apt-get install -y libawa-blogs2-dev libawa-questions2-dev libawa-wikis2-dev libawa-setup2-dev
    

    The documentation is packaged in separate Debian packages, you can install them with:

    sudo apt-get install -y libawa-doc libado-doc libawa-doc libkeystoreada-doc libsecurity-doc libutilada-doc libwikiada-doc
    

    Advanced Resource Embedder 1.1.0

    4 July 2021 at 16:52

    The resource embedder allows to embed files in binaries by producing C, Ada or Go source files that contain the original files. The first version of the tool was representing the file content as an array of bytes. In Ada, they are accessed through an `Ada.Streams.Stream_Element_Array` which is not easy to use when you need the content as a string.

    The new release introduces the customization of the format for each resource. The format is controlled through the XML description by the `format` attribute. The following data formats are supported:

    • `binary` format provides the file content as a binary data,
    • `string` format provides the file content as string,
    • `lines` format splits the content in several lines and according to a set of customizable rules.

    With the `string` format, the Ada code generator can generate the following function:

    ```

     function Get_Content (Name : in String)
       return access constant String;
    

    ```

    The `lines` format tells the code generator to represent the content as an array of separate lines. For this integration, some control is available to indicate how the content must be split and optionally apply some filter on the input content. These controls are made within the XML description by using the `line-separator` and `line-filter` description: The `line-separator` indicates the characters that represent a line separation. There can be several `line-separator` definition. The `line-filter` defines a regular expression that when matched must be replaced by an empty string or a specified content. The `line-filter` are applied in the order of the XML definition.

    The example below is intended to integrate an SQL scripts with:

    • a separate line for each SQL statement,
    • remove spurious empty lines and SQL comments.

    The SQL statements are separated by `;` (semi-colon) and the `line-separator` indicates to split lines on that character. By splitting on the `;` we allow to have an SQL statement on multiple lines in the original SQL source file.

    ```XML <package>

     <resource name='Scripts'
               format='lines'
               type='access constant String'>
       <line-separator>;</line-separator>
    
       <!-- Remove new lines -->
       <line-filter>[\r\n]</line-filter>
    
       <!-- Remove C comments -->
       <line-filter>/\*[^/]*\*/</line-filter>
    
       <!-- Remove contiguous spaces after C comments removal -->
       <line-filter replace=' '>[ \t][ \t]+</line-filter>
    
       <install mode='copy' strip-extension='yes'>
         <fileset dir="sql">
           <include name="**/*.sql"/>
         </fileset>
       </install>
     </resource>
    

    </package> ```

    Then the first `line-filter` will remove the `r` and `n` characters.

    The regular expression `/\*[./]*\*/` matches a C style comment and remove it.

    The last `line-filter` replaces multiple tabs and spaces by a single occurrence.

    Below is an example of generated code with the above resource description. Each file is accessed through a separate variable whose name is built from the base name of the original file.

    [Resource Embedder Overview](Ada/are-scripts.png)

    The command used to generate this code was:

    ``` are lang=Ada -o src var-access content-only rule=package.xml . ```

    and the `sql` directory contains only two files: `create-database.sql` and `drop-database.sql`.

    The complete example is available for two languages:

    To install the tool, follow the instructions given in the initial announcement: Advanced Resource Embedder for Ada, C and Go(https://blog.vacs.fr/vacs/blogs/post.html?post=2021/06/11/Advanced-Resource-Embedder).

    If you want to know more about the tool, have a look at its documentation:

    • Resource Embedder Guide(https://resource-embedder.readthedocs.io/en/latest/) PDF(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are-book.pdf)
    • Man page: are (1)(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are.md)

    and if you have ideas for improvements, fork the project and submit a pull request!

    Advanced Resource Embedder for Ada, C and Go

    11 June 2021 at 12:43

    The tool is able to apply some transformations on the collected files:

    • it can run a Javascript minifier such as `closure`,
    • it can compress CSS files by running `yui-compressor`,
    • it can compress files by running `gzip` or another compression tool.

    Once these transformations are executed, it invokes a target generator to produce a source file either in C, Ada or Go language. The generated source file can then be used in the final program and taken into account during the compilation process of that program. At the end, the binary will contain the embedded files with their optional transformations.

    [Resource Embedder Overview](Ada/resource-embedder.png)

    The process to use ARE is simple:

    • You describe the resources that you want to embed.
     The description is either made on command line arguments or by writing an XML file.
     The XML description gives more flexibility as it allows to define a transformation rule that
     must be executed on the original file before being embedded.  This allows to minify a Javascript
     or CSS file, compress some files and even encrypt a file before its integration.
    
  • You run the ARE command with your target language and rule description and you give the tool
  •  a list of directories that must be scanned to identify the files that must be collected.
     The ARE tool scan the directories according to the patterns that you have given either on
     the command line or in the XML rule description.  After identifying the files, the tool applies
     the rules and execute the transformations.
     The ARE tool then invokes the target language generator that writes one or several files depending
     on the list of resources.
    
  • Once the files are generated, you use them in your program and add them in your build process
  •  as they are now part of your sources.  After building your program, it now embeds the
     resource files that were collected and optionally transformed.
    
      1. Defining resources to embed

    The first step is to write some `package.xml` file which describes a list of resources with their content. The root XML element is *`package`* and each resource is described by a `resource` XML element. The resource is assigned a name that will be used by the code generator. The C generator will use the name as a prefix for the C function and C type declaration. The Ada and Go generator will use that name for the Ada or Go package.

    The following resource definition declares the `Help` resource. It contains an installation rule that will copy the files under the `help` directory in the resource set. Only files matching the `.txt` pattern will be taken into account.

    ```XML <package>

     <resource name='Help'>
       <install mode='copy'>
         <fileset dir="help">
           <include name="**/*.txt"/>
         </fileset>
       </install>
     </resource>
     ...
    

    </package> ```

    The next resource definition will run an external program to get the content that must be embedded. The `man` directory is scanned and it will execute the command `man #{name}` on each filename found. That directory contains the empty files `ls`, `pwd` and `sh` and this will run and embed the man page for these commands.

    ```XML <package>

      ...
     <resource name='Man'>
       <install mode='exec'>
         <command output='#{dst}'>man #{name}</command>
         <fileset dir="man">
           <include name="*"/>
         </fileset>
       </install>
     </resource>
    

    </package> ```

    You may run other commands such as:

    ``` <command>closure charset UTF-8 #{src} js_output_file #{dst}</command> <command>yui-compressor type css charset UTF-8 -o #{dst} #{src}</command> <command output="#{dst}">gzip --no-name -c #{src}</command> ```

      1. Running the Advanced Resource Embedder

    The tool has several options that allows you to control the target code generator and tune the generation according to your needs. Assuming that the files to embed are located in the current directory, you would use the following command to generate C source files in the `src` directory:

    ``` are lang=c -o src rule=package.xml list-access name-access . ```

    For C, this would generate a `src/man.h`, `src/man.c`, `src/help.h` and `src/help.c`. You now have to include these files in the compilation of your program.

    You would use the following command for Go:

    ``` are lang=go rule=package.xml list-access name-access . ```

    and it would generate in `man/man.go` and `help/help.go` the Go source files.

    You would use the following command for Ada:

    ``` are lang=Ada -o src rule=package.xml list-access name-access . ```

    and it would generate `src/man.ads`, `src/man.adb`, `src/help.ads` and `src/help.adb`.

      1. Using the resource

    The code generator emits by default a set of type and function declaration that give access to the resource. For C, the following structure and declaration are generated in the header for the `man` resource:

    ```C struct man_content {

     const unsigned char *content;
     size_t size;
     time_t modtime;
     int format;
    

    };

    extern const struct man_content* man_get_content(const char* name); ```

    But for the Go language, the `man` generated package declares:

    ```Go type Content struct {

       Content  []byte
       Size  int64
       Modtime  int64
       Format   int
    

    } ... func Get_content(name string) (*Content) {...} ```

    And for Ada, it will generate:

    ```Ada package Resources.Man is

      type Content_Access is access constant Ada.Streams.Stream_Element_Array;
      type Name_Access is access constant String;
      type Format_Type is (FILE_RAW, FILE_GZIP);
      type Content_Type is record
         Name    : Name_Access;
         Content : Content_Access;
         Modtime : Interfaces.C.long := 0;
         Format  : Format_Type := FILE_RAW;
      end record;
      Null_Content : constant Content_Type;
    
      type Name_Array is array (Natural range <>) of Name_Access;
      Names : constant Name_Array;
    
      function Get_Content (Name : String) return Content_Type;
    

    private

      ...
    

    end Resources.Man; ```

    You can avoid the type declaration by using the `--no-type-declaration` option and in that case you have to make available these types somehow. In Ada, this can easily be made by providing these types in a parent Ada package (see the Embedding help and documentation in Ada(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-help) for the example).

    Now with the generated code, you only need to call the generated get content method to obtain the embedded content. Simple!

      1. Examples

    The project proposes several detailed examples to illustrate various integrations.

    This first set of example shows how to you can embed configuration files in a C, Ada or Go program. The Advance Resource Embedder simply puts the configuration files in an array of bytes that can easily be retrieved by a generated function.

    A second set of example is more advanced by the use of an XML file that describes what must be embedded with the transformations that must be made. It creates two distinct resource sets `help` and `man`. The `help` resource set is composed of a set of fixed documentation files provided in the example. The `man` resource set is created by running the `man` Unix command on various names to embed the man page of `ls`, `pwd` and `sh`.

    More specific examples show how to make specific transformations on the files before integrating them:

      1. License considerations

    The generated code produced by Advance Resource Embedder(https://gitlab.com/stcarrez/resource-embedder) is not covered by any license. However, when you integrate some resource files with the tool, the generated code will contain the original file in some binary form and therefore it may be covered by the license associated with these resource files.

    For example, if you include a file covered by the GNU General Public license, then the generated file is covered by the GPL.

      1. How can you get Advanced Resource Embedder?
        1. Use the source Luke!

    Quick instructions to build the tool:

    ``` git clone --recursive https://gitlab.com/stcarrez/resource-embedder.git cd resource-embedder make build test install ```

        1. Debian Packages for x86_64

    You can install ARE by using the Debian 10 and Ubuntu 20.04 or 18.04 packages. First, setup to accept the signed packages:

    ``` wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add - ```

    and choose one of the `echo` command according to your Linux distribution:

    Ubuntu 20.04 ``` echo "deb https://apt.vacs.fr/ubuntu-focal focal main" | sudo tee -a /etc/apt/sources.list.d/vacs.list ```

    Ubuntu 18.04 ``` echo "deb https://apt.vacs.fr/ubuntu-bionic bionic main" | sudo tee -a /etc/apt/sources.list.d/vacs.list ```

    Debian 10 ``` echo "deb https://apt.vacs.fr/debian-buster buster main" | sudo tee -a /etc/apt/sources.list.d/vacs.list ```

    Then, launch the apt update command:

    ``` sudo apt-get update ```

    and install the tool using:

    ``` sudo apt-get install -y are ```

      1. Conclusion

    Embedding files, scripts, documentation and other contents in C and Ada is easily done by using the Advance Resource Embedder(https://gitlab.com/stcarrez/resource-embedder). Go developers are already familiar with these mechanisms thanks to the `go:embed` directive. But the tool provides custom transformations that gives more flexibility for the integration.

    Some benefits in embedding contents in final binaries:

    • the content is embedded within the readonly `.rodata` section,
    • you avoid at least 3 system calls to access the content: an `open`, a `read` and a `close`,
    • you reduce the size of your application on the disk: contents are contiguous within the `.rodata` section
     whereas written on a file system they each require full dedicated data blocks (4K in most cases),
    
  • you get a portable `mmap` of your files for free and without pain.
  • If you want to know more about the tool, have a look at its documentation:

    • Resource Embedder Guide(https://resource-embedder.readthedocs.io/en/latest/) PDF(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are-book.pdf)
    • Man page: are (1)(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are.md)

    and if you have ideas for improvements, fork the project and submit a pull request!

    New release Ada BFD 1.2.0

    11 April 2021 at 18:37

    The new release is a cleanup and update of the library to support newer version of GNU binutils. The main changes are below:

    • Cleanup build process and use gprinstall for installation
    • Fix build with binutils > 2.34
    • Remove invalid API: `Bfd.Set_Error_Handler`
    • Remove fake API: `Bfd.Symbols.Is_Local_Label_Name`
      1. Installation

    To use the Ada BFD library, you may either build it from the sources ada-bfd-1.2.0.tar.gz(https://download.vacs.fr/ada-bfd/ada-bfd-1.2.0.tar.gz) or install the Debian packages.

    To build from the sources, you need to have a working GNAT Ada compiler as well as the `binutils-dev` Debian package installed. Then, run the following commands:

    ```sh git clone https://github.com/stcarrez/ada-bfd.git cd ada-bfd ./configure make build check install ```

    For the Debian package installation, use the configuration that corresponds to your setup:

    ``` deb https://apt.vacs.fr/ubuntu-bionic bionic main deb https://apt.vacs.fr/ubuntu-focal focal main deb https://apt.vacs.fr/debian-buster buster main ```

    and then run:

    ``` sudo apt-get update

    1. Bionic: sudo apt-get install libbfdada1-dev
    2. Focal: sudo apt-get install libbfdada2-dev
    3. Buster: sudo apt-get install libbfdada3-dev

    ```

      1. Reading the ELF sections

    Using the Ada BFD library in a projet is quite easy, the first step is to add the following line in your GNAT project file:

    ``` Ada with "bfdada"; ```

    To access the information of a binary, you must first define an instance of the `File_Type` and open the file. You will do this as follows:

    ``` Ada with Bfd.Files;

     ...
     Path : constant String := "..."; --  the binary to load
     File : Bfd.Files.File_Type;
     ...
     Bfd.Files.Open (File, Path, "");
    

    ```

    Looking at the ELF section is easily made by using the `Section_Iterator` provided by the `Bfd.Sections` package.

    ```ada with Bfd.Sections;

     ...
     Iter : Bfd.Sections.Section_Iterator := Bfd.Sections.Get_Sections (File);
     ...
     while Bfd.Sections.Has_Element (Iter) loop
       declare
          S   : constant Bfd.Sections.Section := Bfd.Sections.Element (Iter);
       begin
          Ada.Text_IO.Put_Line (Bfd.Sections.Get_Name (S));
       end;
       Bfd.Sections.Next (Iter);
     end loop;
    

    ```

    The library contains several examples that show different features of the Ada BFD library:

    | bfdinfo.adb(https://github.com/stcarrez/ada-bfd/blob/master/samples/bfdinfo.adb) | ./bin/bfdinfo ./bin/bfdgen | Open BFD file, get flags, list sections and symbols | | sections.adb(https://github.com/stcarrez/ada-bfd/blob/master/samples/sections.adb) | ./bin/sections ./bin/bfdgen | Display the ELF sections with the `Bfd.Sections` | | symbol.adb(https://github.com/stcarrez/ada-bfd/blob/master/samples/symbol.adb) | ./bin/symbol ./bin/bfdgen main | Read the symbol table with `Bfd.Symbols` | | disassemble.adb(https://github.com/stcarrez/ada-bfd/blob/master/samples/disassemble.adb) | ./bin/disassemble ./bin/bfdgen | Disassemble the text section with `Bfd.Disassemble` |

    AWA 2.2.0

    20 March 2021 at 16:33

    The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.

    AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.

    A typical architecture of an AWA application is represented by the picture below:

    ((Ada/awa_architecture_overview.png|awa_architecture_overview.png|C|Ada Web Application architecture)

    This version of AWA integrates smoothly with Ada Keystore in order to protect the server sensitive configuration.

    AWA, Version 2.2.0

    • Fixed the Markdown js editor configuration
    • Send an event when a blog post is published (allows customisation such as sending e-mails)
    • Use Dynamo 1.2.1, Ada Server Faces 1.4.2, Ada Servlet 1.5.1, OpenAPI Ada 0.4.0
    • Use Ada Security 1.4.0, Ada Wiki 1.3.1, Ada Database Objects 2.1.2
    • Use Ada Keystore 1.3.1, Ada EL 1.8.2, Ada Utility Library 2.4.0

    Download: http://download.vacs.fr/ada-awa/ada-awa-2.2.0.tar.gz

    GitHub: https://github.com/stcarrez/ada-awa

    GitLab: https://gitlab.com/stcarrez/ada-awa

    Dynamo, Version 1.2.1

    • Fix uninitialized float values in generated code

    Download: http://download.vacs.fr/dynamo/dynamo-1.2.1.tar.gz

    GitHub: https://github.com/stcarrez/dynamo

    GitLab: https://gitlab.com/stcarrez/dynamo

    Swagger Ada, Version 0.4.0

    • Update the openapi generator to version 5.1.0

    Download: http://download.vacs.fr/swagger-ada/swagger-ada-0.4.0.tar.gz

    GitHub: https://github.com/stcarrez/swagger-ada

    GitLab: https://gitlab.com/stcarrez/swagger-ada

    Ada Servlet, Version 1.5.1

    • Cleanup the examples
    • Fix registration and support of application with an empty registration URI

    Download: http://download.vacs.fr/ada-servlet/ada-servlet-1.5.1.tar.gz

    GitHub: https://github.com/stcarrez/ada-servlet

    GitLab: https://github.com/stcarrez/ada-servlet

    Ada Wiki Library, Version 1.3.1

    • Minor cleanup for the build

    Download: http://download.vacs.fr/ada-wiki/ada-wiki-1.3.1.tar.gz

    GitHub: https://github.com/stcarrez/ada-wiki

    GitLab: https://gitlab.com/stcarrez/ada-wiki

    Ada Server Faces, Version 1.4.2

    • Fix compilation warnings
    • Cleanup build and examples

    Download: http://download.vacs.fr/ada-asf/ada-asf-1.4.2.tar.gz

    GitHub: https://github.com/stcarrez/ada-asf

    GitLab: https://gitlab.com/stcarrez/ada-asf

    Ada Database Objects, Version 2.1.2

    • Fix uninitialized float values in generated code used by unit tests

    Download: http://download.vacs.fr/ada-ado/ada-ado-2.1.2.tar.gz

    GitHub: https://github.com/stcarrez/ada-ado

    GitLab: https://gitlab.com/stcarrez/ada-ado

    Ada EL Library, Version 1.8.2

    • Minor build cleanup

    Download: http://download.vacs.fr/ada-el/ada-el-1.8.2.tar.gz

    GitHub: https://github.com/stcarrez/ada-el

    GitLab: https://gitlab.com/stcarrez/ada-el

    Ada Utility Library, Version 2.4.0

    • Add support to customize and provide application specific log appenders (example in ada-keystore)
    • Improvement of read/write streams to chain LZMA, AES, Base64
    • Add examples to show LZMA compress+AES encryption, AES decryption+LZMA decompress
    • Fix compilation with GNAT 10
    • New package Util.Properties.Form to help in parsing application/x-www-form-urlencoded
    • Fixed the Util.Tests.Get_Test_Path semantic and use the results directory
    • Drop detection of buggy gcc 4.7.2

    Download: http://download.vacs.fr/ada-util/ada-util-2.4.0.tar.gz

    GitHub: https://github.com/stcarrez/ada-util

    GitLab: https://gitlab.com/stcarrez/ada-util

    Ada Security Library, Version 1.4.0

    • Add support to authenticate with Gitlab and GitHub
    • Update Yahoo! authenticate to use OpenId connect
    • Update the AWS demo to add Gitlab and Github

    Download: http://download.vacs.fr/ada-security/ada-security-1.4.0.tar.gz

    GitHub: https://github.com/stcarrez/ada-security

    GitLab: https://github.com/stcarrez/ada-security

    All these Ada projects can be downloaded individually but they are also packaged together to help in their download and build process. You can also download everything at http://download.vacs.fr/ada-awa/awa-all-2.2.0.tar.gz

    After downloading the awa-all-2.2.0.tar.gz package, have a look at the Ada Web Application Programmer's Guide to learn how to build, install and start using all this.

    If you don't have time to build all this, a docker container is available: https://hub.docker.com/r/ciceron/ada-awa/

    Debian packages

    I've created and setup a Debian repository to give access to several Debian packages for several Ada projects that I manage. The goal is to provide some easy and ready to use packages to simplify and help in the installation of various Ada libraries.

    Access to the repository

    The repository packages are signed with GPG. To get the verification key and setup the apt-get tool, you should run the following command:

    wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
    

    Ubuntu 18.04 Bionic Beaver

    A first repository provides Debian packages targeted at Ubuntu 18.04 bionic. They are built with the gnat-7 package and depend on libgpr1-dev, libaws3.3.2.2-dev and libxmlada-sax7-dev. Add the following line to your /etc/apt/sources.list configuration:

    deb https://apt.vacs.fr/ubuntu-bionic bionic main
    

    Ubuntu 20.04 Focal Fossa

    A second repository provides Debian packages targeted at Ubuntu 20.04 focal. They are built with the gnat-9 package and depend on libgnatprj7-dev, libaws19-dev and libxmlada-sax9-dev. Add the following line to your /etc/apt/sources.list configuration:

    deb https://apt.vacs.fr/ubuntu-focal focal main
    

    Debian 10 Buster

    A third repository provides Debian packages targeted at Debian 10 buster. They are built with the gnat-8 package and depend on libgpr2-dev, libaws18-dev and libxmlada-sax8-dev. Add the following line to your /etc/apt/sources.list configuration:

    deb https://apt.vacs.fr/debian-buster buster main
    

    Installation

    Once you've added the configuration line, you can install the packages:

    sudo apt-get update
    sudo apt-get install -y dynamo akt libawa2-dev libawa-unit2-dev
    

    The Ada Web Application framework contains several plugins which are provided by separate Debian packages.

    sudo apt-get install -y libawa-blogs2-dev libawa-questions2-dev libawa-wikis2-dev libawa-setup2-dev
    

    The documentation is packaged in separate Debian packages, you can install them with:

    sudo apt-get install -y libawa-doc libado-doc libawa-doc libkeystoreada-doc libsecurity-doc libutilada-doc libwikiada-doc
    

    For the curious, you may browse the repository here.

    New release of AKT with Fuse support

    7 March 2021 at 19:08

    Data are stored in secure wallets protected and signed by their own AES-256 key.

    Wallets are protected by a master key using AES-256-CBC and the wallet master key is protected by a user password. The wallet defines up to 7 slots that identify a password key that is able to unlock the master key. To open a wallet, it is necessary to unlock one of these 7 slots by providing the correct password. Wallet key slots are protected by the user's password and the PBKDF2-HMAC-256 algorithm, a random salt, a random counter and they are encrypted using AES-256-CBC. A wallet key slot can also be protected by a GPG private key.

    Data stored in the wallet are protected by their own encryption keys using AES-256-CBC. Big contents are stored in several data blocks and each data block is encrypted by using its own AES-256-CBC encryption key. Encryption keys are protected by the master key using AES-256-CBC. Data blocks are then signed using HMAC-256.

    A wallet can contain another wallet which is then protected by its own encryption keys and passwords (with 7 independent slots). Because the child wallet has its own master key, it is necessary to known the primary password and the child password to unlock the parent wallet first and then the child wallet.

    Installation

    You can install AKT by using the Debian 10 and Ubuntu 20.04 or 18.04 packages. First, setup to accept the signed packages:

    wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
    

    and choose one of the echo command according to your Linux distribution:

    # Ubuntu 20.04
    # echo "deb https://apt.vacs.fr/ubuntu-focal focal main" | sudo tee -a /etc/apt/sources.list.d/vacs.list
    # Ubuntu 18.04
    # echo "deb https://apt.vacs.fr/ubuntu-bionic bionic main" | sudo tee -a /etc/apt/sources.list.d/vacs.list
    # Debian 10
    # echo "deb https://apt.vacs.fr/debian-buster buster main" | sudo tee -a /etc/apt/sources.list.d/vacs.list
    

    Then, launch the apt update command:

    sudo apt-get update
    

    and install the tool using:

    sudo apt-get install -y akt
    

    You can also build from the source by using the following commands:

    sudo apt-get install -y make gnat-7 gprbuild git gnupg2 libfuse-dev
    git clone --recursive https://github.com/stcarrez/ada-keystore.git
    cd ada-keystore
    ./configure --enable-fuse
    

    Using AKT

    To setup a keystore file and protect it with your GPG key, use the following command:

    akt create secure.akt --gpg your-gpg-key-id
    

    The keystore content can be mounted as a filesystem so that the encrypted content are directly available to existing applications as regular files. The mount command starts the fuse daemon and puts the command as a background fuse daemon to serve as a fuse filesystem and make the content visible through the mount point.

    akt mount secure.akt /mnt
    

    After the mount is successful, you can copy or access any file stored in the secure keystore. By default the Fuse filesystem is configured to ask the kernel to avoid caching the decrypted content. This has an impact on the performance since every access requires to decrypt the content.

    To unmount the filesystem, you must use the umount command.

    umount /mnt
    

    Sometimes, you may have to force the umount by running:

    sudo umount /mnt
    

    Ada Web Application release 2.1.0

    15 November 2020 at 21:12

    The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.

    AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.

    A typical architecture of an AWA application is represented by the picture below:

    ((Ada/awa_architecture_overview.png|awa_architecture_overview.png|C|Ada Web Application architecture)

    This version of AWA integrates smoothly with Ada Keystore in order to protect the server sensitive configuration.

    AWA, Version 2.1.0

    • Update Trumbowyg editor to version 2.21.0
    • Fix compilation issues with GNAT 2020
    • Update mail UI component to attach external files
    • Improved setup for secure configuration with Ada Keystore
    • Use Dynamo 1.2.0, Ada Server Faces 1.4.1, Ada Servlet 1.5.0, OpenAPI Ada 0.3.0
    • Use Ada Security 1.3.1, Ada Wiki 1.3.0, Ada Database Objects 2.1.1
    • Use Ada Keystore 1.2.1, Ada EL 1.8.1, Ada Utility Library 2.3.0

    Download: http://download.vacs.fr/ada-awa/ada-awa-2.1.0.tar.gz

    GitHub: https://github.com/stcarrez/ada-awa

    GitLab: https://gitlab.com/stcarrez/ada-awa

    Dynamo, Version 1.2.0

    • Integrate ArgoUML 0.35.2-2020-07-05
    • Fix SQL generation with 'auto' generator
    • Fix XML Hibernate mapping support
    • Improvement in SQL schema generation

    Download: http://download.vacs.fr/dynamo/dynamo-1.2.0.tar.gz

    GitHub: https://github.com/stcarrez/dynamo

    GitLab: https://gitlab.com/stcarrez/dynamo

    Swagger Ada, Version 0.3.0

    • Install the openapi generator
    • Update the openapi generator to version 5.0.0
    • Update to use Swagger UI 3.36.0
    • Fixed Ada client code generator to support `FreeFormObject` and multi-path parameters
    • Add support for HTTP HEAD, OPTIONS and PATCH requests
    • Initial implementation of Swagger OpenAPI to easily implement REST clients and servers

    Download: http://download.vacs.fr/swagger-ada/swagger-ada-0.3.0.tar.gz

    GitHub: https://github.com/stcarrez/swagger-ada

    GitLab: https://gitlab.com/stcarrez/swagger-ada

    Ada Servlet, Version 1.5.0

    • Added support for PATCH and fix OPTIONS and HEAD requests
    • Added a Docker image to play with Ada Servlet

    Download: http://download.vacs.fr/ada-servlet/ada-servlet-1.5.0.tar.gz

    GitHub: https://github.com/stcarrez/ada-servlet

    GitLab: https://github.com/stcarrez/ada-servlet

    Ada Wiki Library, Version 1.3.0

    • New plugin and filter to setup and expand variables in Wiki texts
    • Add support for Markdown tables
    • Fixes in the Markdown and Creole syntax parsers
    • New Ada Wiki Engine Programmer's Guide

    Download: http://download.vacs.fr/ada-wiki/ada-wiki-1.3.0.tar.gz

    GitHub: https://github.com/stcarrez/ada-wiki

    GitLab: https://gitlab.com/stcarrez/ada-wiki

    Ada Server Faces, Version 1.4.1

    • Fix translations, compilation warnings

    Download: http://download.vacs.fr/ada-asf/ada-asf-1.4.1.tar.gz

    GitHub: https://github.com/stcarrez/ada-asf

    GitLab: https://gitlab.com/stcarrez/ada-asf

    Ada Database Objects, Version 2.1.1

    • Fix using the configuration with multiple config environments

    Download: http://download.vacs.fr/ada-ado/ada-ado-2.1.1.tar.gz

    GitHub: https://github.com/stcarrez/ada-ado

    GitLab: https://gitlab.com/stcarrez/ada-ado

    Ada EL Library, Version 1.8.1

    • Fix compilation issue with GNAT 2020

    Download: http://download.vacs.fr/ada-el/ada-el-1.8.1.tar.gz

    GitHub: https://github.com/stcarrez/ada-el

    GitLab: https://gitlab.com/stcarrez/ada-el

    Ada Utility Library, Version 2.3.0

    • New stream operations to read/write UTF-8 sequences in Wide_Wide character
    • Fix AES encryption in CFB, OFB and CTR modes
    • Add HTTP support for HEAD, OPTIONS, PATCH requests

    Download: http://download.vacs.fr/ada-util/ada-util-2.3.0.tar.gz

    GitHub: https://github.com/stcarrez/ada-util

    GitLab: https://gitlab.com/stcarrez/ada-util

    Ada Security Library, Version 1.3.1

    • Fix AWS example

    Download: http://download.vacs.fr/ada-security/ada-security-1.3.1.tar.gz

    GitHub: https://github.com/stcarrez/ada-security

    GitLab: https://github.com/stcarrez/ada-security

    All these Ada projects can be downloaded individually but they are also packaged together to help in their download and build process. You can also download everything at http://download.vacs.fr/ada-awa/awa-all-1.1.0.tar.gz

    After downloading the awa-all-1.1.0.tar.gz package, have a look at the Ada Web Application Programmer's Guide to learn how to build, install and start using all this.

    If you don't have time to build all this, a docker container is available: https://hub.docker.com/r/ciceron/ada-awa/

    Debian packages

    I've created and setup a Debian repository to give access to several Debian packages for several Ada projects that I manage. The goal is to provide some easy and ready to use packages to simplify and help in the installation of various Ada libraries.

    Access to the repository

    The repository packages are signed with GPG. To get the verification key and setup the apt-get tool, you should run the following command:

    wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
    

    Ubuntu 18.04 Bionic Beaver

    A first repository provides Debian packages targeted at Ubuntu 18.04 bionic. They are built with the gnat-7 package and depend on libgpr1-dev, libaws3.3.2.2-dev and libxmlada-sax7-dev. Add the following line to your /etc/apt/sources.list configuration:

    deb https://apt.vacs.fr/ubuntu-bionic bionic main
    

    Ubuntu 20.04 Focal Fossa

    A second repository provides Debian packages targeted at Ubuntu 20.04 focal. They are built with the gnat-9 package and depend on libgnatprj7-dev, libaws19-dev and libxmlada-sax9-dev. Add the following line to your /etc/apt/sources.list configuration:

    deb https://apt.vacs.fr/ubuntu-focal focal main
    

    Installation

    Once you've added the configuration line, you can install the packages:

    sudo apt-get update
    sudo apt-get install -y dynamo akt libawa2-dev libawa-unit2-dev
    

    The Ada Web Application framework contains several plugins which are provided by separate Debian packages.

    sudo apt-get install -y libawa-blogs2-dev libawa-questions2-dev libawa-wikis2-dev libawa-setup2-dev
    

    The documentation is packaged in separate Debian packages, you can install them with:

    sudo apt-get install -y libawa-doc libado-doc libawa-doc libkeystoreada-doc libsecurity-doc libutilada-doc libwikiada-doc
    

    For the curious, you may browse the repository here.

    Easy reading and writing files with Ada Utility Library

    9 August 2020 at 20:49

    The Ada Utility Library provides several simple operations that simplify the reading and writing of files through a single procedure call. These operations are not new since I've implemented most of them 10 years ago!!!

    Reading a file line by line by using Ada.Text_IO is quite easy but annoying due to the fact that you have to open the file, iterate over the content getting one line at a time and then closing the file. To simplify this process, the Util.Files package of Ada Utility Library provides a simple Read_File procedure that uses a procedure as parameter and that procedure will be called for each line that is read.

    with Util.Files;
    
      procedure Read_Line (Line : in String) is ...;
    
      Util.Files.Read_File (Path => "file.txt",
                            Process => Read_Line'Access);
    

    Another Read_File procedure allows to read the file and get its content in a vector of strings so that once the file is read the vector contains each line. Yet, another Read_File procedure reads the file content in an Unbounded_String. You will use that later form as follows:

    with Util.Files;
    with Ada.Strings.Unbounded;
    
      Content : Ada.Strings.Unbounded.Unbounded_String;
    
      Util.Files.Read_File (Path => "file.txt",
                            Into => Content);
    

    Very often it is also necessary to write some content in a file. Again, this is easy to do but a simple Write_File procedure that takes the file path and the content to write is easier to use in several cases:

    with Util.Files;
    
      Util.Files.Write_File (Path => "file.txt",
                             Content => "something");
    

    The Ada Utility Library contains other useful operations and features that have helped me in implementing various projects such as Ada Web Application and Ada Keystore. Have a look at the Ada Utility Library Programmer's Guide!

    Ada Stemmer Library

    16 May 2020 at 07:55

    Stemming is not new as it was first introduced in 1968 by Julie Beth Lovis who was a computational linguist that created the first algorithm known today as the Lovins Stemming algorithm. Her algorithm has significantly influenced other algorithms such as the Porter Stemmer algorithm which is now a common stemming algorithm for English words. These algorithms are specific to the English language and will not work for French, Greek or Russian.

    To support several natural languages, it is necessary to have several algorithms. The Snowball stemming algorithms project provides such support through a specific string processing language, a compiler and a set of algorithms for various natural languages. The Snowball compiler has been adapted to generate Ada code (See Snowball Ada on GitHub).

    The Ada Stemmer Library integrates stemming algorithms for: English, Danish, Dutch, French, German, Greek, Italian, Serbian, Spanish, Swedish, Russian. The Snowball compiler provides several other algorithms but they are not integrated yet: their integration is left as an exercise to the reader.

    Stemmer Overview

    Snowball is a small string processing language designed for creating stemming algorithms for use in Information Retrieval. A Snowball script describes a set of rules which are applied and checked on an input word or some portion of it in order to eliminate or replace some terms. The stemmer will usually transform a plural into a singular form, it will reduce the multiple forms of a verb, find the noun from an adverb and so on. Romance languages, Germanic languages, Scandinavian languages share some common rules but each language will need its own snowball algorithm. The Snowball compiler provides a detailed list of several stemming algorithms for various natural languages. This list is available on: https://snowballstem.org/algorithms/

    C

    The Snowball compiler reads the Snowball script and generates the stemmer implementation for a given target programming language such as Ada, C, C#, Java, JavaScript, Go, Python, Rust. The Ada Stemmer Library contains the generated algorithms for several natural languages. The generated stemmers are not able to recognize the natural language and it is necessary to tell the stemmer library which natural language you wish to use.

    The Ada Stemmer Library supports only UTF-8 strings which simplifies both the implementation and the API. The library only uses the Ada String type to handle strings.

    Setup

    To use the library, you should run the following commands:

      git clone https://github.com/stcarrez/ada-stemmer.git
      cd ada-stemmer
      make build install
    

    This will fetch, compile and install the library. You can then add the following line in your GNAT project file:

      with "stemmer";
    

    Stemming examples

    Each stemmer algorithm works on a single word at a time. The Ada Stemmer Library does not split words. You have to give it one word at a time to stem and it returns either the word itself or its stem. The Stemmer.Factory is the multi-language entry point. The stemmer algorithm is created for each call. The following simple code:

      with Stemmer.Factory; use Stemmer.Factory;
      with Ada.Text_IO; use Ada.Text_IO;
        ...
        Put_Line (Stem (L_FRENCH, "chienne"));
    

    will print the string:

     chien
    

    When multiple words must be stemmed, it may be better to declare the instance of the stemmer and use the same instance to stem several words. The Stem_Word procedure can be called with each word and it returns a boolean that indicates whether the word was stemmed or not. The result is obtained by calling the Get_Result function. For exemple,

      with Stemmer.English;
      with Ada.Text_IO; use Ada.Text_IO;
      ..
        Ctx : Stemmer.English.Context_Type;
        Stemmed : Boolean;
        ..
        Ctx.Stem_Word ("zealously", Stemmed);
        if Stemmed then
           Put_Line (Ctx.Get_Result);
        end if;
    

    Integrating a new Stemming algorithm

    Integration of a new stemming algorithm is quite easy but requires to install the Snowball Ada compiler.

      git clone --branch ada-support https://github.com/stcarrez/snowball
      cd snowball
      make
    

    The Snowball compiler needs the path of the stemming algorithm, the target programming language, the name of the Ada child package that will contain the generated algorithm and the target path. For example, to generate the Lithuanian stemmer, the following command can be used:

      ./snowball algorithms/lithuanian.sbl -ada -P Lithuanian -o stemmer-lithuanian
    

    You will then get two files: stemmer-lithuanian.ads and stemmer-lithuanian.adb. After integration of the generated files in your project, you can access the generated stemmer with:

      with Stemmer.Lithuanian;
      ..
        Ctx : Stemmer.Lithuanian.Context_Type;
    

    Conclusion

    Thanks to the Snowball compiler and its algorithms, it is possible to do some natural language analysis. Version 1.0 of the Ada Stemmer Library being available on GitHub, it is now possible to start doing some natural language analysis in Ada!

    New version of Ada Web Application

    1 May 2020 at 20:49

    The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.

    AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.

    A typical architecture of an AWA application is represented by the picture below:

    ((Ada/awa_architecture_overview.png|awa_architecture_overview.png|C|Ada Web Application architecture)

    This version of AWA integrates smoothly with Ada Keystore in order to protect the server sensitive configuration.

    Ada Web Application, Version 2.0

    • Refactoring of build process and installation
    • New audit manager for database auditing
    • Support for Postgresql
    • Improvements of images and storage plugins
    • Update Trumbowyg editor to version 2.18.0
    • Update Flot library to version 4.2.0
    • Support for commands to configure, start, stop the server
    • New mail UI component <mail:attachment> to send attachments

    Dynamo, Version 1.0.0

    • Improvement and fixes in the YAML database model files
    • Add support for Nullable_String type
    • Generate Postgresql SQL files from the model files
    • Add support for database record auditing
    • Add support for floating point
    • Add support for CSS and Javascript merge in the dist command

    Ada Database Objects, Version 2.1.0

    • Added Is_Modified predicate on database objects
    • Fix SQLite Load_Schema to avoid loading SQLite specific tables
    • Support for Postgresql database
    • Improvement for errors reported by database drivers
    • New audit framework to track database record changes
    • Added support for floating point numbers
    • Serialize queries in JSON/XML streams

    Ada Keystore, Version 1.2.0

    • Added support for Fuse with a new mount command in akt (beta!)
    • Fix the implementation to iterate with Keystore.Properties

    Ada Server Faces, Version 1.4.0

    • Performance improvement for the Facelet cache
    • Integrate jQuery 3.4.1, jQuery UI 1.12.1, jQuery Chosen 1.8.7
    • New <f:validateRegex> to validate an input field with a regular expression

    Ada Utility Library, Version 2.2.0

    • New Wait_Empty operation on fifo.
    • Add Get_Count and Wait operation on executors

    Ada EL Library, Version 1.8.0

    • New Expand procedure to expand the properties in place

    Ada Wiki Library, Version 1.2.1

    • Minor configuration and code coverage support
    • Corrections in the Markdown syntax parser

    Ada Security Library, Version 1.3.0

    • Add support to extend the authenticate manager and allow to use custom authentication through the Set_Default_Factory operation.

    Ada Servlet, Version 1.4.0

    • Added support to configure the web container

    AKT a tool to store and protect your sensitive information

    26 December 2019 at 17:47

    AKT stores information in secure wallets and protects the stored information by encrypting the content with different keys. AKT can be used to safely store passwords, credentials, bank accounts, documents and even directories.

    Wallets are protected by a master key using AES-256 and the wallet master key is protected by a user password or a user GPG encrypted key. The wallet defines up to 7 slots that identify a password key that is able to unlock the master key. To open a wallet, it is necessary to unlock one of these 7 slots by providing the correct password. Wallet key slots are protected by the user's password and the PBKDF2-HMAC-256 algorithm, a random salt, a random counter and they are encrypted using AES-256.

    C

    Values stored in the wallet are protected by their own encryption keys using AES-256. A wallet can contain another wallet which is then protected by its own encryption keys and passwords (with 7 independent slots). Because the child wallet has its own master key, it is necessary to known the primary password and the child password to unlock the parent wallet first and then the child wallet.

    The data is organized in 4K blocks whose primary content is encrypted either by the wallet master key or by the entry keys. The data block is signed by using HMAC-256. A data block can contain several values but each of them is protected by its own encryption key. Each value is also signed using HMAC-256.

    The keystore uses several encryption keys at different levels to protect the content. A document stored in the keystore is split in data fragment and each data fragment is encrypted by using its own key. The data fragments are stored in specific data blocks so that they are physically separated from the encryption keys.

    The data fragment encryption keys are stored in the directory blocks and they are encrypted by using a specific directory key.

    akt-keys.png

    For example, a 10K document will be split in 3 data fragments, each of them encrypted by their own AES-256 key. A 5K document will be encrypted with two AES-256 keys, one for the two data fragments. All these keys are protected by the wallet data key key. The directory part of the wallet which describes entries in the wallet is also encrypted by another wallet key: the directory key.

    The tool allows to separate the data blocks which contain data fragments from other blocks. This allows to keep the wallet keys separate from the data. It is then possible to export the data blocks, which are encrypted in AES-256-CBC, to the Cloud without exposing the keys used for encryption.

    If you want to know more about the implementation, have a look at the Ada Keystore Implementation chapter.

    Using AKT

    akt is the command line tool that you can use to protect and store your documents. It contains several commands:

    • create: create the keystore
    • edit: edit the value with an external editor
    • get: get a value from the keystore
    • help: print some help
    • list: list values of the keystore
    • remove: remove values from the keystore
    • set: insert or update a value in the keystore

    To create the secure file, use the following command and enter your secure password (it is recommended to use a long and complex password):

      akt create secure.akt
    

    You may also protect the keystore by using your GPG key. In that case, you can use the --gpg option and specify one or several GPG key ids. Using GPG is probably the best method to protect your akt files.

      akt create secure.akt --gpg 0xFC15CA870BE470F9
    

    At this step, the secure file is created and it can only be opened by providing the password you entered. To add something, use:

      akt set secure.akt bank.password 012345
    

    To store a file, use the following command:

      akt store secure.akt contract.doc
    

    If you want to retrieve a value, you can use one of:

      akt get secure.akt bank.password
      akt extract secure.akt contract.doc
    

    You can also use the akt command together with the tar command to create secure backups. You can create the compressed tar file, pipe the result to the akt command to store the content in the wallet.

      tar czf - dir-to-backup | akt store secure.akt -- backup.tar.gz
    

    To extract the backup you can use the extract command and feed the result to the tar command as follows:

      akt extract secure.akt -- backup.tar.gz | tar xzf -
    

    Using Ada Keystore

    The Ada Keystore is the Ada 2012 library that is behind AKT. It should be quite easy to integrate the library in an existing Ada application to protect for example some sensitive configuration file. The Keystore is the main package that provides operations to store information in secure wallets and protect the stored information by encrypting the content. To use it, add the following with clause at beginning of your GNAT project:

       with "keystoreada";
    
    Creation

    To create a keystore you will first declare a Wallet_File instance. You will also need a password that will be used to protect the wallet master key.

    with Keystore.Files;
    ...
      WS   : Keystore.Files.Wallet_File;
      Pass : Keystore.Secret := Keystore.Create ("There was no choice but to be pioneers");
    

    You can then create the keystore file by using the Create operation:

      WS.Create ("secure.akt", Pass);
    
    Storing

    Values stored in the wallet are protected by their own encryption keys using AES-256. The encryption key is generated when the value is added to the wallet by using the Add operation.

      WS.Add ("Grace Hopper", "If it's a good idea, go ahead and do it.");
    

    The Get function allows to retrieve the value. The value is decrypted only when the Get operation is called.

      Citation : constant String := WS.Get ("Grace Hopper");
    

    The Delete procedure can be used to remove the value. When the value is removed, the encryption key and the data are erased.

      WS.Delete ("Grace Hopper");
    

    Getting AKT

    You can get AKT by using the Ubuntu 18.04 binary packages. You can do this by running:

    wget -O - http://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
    sudo add-apt-repository "deb http://apt.vacs.fr/ubuntu-bionic bionic main"
    sudo apt-get install akt
    

    For other platforms, you have to get it from the sources. Install the GNAT Ada compiler, either the FSF version or the GNAT GPL version and then, run the following commands:

    git clone --recursive https://github.com/stcarrez/ada-keystore.git
    cd ada-keystore
    ./configure --disable-nls
    make build install
    

    You can browse the documentation online: Ada Keystore Guide.

    Ada, Java and Python database access

    17 November 2018 at 14:02

    The database also has a serious impact on such benchmark and I've measured the following three famous databases:

    The purpose of the benchmark is to be able to have a simple comparison between these different databases and different programming languages. For this, a very simple database table is created with only two integer columns one of them being the primary key with auto increment. For example the SQLite table is created with the following SQL:

    CREATE table test_simple (
      id INTEGER PRIMARY KEY AUTOINCREMENT,
      value INTEGER
    )
    

    The database table is filled with a simple INSERT statement which is also benchmarked. The goal is not to demonstrate and show the faster insert method, nor the faster query for a given database or language.

    Benchmark

    The SQL benchmarks are simple and they are implemented in the same way for each language so that we can get a rough comparison between languages for a given database. The SELECT query retrieves all the database table rows but it includes a LIMIT to restrict the number of rows returned. The query is executed with different values for the limit so that a simple graph can be drawn. For each database, the SQL query looks like:

    SELECT * FROM test_simple LIMIT 10
    

    The SQL statements are executed 10000 times for SELECT queries, 1000 times for INSERT and 100 times for DROP/CREATE statements.

    Each SQL benchmark program generates an XML file that contains the results as well as resource statistics taken from the /proc/self/stat file. An Ada tool is provided to gather the results, prepare the data for plotting and produce an Excel file with the results.

    Python code
    
    def execute(self):
      self.sql = "SELECT * FROM test_simple LIMIT " + str(self.expect_count)
      repeat = self.repeat()
      db = self.connection()
      stmt = db.cursor()
    
      for i in range(0, repeat):
        stmt.execute(self.sql)
        row_count = 0
        for row in stmt:
          row_count = row_count + 1
    
        if row_count != self.expect_count:
          raise Exception('Invalid result count:' + str(row_count))
    
        stmt.close()
    
    Java code
    public void execute() throws SQLException {
      PreparedStatement stmt
     = mConnection.prepareStatement("SELECT * FROM test_simple LIMIT " + mExpectCount);
    
      for (int i = 0; i < mRepeat; i++) {
        if (stmt.execute()) {
          ResultSet rs = stmt.getResultSet();
          int count = 0;
          while (rs.next()) {
            count++;
          }
          rs.close();
          if (count != mExpectCount) {
            throw new SQLException("Invalid result count: " + count);
          }
        } else {
          throw new SQLException("No result");
        }
      }
      stmt.close();
    }
    
    Ada code
    procedure Select_Table_N (Context : in out Context_Type) is
       DB    : constant ADO.Sessions.Master_Session := Context.Get_Session;
       Count : Natural;
       Stmt  : ADO.Statements.Query_Statement
            := DB.Create_Statement ("SELECT * FROM test_simple LIMIT " & Positive'Image (LIMIT));
    begin
       for I in 1 .. Context.Repeat loop
          Stmt.Execute;
          Count := 0;
          while Stmt.Has_Elements loop
             Count := Count + 1;
             Stmt.Next;
          end loop;
          if Count /= LIMIT then
             raise Benchmark_Error with "Invalid result count:" & Natural'Image (Count);
          end if;
       end loop;
    end Select_Table_N;
    

    The benchmark were executed on an Intel i7-3770S CPU @3.10Ghz with 8-cores running Ubuntu 16.04 64-bits. The following database versions are used:

    • MariaDB 10.0.36
    • PostgreSQL 9.5.14

    Resource usage comparison

    The first point to note is the fact that both Python and Ada require only one thread to run the SQL benchmark. On its side, the Java VM and database drivers need 20 threads to run.

    The second point is not surprising: Java needs 1000% more memory than Ada and Python uses 59% more memory than Ada. What is measured is the the VM RSS size which means this is really the memory that is physically mapped at a given time.

    The SQLite database requires less resource than others. The result below don't take into account the resource used by the MariaDB and PostgreSQL servers. At that time, the MariaDB server was using 125Mb and the PostgreSQL server was using 31Mb.

    sql-memory.png

    Speed comparison

    Looking at the CPU time used to run the benchmark, Ada appears as a clear winner. The Java PostgreSQL driver appears to be very slow at connecting and disconnecting to the database, and this is the main reason why it is slower than others.

    sql-time.png

    It is interesting to note however that both Java and Python provide very good performance results with SQLite database when the number of rows returned by the query is less than 100. With more than 500 rows, Ada becomes faster than others.

    sql-sqlite.png

    With a PostgreSQL database, Ada is always faster even with small result sets.

    sql-postgresql.png

    sql-mysql.png

    Conclusion and references

    SQLite as an embedded database is used on more than 1 billion of devices as it is included in all smartphones (Android, iOS). It provides very good performances for small databases.

    With client-server model, MariaDB and PostgreSQL are suffering a little when compared to SQLite.

    For bigger databases, Ada provides the best performance and furthermore it appears to be more predictable that other languages (ie, linear curves).

    The Excel result file is available in: sql-benchmark-results.xls

    Sources of the benchmarks are available in the following GitHub repository:

    ❌
    ❌