Project for an Autonomous Sailing Boat



  • My final project consist to build an autonomous sailing boat to participate to the Microtransat Challenge.
    My boat has to navigate from the Noirmoutier Island (West coast of France) to Martinique island (West Indies) across Atlantic.

    Actually my inboard software consist in 3 process.
    1 - the brain software written in 4300 lines of Javascript/Nodejs and running with a PI0.
    2 - the software for an independent Wingsail on ESP32 C++
    3 - the software for the rudder on ESP32 C++

    My actual difficulties consists in optimization of power consumption of the PI0 in Nodejs.

    I would like to test the possibility to replace the PI0 by an ESP32/Neonious One.

    My first questions ?

    • the source compatibility between nodejs and low ?
    • the possibility to store my program in the user flash memory, (with extension?)
    • the possibility to install and store npm packages dependencies
    • the possibility to deep-sleep/hibernate the ESP32
    • the possibility to use Neonious One if the power consumption en 3V is very low

    I will give some informations or trick about my progress.



  • First step : success !

    I had only to find walkarouds for fs.readdir() and fs.truncate().

    I transpile all my javascript source with my Mac, not the Neonious One, using gulp

    const gulp = require('gulp');
    const babel = require('gulp-babel');
    
    gulp.task('default', function(){
        return gulp.src(['src/**/*.js'])
            .pipe(babel({presets: ['@babel/preset-env']}))
            .pipe(gulp.dest('dist'));
    });
    

    With the transpiled version, I can execute well on my Mac with low all this software as it runs with node.

    Good !



  • Second step : success !

    I was afraid by the size of my javascript software.
    It must store in the user Flash memory file system 4MB.

    In // I bought W25Q128JVSIQTR flash memory to have the possibility in the future to extends this storage to 12MB if necessary.

    To reduce the size of source code, I tries first to suppress the comments.
    But in fact I decide to compact more and used uglify to suppress spaces, CRLF,....

    const gulp = require('gulp');
    const babel = require('gulp-babel');
    const gp_uglify = require('gulp-uglify');
    
    gulp.task('default', function(){
        return gulp.src(['src/**/*.js'])
            .pipe(babel({presets: ['@babel/preset-env']}))
            .pipe(gp_uglify())
            .pipe(gulp.dest('/Users/yvesgageot/PI/microtransat-lowsync/dist'));
    });
    

    It is very efficient.

    I used lowsync to push to the mcu

    lowsync  --no-transpile --restart=false
    

    Fabulous, le total size used is displayed as 284KB. Very small !!!!

    Good !



  • Once more !

    My main program need to snapshot a global object.
    When stringified it spend 1500-3000 Bytes non compressed.

    The user Flash file system where the file is stored is documented as compressed.

    Thomas help me.
    He proposed to use 'rc' flag in fs.fstats() to display the true physical size.

    It was a pleasure to see a 1445 Bytes using only 598 Bytes in the storage.

    Good !



  • One more thing :

    For an http client, I used before the package request, a very fat package.
    Thomas advise me to use the package axios.

    • I saved a lot of module storage,
    • the syntax is simple and clean
    • I can use the integrated timeout feature and don't use mine

    Simple !



  • Sounds like a fun project.

    We have some companies who we actively talk to and now use low.js and who now pay our bills as we adapt low.js to their needs. Hobbyists on the other hand do not use low.js much yet, as the word has not spread much yet.

    So I am very interested to help you to get this done. I am happy to hear about any problems you have in the future..



  • Thanks Thomas.

    I made some progress :

    • I made integration of two external packages in my project because I had NPM install troubles (Cf. forum) , it was basic for me.
    • I have no problem to store javascript sources in Flash memory (I uglified the sources)
    • I had more problems with RAM but I did some modifications. The display graph values start vers 1.3Mb to go to 3.3Mb, I leave the garbage collector as it is. But it is the limit !
    • There is a starting delay of 80 seconds. I presume to generate the byte codes. It is a main problem if I want to use a deep-sleep mode. Is it possible to store it somewhere ?

    Globally I stay confident !



  • @ygageot regarding the starting delay...

    Yes the bytecode compilation takes too long.

    I think the thing to do is to add a precompile step to lowsync.. This would not be too difficult.

    What’s your time schedule for your project. As I wrote in other posts, I will start having time for low.js again in January..

    Thomas



  • @neoniousTR
    With the size of my JavaScript source, it is not easy to change and debug on the esp32.
    When I try to optimize memory usage, I test firs on my Mac with Node, minify automatically the source and lowsync...
    It is not important for me to modify the source with IDE.
    If lowsync generate bytecodes and store it in Flash memory it’s ok for me.
    The bytecode seems to be larger than Javascript.

    Questions :

    • Is the bytecode loaded completely in RAM for execution ?
    • Is there some trick to reduce the global size of the bytecode ?


  • You can split up code in multiple files and only require what is needed. But yes, what is required stays in RAM



  • Now I have more than 40 files + external modules in production
    plus tests files not used in production

    They are all useful. But some of them are used only periodically.
    I can imagine for the functions used with a low frequency :

    • To ask late requires in place of requires at the beginning of each module.
    • Execute the functions
    • Invalidate the module when it is not used with delete require.cache
    • Why not gc()

    It is not very simple because my modules are uses in a network way not hierarchicly.



  • This post is deleted!


  • No, this is just syntax sugar, would not change anything.

    I think with the custom firmware build you will see drastic improvements in loading speed.



  • OK the feature is ready, see https://www.neonious.com/lowjs/documentation/custom-firmware.html

    Can you try to set up your project as a custom firmware project with the code files being static files? This drastically improves the performance, because the file system is slow...



  • Thomas thanks a lot,
    During this period, I worked hard to simplify the architecture and the reduce the memory usage.
    It works better and longer.
    The garbage collection delete 1MB periodicly.
    I stop now the optimisation.
    And I will look at your last development to obtain a fast startup time.



  • I just made a first test with the configuration file :

    {
      "lowjs": {
        "version": "20200105",
        "pro": true,
        "system_flash_size": 8388608,
        "ide_support": false,
        "ota_update_support": false
      },
      "static_files": null,
      "factory_files": null,
      "modules": {
        "async": "^3.1.0",
        "axios": "^0.19.0",
        "kalmanjs": "^1.1.0",
        "pako": "^1.0.10"
      },
      "settings": {
        "code": {
          "main": "/dist/index.js",
          "auto_restart_on_fatal": false,
          "console_kb": 0,
          "only_static_files": false
        },
        "wifi": {
          "ssid": "****************",
          "password": "*********"
        },
        "web": {
          "http_enabled": false,
          "https_enabled": false
        }
      }
    }
    

    I obtained

    Building firmware...
    ****** Used flash space: ******
    low.js code       2097152 bytes
    low.js data       3365605 bytes
    Static files            0 bytes
    Factory files           0 bytes
    Modules            498505 bytes
    Settings             1148 bytes
    Reserved          2426198 bytes
    -------------------------------
    Total             8388608 bytes
    

    and during the flash step :

    Error: No port specified. Please use --port=.. to specify the port.
    

    What port is it ?



  • Sorry ! I looked only the doc on the blog.
    Reading the full documentation and Github, I red it was necessary to connect the Neonious One via USB port.
    I continue...



  • Let me stop you right here!

    I did a mistake! neonious one is still based on the idea that the user will never need to flash low.js. Just update over Wifi/Ethernet and add user data via lowsync. So it does not have an USB/UART adapter! And I forgot that you are working with the neonious one. With the "C/C++ Hacking Kit" you can Flash, but the neonious one needs a special low.js version because of the LPC coprocessor, so the custom firmware will not work this way either.

    I guess you like the neonious one because of the Ethernet port and the additional Flash? In that case.. we are working on the successor of the neonious one, to streamline everything and have the same possibilities with all hardware. Till mid February we should have a new board - same size, still with Ethernet, 4 MB more Flash, and Custom Firmware will work here, too.

    If you do not care about Ethernet/the amount of ports, a third-party ESP32-WROVER board with 16 MB Flash might be another option. Here the custom firmware would work right now.

    Second thing: Your configuration

    The config file you show above has no static files, no factory files, so no user files at all => no user program will run. Also no port for lowsync or neonious IDE enabled (web.http_enabled/web.https_enabled). So, it would show a Wifi access point, but there will be now way to interact with the board after connecting to the AP.



  • In fact I bought a Neonious One to save time to check the possibility to run my Nodejs software on ESP32.

    I used the Neonious One ethernet port only to set the Wifi network and after it was easier for me to work via Wifi and have the processor near my desktop.

    My final goal is to use an ESP32 with minimum power usage. It means :

    • no extra hardware.
    • a solar power management
    • deep sleep or hibernate management

    For the moment my target configuration :

    In the hull :

    • 1 large ESP32 running lowjs as the main brain with no sensors.
    • 1 small ESP32 with sensors IMU,GPS,RF24 and servo written in C++ using deep-sleep, wakeup via timer&gpio
      interconnected actually via Wifi, tested BLE but it can be no Serial.
    • a satellite gateway

    In the independant wingsail :

    • 1 small ESP32 with servo written in C++ using deepsleep, wakeup and data via RF24, solar management.

    I have to find a minimum ESP32-WROVER but with large Flash memory



  • @neoniousTR I don't know our mapping of the flash memories.

    If I use a standard ESP32-WROVER with 4Mb flash as in your store : How the flash memory is used ?

    If fact I need only the place for the firmware and a place to store and update 2/3 Kb of permanent data.
    No IDE, no ETA for the final configuration.

    Is it possible ?


Log in to reply