Sqlite µLogger

SparkFun microSD Shield

Arduino UNO

<p >This repo is an Arduino library that can work with Arduino Uno board or any Arduino board that has minimum 2kb RAM and a SD Shield attached.</p><p >It has been tested with Arduino Uno with SparkFun MicroSD Shield, WeMos ESP8266 D1 Mini with WeMos MicroSD Shield and ESP32 SD_MMC breakout board.</p><figure><img src=https://projects.arduinocontent.cc/f0b2f233-9216-4d03-9552-a3a2021922b3.png /><figcaption> </figcaption></figure><h3 >Features</h3><ul ><li ><span >Low Memory requirement: </span><code >page_size</code><span > + some stack</span></li></ul><ul ><li ><span >Can log using Arduino UNO (</span><code >2kb</code><span > RAM) with 512 bytes page size</span></li></ul><ul ><li >Can do quick binary search on RowID or Timestamp without any index in logarithmic time</li></ul><ul ><li >Recovery possible in case of power failure</li></ul><ul ><li >Rolling logs are possible (not implemented yet)</li></ul><ul ><li >Can use any media using any IO library/API or even network filesystem</li></ul><ul ><li >DMA writes possible (not shown)</li></ul><ul ><li >Virtually any board and any media can be used as IO is done through callback functions.</li></ul><h3 >Getting started</h3><p ><span >The example </span><code >Uno_and_above</code><span > shows how data read from Analog pins can be stored along with Timestamp into Sqlite database and retrieved by RowId.</span></p><p >Records can also be located using Timestamp in logarithmic time by doing a Binary Search on the data logged. This is not possible using conventional loggers.</p><p >For example, locating any record in a 70 MB db having 1 million records on Arduino UNO with SparkFun microSD Shield took only 1.6 seconds.</p><p ><span >The examples </span><code >ESP8266_Console</code><span > and </span><code >ESP32_Console</code><span > can be used to log and retrieve from ESP8266 and ESP32 boards respectively on Micro SD and SPIFFS filesystems.</span></p><h3 >API</h3><p ><span >For finding out how the logger works and a complete description of API visit </span><a href="https://github.com/siara-cc/sqlite_micro_logger_c">Sqlite Micro Logger C Library.</a></p><h3 >Ensuring integrity</h3><p ><span >If there is power failure during logging, the data can be recovered using </span><code >Recover database</code><span > option in the menu.</span></p><h3 >Examples</h3><h3 >Arduino Uno</h3><p >This screenshot shows how analog data can be logged and retrieved using Arduino Uno and Sparkfun Micro SD Shield:</p><figure><img src=https://projects.arduinocontent.cc/727ac4bb-d063-45f7-b259-df028c44d373.png /><figcaption> </figcaption></figure><p >This screenshot shows how binary search can be performed on the timestamp field:</p><figure><img src=https://projects.arduinocontent.cc/3be43044-485a-4bbd-a19d-6bd81d6735dc.png /><figcaption> </figcaption></figure><h3 >ESP8266</h3><p >This screenshot shows how analog data can be logged and retrieved using ESP8266 (WeMos D1 Mini and Micro SD Shield):</p><figure><img src=https://projects.arduinocontent.cc/e3d052fe-6f1a-41a0-8422-765ee06bc572.png /><figcaption> </figcaption></figure><p >This screenshot shows how binary search can be performed on the timestamp field using ESP8266:</p><figure><img src=https://projects.arduinocontent.cc/d32d4fc1-2615-4a6c-92d2-8ca51b01c96a.png /><figcaption> </figcaption></figure><h3 >ESP32</h3><p >This screenshot shows how analog data can be logged and retrieved using ESP32 breakout board having a Micro SD Slot on the SD_MMC port:</p><figure><img src=https://projects.arduinocontent.cc/512a0704-ae0c-41d5-b69a-e625fec00f6d.png /><figcaption> </figcaption></figure><p >This screenshot shows how binary search can be performed on the timestamp field using ESP32:</p><figure><img src=https://projects.arduinocontent.cc/31408f1f-5d35-4c65-aa59-0f1dcbc962d5.png /><figcaption> </figcaption></figure><h3 >Limitations</h3><p >Following are limitations of this library:</p><ul ><li >Only one table per Sqlite database</li></ul><ul ><li ><span >Length of table script limited to (</span><code >page size</code><span > - 100) bytes</span></li></ul><ul ><li ><code >Select</code><span >, </span><code >Insert</code><span > are not supported. Instead C API similar to that of Sqlite API is available.</span></li></ul><ul ><li >Index creation and lookup not possible (as of now)</li></ul><p >However, the database created can be copied to a desktop PC and further operations such as index creation and summarization can be carried out from there as though its a regular Sqlite database. But after doing so, it may not be possible to use it with this library any longer.</p><h3 >Future plans</h3><ul ><li >Index creation when finalizing a database</li></ul><ul ><li >Allow modification of records</li></ul><ul ><li >Rolling logs</li></ul><ul ><li >Show how this library can be used in a multi-core, multi-threaded environment</li></ul><h3 >Support</h3><p >If you find any issues, please create an issue here or contact the author (Arundale Ramanathan) at </p>

Sqlite µLogger

Sqlite µLogger is a fast and lean database logger that can log data into Sqlite databases even with SRAM as low as 2kb as in an Arduino Uno.