RealTerm Data Logger

This Instructable concerns data-logging using RealTerm monitor.
Data logging can be done in many ways as the number of available Instructables here will show but I had a unit that gave serial ASCII data which is explained on my previous Instructable:

Temperature-and-Humidity-Data-Transmitter

The Sensor that I am using gives a reading every 2 seconds but I only require readings every 30 minutes. We can use a terminal programme such as Hyperterminal to get the data but this will simply log all traffic i.e every 2 seconds. There are usually no methods of filtering data. Some terminal programmes are available with Macros and there are methods available to edit a file to extract specified data but that adds an extra stage of translation.

So, to avoid the addition of a control processor or further complication, I looked around the Web for solutions and found RealTerm.

Realterm offers a wide range of monitors and control options. The formats supported include

Serial Ports, USB Serial and TCP/IP & Telnet, I2C Bus, SPI& 1-Wire chip control via BL233B / I2C2PC

RealTerm seemed like the ideal solution as it offered data collection, filtering and re-start. Unfortunately for me, only the later versions offer re-start and I was only able to use the older version. This instructable explains how I managed to make a file of received data on 30 minute intervals using a Batch file.

RealTerm is available for several software platfoms and I understand that this method can be adapted to each but this method relates to Windows XP and RealTerm version 2.

The picture above shows the problem: a steam of data in a Window, Hyperteminal & RealTerm examples.

So, we need:

a Personal Computer or other supported processor

Realterm_2.0.0.70_Signed_Wrapper_setup.exe (RealTerm monitor)

A data source i.e a sensor or processor that is supported by RealTerm communications

A Word Processor with "Find & replace"

There are four stages to get from a data stream to usable data;

and a final step to check all is working and then run the Batch file:

From Web site

RealTerm Source

and read all information from here:

RealTerm Information

I needed the Utility SLEEP.EXE from the same source as the "re-start" function was not available on the RealTerm version for WINDOWS XP and that is why I had to write this simple BATCH programme; you might be lucky in using a later version but I used an older computer for this simple task.

The idea being that after the terminal programme was started then I would stop and re-start every 30 minutes.

Some later versions of Microsoft WINDOWS have a TIMEOUT function for batch files which did appear to be difficult to use so you might like to use SLEEP.EXE anyway.

This picture shows the "capture" tab of RealTerm.

There are many useful features like naming the "capture" file, adding a time-stamp and a choice of record separator.

A very important feature is the character count to seperate each record as it is received.

Our Batch file is going to set all the required parameters for capture and communication.

The Batch file collects and saves the file data and also controls the timing of the next reading.

The Batch file contains the instructions to start RealTerm with the configuration required and a 30 minute delay before repeating.

Download the sample "RealTermTest.bat" or copy the lines into 'Notepad' or Word Processor of your choice and save it with the configuration that YOU need, this one is for me!

To make and save a Batch file you use the extension .bat e.g "RealTerm.bat"; make sure to use the quotation marks to ensure that the Microsoft environment knows not to change it. i.e RealTerm.bat (no quotation marks) will get saved as RealTerm.bat.txt

@Batch file@

:START

CD C:\Progra~1\BEL\Realterm C:\Progra~1\BEL\Realterm\realterm.exe FIRST=1 baud=9600 port=30 TIMESTAMP=4 CAPCOUNT=11 CAPTURE=2 CAPQUIT=C:\Temp\capture.txt copy C:\Electronics\RealTerm.csv+C:\Temp\capture.txt C:\Electronics\RealTerm.csv

CALL SLEEP.EXE 1800000

GOTO :START

@End Batch file@

Explanation:

":START"

This might seem obvious but it is actually a label and is used to find the start when the Batch file runs to the end.

"CD C:\Progra~1\BEL\Realterm"

This line is to change the working directory to Realterm to find the RealTerm.exe and SLEEP.exe programmes.

"C:\Progra~1\BEL\Realterm\realterm.exe FIRST=1 baud=9600 port=30 TIMESTAMP=4 CAPCOUNT=11 CAPTURE=2 CAPQUIT=C:\Temp\capture.txt"

This line is to launch RealTerm with the configuration needed and to tell it where to save the data In this case port=30 baud rate=9600, data bits 8-N-1 TIMESTAMP=4 adds DD/MM/YYYY to the beginning of the saved data CAPCOUNT= 11 is the character format count, in this case:- 22.0,29.2% (including a "space" character) so the saved data is like "06/04/2018 16:54:40", 22.0,29.2% (including a "space") to C:\Temp\capture.txt CAPQUIT=C:\Temp\capture.txt" tells Real Term to wait for 11 characters received and, after saving, CLOSE

"copy C:\Electronics\RealTerm.csv+C:\Temp\capture.txt C:\Electronics\RealTerm.csv"

This line opens a file, RealTerm.csv and adds the contents of capture.txt; The file RealTerm.csv gets bigger by one reading each time If you look carefully then you will see the "+" sign that adds the two files; the command format being: COPY file1+file2 (to) file1.

"CALL SLEEP.EXE 1800000"

This is a special command that 'calls' the programme SLEEP.EXE whose job is to count 1800000 milli seconds (which is aproximately 30 minutes). So the Batch file is kept waiting, in the background.

"GOTO :START"

The Batch file continues when SLEEP.EXE has finished and finds GOTO and runs again, starting RealTerm once more.

Note: This Batch file is never-ending but can be stopped by using 'CTRL+C' for a y/n prompt (or better programming!). The Computer can be safely shut-down any time when not taking a reading as the files are only open when taking a reading.

My data is in the form:

"06/04/2018 16:54:40", 22.0,29.2% and I want to format it as

06/04/2018,16:54:40,22.0,29.2CRLF

In this case, the data has been saved to "RealTerm.csv".
The file extension .csv is used for "comma seperated variables" which many programmes, particularly spread-sheets, recognise so we need to make sure that the data conforms to this style so that it is easily manipulated. Make a working copy with the same file name extension i.e "Data.csv", "Today.csv" Do not forget to use the quotation marks!

So if the data is in the form: "06/04/2018 16:54:40", 22.0,29.2% and I want 06/04/2018,16:54:40,22.0,29.2CRLF

then that is to say,

1). replace % with carriage return and line feed (which seperates each record onto a new line)

2). remove the quotation marks

3). remove the un-necessary 'space'

4). change the 'space' between Date and Time to a 'comma'

1). You will have to find the "special character" for carriage return and line feed in your Word Processor, in my case I "find&replace" & with ^r

2). "find&replace" quotation mark with nothing

3). "find&replace" comma-space with comma

4). "find&replace" 2018-space with 2018-comma

Saving the file whilst ensuring that the file is saved with the .csv extension will give you a file that looks like this:

04/04/2018,15:40:12,18.9,41.3

04/04/2018,16:20:12,18.9,41.3

04/04/2018,16:40:12,18.4,40.8

04/04/2018,17:20:12,18.5,40.9

04/04/2018,17:40:12,18.5,40.9

Here is a file, "RealTerm.csv" for practice if you like other wise,

Connect your data source and run your Batch file; RealTerm will capture according to your settings and then QUIT until run again by the Batch file.

After you have collected your readings then you need to format it for your use as you wish, as numerical statistics or graphing etc.

Trusting that this is helpful, have fun!

This screen shot shows the 8.3 format filenames with the "long" filenames on the right side.

You might find the same as me that you need to use the 8.3 file name style.

Microsoft WINDOWS gave us "long filenames" but they are still stored in the original 8.3 format i.e NAME5678.EXT

You may have noticed that my Batch file used the short names for Directories so you might have to check how your files have been saved.

This was o.k as the files worked from the Root and within the 8.3 format

C:\Electronics\RealTerm.csv+C:\Temp\capture.txt C:\Electronics\RealTerm.csv

but I had to use C:\Progra~1\BEL\Realterm\realterm.exe as the Directory "Program Files" is not.