How to Store Data Logger Data to a Memory Card

by Carolyn Ivans | Updated: 06/22/2016 | Comments: 7

Blog Topics


Search the Blog


Subscribe to the Blog

Get an email when a new article is posted. Choose the topics that interest you most.


Enter your email address:



Suggest an Article

Is there a topic you would like to learn more about? Let us know.

Leave this field empty

datalogger and microSD card

You may have identified the need to add external data storage to your data logger with flash-memory cards. But do you know which CRBasic instruction to use to do it? In this article, I’ll help you choose the most suitable instruction for your application. In a future article, I’ll discuss some more complex issues and offer handy card tips and tricks that you may not be familiar with.

If you're not that familiar with memory cards, you may find it helpful to review the "Pick a Memory Card, but Not Just Any Memory Card" blog article.

How do I store data to a memory card?

There are two CRBasic instructions that are commonly used to instruct the data logger to write final-storage data to a memory card:

  1. The CardOut() instruction
  2. The TableFile() with Option 64 instruction

These instructions are added between the DataTable() and EndTable() instructions as shown in the examples later in this article. Note that you cannot use both CardOut() and TableFile() Option 64 in the same data table. You must choose which instruction is best for you. So, why are there two instructions to store data to a card? Is one method better than the other?

Before answering these questions, it is important to understand that the maximum size of any one file stored on a data logger is 2 GB. This is because the data logger natively supports signed-4-byte integers. This data type can represent a number as large as 231, or in terms of bytes, roughly 2 GB. Hence, 2 GB is the maximum file length that can be represented in the data logger directory table, and consequently, that is its maximum file size.

Partially because of this file size limitation, and also due to the high cost of large memory cards, Campbell Scientific data loggers did not support memory cards larger than 2 GB prior to operating system (OS) 25 in the CR1000 or CR3000. However, as demands for high-frequency data and greater data storage grew, support for cards up to 16 GB was added to the data logger operating system.

The support for cards larger than 2 GB presented a conundrum. How can we write more than 2 GB of data to a card if the maximum size of a final-storage file is 2 GB? The solution was provided through the addition of a new option (Option 64) to the TableFile() instruction that allows more than one final-storage data file to be created from the same data table. Hence, many small files can be created from the same data table, thus allowing up to 16 GB of final-data storage from one data table.

So, prior to OS 25, only the CardOut() instruction was available to store data to a card, and only one file could be created for each data table. However, with OS 25 and later, TableFile() Option 64 is also available for storing more than one file for a given data table to a card.

Is one method of writing data to a card better than the other?

An advantage of the CardOut() instruction is that it is very simple to add to your program. More information on the CardOut() instruction is available in the CRBasic Editor Help files.

In many applications, however, the TableFile() instruction with Option 64 has several advantages over the CardOut() instruction. These advantages include:

  • Allowing multiple small files to be written from the same data table so that storage for a single table can exceed 2 GB. The TableFile() instruction controls the size of its output files through the NumRecsTimeIntoInterval, and Interval parameters.
  • Faster compile times when small file sizes are specified
  • Easy retrieval of closed files via File Control utility, FTP, or email
  • Closed files are safe files. Anytime a file is open for writing, it can become corrupted if a power loss occurs or if the writing is interrupted for any reason. TableFile() Option 64 will close files at the programmed time or record interval. Once those files are closed, they are safe from this type of corruption.

CardOut() Program Example


Public batt_volt
DataTable (Test,1,1000)
	DataInterval (0,15,Sec,10)
	CardOut (0,-1)
	Sample (1,batt_volt,FP2)
EndTable
BeginProg
	Scan (1,Sec,0,0)
		Battery (batt_volt)
		CallTable Test
	NextScan
EndProg

TableFile() Program Example


Public batt_volt, OutStat As Boolean, LastFileName As String * 32
DataTable (Test,1,1000)
	DataInterval (0,15,Sec,10)
	TableFile ("CRD:Test_",64,10,0,1,Min,OutStat,LastFileName)
	Sample (1,batt_volt,FP2)
EndTable
BeginProg
	Scan (1,Sec,0,0)
		Battery (batt_volt)
		CallTable Test
	NextScan
EndProg

Recommended for You: To learn more about the TableFile() instruction with Option 64, read the “A Better Way to Write High-Frequency Data to 16 GB and Smaller CF Cards” technical paper.

Data Collection and Access

CardOut() files are open for writing as long as the CardOut() instruction is running and are, thus, always available for remote data collection. With TableFile() Option 64, the most recent file is open and available for remote collection. After a file is closed, data can be accessed only by first retrieving the file (for example, via the File Control Utility, FTP, email or, copying files from the card).

For both CardOut() and TableFile() Option 64 instructions, open files on the card become an extension of the table memory. Any time data is retrieved via the Collect or Custom Collect utilities of LoggerNet, PC400, or RTDAQ, the internal SRAM is searched first, then the card. If a file is found on the card with the same name and file header as the data table in the internal memory, the data from the card is appended to that file.

Note: To be considered identical, the file must have been created on the same data logger using the same program.

You can view your memory card data using data logger support software, such as the View Pro client application in the LoggerNet software package. You can also access your memory card data through the data logger program using data table access syntax such as TableName.FieldName. (Refer to the CRBasic Editor Help file.)

Recommended for You: For more information, read the "How to Programmatically Access Stored Data Values" blog article.

CardConvert Software

Note that for both CardOut() and TableFile() Option 64, data is stored on the card in TOB3 format. TOB3 is a binary format that incorporates features to improve reliability of the CF cards. TOB3 format is different from the data file formats created when data are collected via a communications link, which is TOA5 or ASCII format. Hence, data files that are read directly from the CF card need to be converted into another format to be human readable. You can convert files from TOB3 to TOA5 or other formats using the CardConvert software that is included in LoggerNet, PC400, and PC200W.

Conclusion

I hope this article has helped you to understand how easy it is to add a CardOut() or TableFile() instruction to a CRBasic program that will direct your data logger to store data to a memory card. (For details on each of these instructions, see the CRBasic Editor Help files.) In a future article, I’ll address some more complex card issues and offer handy card tips and tricks.

If you have any questions or comments about memory cards, post them below.


Share This Article



About the Author

carolyn ivans Dr. Carolyn Ivans is the Technical Communications Manager at Campbell Scientific, Inc. She has a doctorate in Ecology, Evolution, and Conservation Biology from the University of Nevada, Reno. She enjoys working with the Technical Communications team, product developers, and customers to create documentation and tutorials for Campbell Scientific products. In her spare time, Carolyn enjoys kayaking, hiking, and bike rides with her family.

View all articles by this author.


Comments

ReadyDAQ | 07/29/2016 at 10:59 PM

Great tips.Learned a lot from this blog about data logger software.Thanks for sharing.

flipper66 | 04/11/2017 at 02:08 PM

Is there any way to programatically determine the amount of free space on a CARD ?

Carolyn | 04/11/2017 at 03:09 PM

Yes there is, using Tablename.Fieldname syntax to read CardBytesFree from the Status Table, as shown below:

Public Card_Bytes_Free as Long 'declare a variable to store the card bytes free

BeginProg
 Scan (1,Sec,0,0)

 Card_Bytes_Free = Status.CardBytesFree 'read the card bytes free from the Status Table

Nikki | 04/17/2018 at 07:34 AM

Thank you for the explanantion!
Is it true that u can store a maximum of 30 data tables on a microSD card? (CR6-datalogger) 

Carolyn | 04/20/2018 at 05:02 PM

You can store many more than thirty files on a micro SD card. The number of files is limited by the amount of memory available on the card, not the number of tables.

Pascal | 01/17/2023 at 12:03 PM

Hi

I write two data tables "System" und "Sensors" with CardOut() to a 2GB SD Card.

When i check the "Table Fill Times" I see 4 Tables System + CRD:System and Sensors + CRD:Sensors. The Sensors for example will last 19 Days and the CRD:Sensors will last about 10000 days. Do I get it right that the Sensor Table Data will be transfered (after 19 days ?) to the CRD:Sensors Table on the card wich is actualy the extension of the Sensor Table on the logger memory?

At least this is what I get out of this Article. Is that correct?

Thank You.

Carolyn | 01/17/2023 at 05:32 PM

Thanks for your question. When CardOut() is used to stored data to a card, data are copied directly from the CPU to the card in ~ 1Kb blocks. The amount of time it takes to generate 1 Kb of data depends on the number of values being written to the data table each time a record is written, the data types of those values (low resolution, or high resolution) and how frequently data are written to the table. Table Fill Times indicates the point in time where data on the card will begin to ring. In otherwords, if Table Fill Times indicates a table will fill in 19 days, that means that in ~ 19 days, the oldest records stored on the card will start being overwritten by new records. For the best technical support, please  visit https://www.campbellsci.com/support for additional information.

Please log in or register to comment.