Database Integrity Checks

Though modern computer hardware is very good, it is still not perfect. In rare cirumstances, both SSD and magnetic hard drives can degrade and cause data loss. When data is stored in memory (in DRAM) it can be altered by cosmic rays or other high energy particles (some computers use ECC memory to combat this, but this option is not available in most Macintosh computers).

Panorama cannot prevent file damage caused by hardware failure, but it can detect damage and prevent you from continuing to work with a damaged database. Panorama has two methods for detecting damage.

- The data in a Panorama database is stored with a specific structure for records and cells within records. Each record must contain the correct number of cells, and the sum of the cell lengths within a record must match the length of the record itself. If this doesn’t match up, the data structure is corrupted. Panorama may limp along for a while in this state, but this sort of corruption will cause errors and possibly crashes. Fortunately Panorama can check and verify that this structure is correct. Important Note: Checking the data structure does not guarantee the validity of the data itself, or of procedures and forms.

- When saving a database, Panorama can include an “integrity seal” that validates the contents of the database, including the data itself, forms, procedures and variables. If a database contains an integrity seal, Panorama will check the contents of the database against the seal when it opens the database. If the contents have been tampered with, Panorama will not open the database.

### Integrity Seal

By default, Panorama always adds an integrity seal when saving a database. If you don’t want to include an integrity seal in a particular database, use the File>Database Options>General panel to disable this option.

Generally you would never want to disable the integrity seal option. However, the process of generating an integrity seal does take a slight amount of time, so if you need the fastest performance and aren’t concerned about database damage you can disable this on a database-by-database basis.

Important: You should disable the integrity seal option if you plan to use this database with an older version of Panorama X. The integrity seal option is only supported by Panorama X 10.2 or later. Older versions of Panorama X will open a database that has an integrity seal without any problem, but if the database contents are modified when using an older version of Panorama, the integrity seal will no longer be valid and the database won’t open in Panorama X 10.2 or later. If you encounter this situation you can temporarily disable integrity checking so that the database can be opened (see below).

### Integrity Checking when Opening a Database

If a file on disk has been damaged while stored on the disk, opening it may cause Panorama to crash – either immediately or when performing data operations. To prevent this, every time Panorama opens a database it checks the integrity seal (if present) and also checks the integrity of the data structure. If damage is detected, Panorama will not allow you to open the file and will display information about the problem, like this:

If you encounter a damaged file, usually the best bet is to locate a recent backup version of the file, and use that. You could also try to disable integrity checking and attempt to open and export the file (see below).

### Integrity Checking when Saving a Database

If the data structure becomes damaged while the database is open in RAM (for example from a cosmic ray), you don’t want that damage to be saved permanently. To prevent this, Panorama checks the data structure integrity before it saves the database. If the structure is damaged, Panorama will refuse to save the database.

If you encounter this you may want to try to export the data as text, then re-import it into a different database (since the data structure is corrupted, the export may not work). You could also try to delete the corrupted data, but it’s quite likely that Panorama may crash when you try that. The good news is that your file on disk is un-touched, so the worst case is losing the recent changes made since the file was opened.

### Disabling Integrity Checking

You may want to temporarily disable integrity checking to attempt to recover a damaged file. Use the Advanced preferences panel to disable this option.

Once checking is disabled, you can attempt to open a damaged file. If Panorama does manage to open the file, we would recommend not attempting to use the file, but to immediately attempt to export the data as text. If that succeeds, you can import the text into a different database. It’s probably a good idea to carefully check the exported data for damage – if the database structure is damaged, there is a high probability that some of the data itself is also damaged.

Important: Make sure to re-enable integrity checking as soon as possible!

### Manually Checking Data Structure Integrity

In addition to the automatic checks when opening and saving, you can also use the datacorruption option of the dbinfo( function to check the integrity of a database’s data structure. This function will return text describing any corruption it finds, or empty text if there is no corruption. This example checks the current database for data structure corruption.

    let corruption = dbinfo("datacorruption","")
    if corruption<>""
        message corruption
        return
    endif
    ... continue with uncorrupted data

Data corruption in memory is exceedingly rare, so it’s probably not necessary to explicitly check for corruption like this – you can simply rely on the automatic check when saving the database.

See Also

- Auxiliary Databases -- automatically opening multiple databases associated with each other.
- closefile -- closes the current database.
- closeserverfile -- closes the current database (used on the server).
- Custom Database Initialization -- using an .Initialize procedure to automatically run custom code when a database opens.
- Database is Damaged Alert -- how to fix the "database is damaged" alert when opening a database.
- opendatabase -- opens a database file.
- openfile -- opens a database file. It can also import data from a text file, another database, or a variable into the current database.
- opensavedwindows -- opens windows that were open the last time file was saved.
- opensecret -- opens a database invisibly, without opening its windows.
- openserverfile -- opens a database (used on the server).
- opentextfile -- imports data from a text file or a variable into the current database.
- panoramastartupdatabases -- opens any databases specified for startup.
- print -- prints the database.
- printonerecord -- prints the the currently active record.
- revert -- reloads the last saved version of the database from the disk.
- save -- saves the database to disk.
- saveacopyas -- saves a copy of the currently active file under a new name. The original database remains open in memory under its original name.
- saveas -- saves a copy of the currently active file under a new name. The original file remains unchanged.

History

Version | Status | Notes
10.2 | New | New in this version.