Appendix D: Data checks and repair -- The Data Repair Wizard

Previous: Appendix C: Data storage in Topicscape

(Go to:Topicscape Pro User Guide - contents list)

Computers can suffer from computer crashes, power failure and accidental deletions. Although the database used in Topicscape (Firebird) is very robust we provide a clean up wizard to help overcome file problems that may come up from time to time.

If things go wrong (and even as an occasional check)

It is a good idea to run the Data Repair Wizard occasionally. You do this in one of three ways:

• While Topicscape is running: From the Tools menu in the main menu bar: It will offer to check the Topicscape that was open at the time, but you can choose another.
• When Topicscape is not running: From the main Start menu: Topicscape | Data check and Repair;
• When Topicscape is starting up: If a possible error is detected you will be given an option to run the Data Repair Wizard

Will I lose control?

The first thing to know is that the Data Repair Wizard won't make any changes without your permission. It has a check phase and a repair phase and you will be given a report after the check and a choice to go on with each type of repair separately or skip it.

The second: It makes a copy of the database it's checking, and tells you exactly what that copy is called and where to find it. If you run the wizard five times, you'll get five backups -- it doesn't overwrite the previous backup. You can rename a backup and you will have the original Topicscape file again.

You'll see progress reporting as well, letting you know the checking stages as they pass.

Topicscape can't protect from, for example, someone deleting files that a Topicscape knows about. And this can happen, particularly if the files are outside the My Topicscapes folder. The Data Repair Wizard can check for such files, ones that are no longer where their occurrence records them as being. It reports these and offers you the chance of fixing them one by one, making a global change or deleting the occurrences. More about this in the following sections, Reporting missing files; Fixing missing topic links occurrences; Fixing doubtful text.

Reporting missing files

The occurrence files you introduce to a Topicscape are generally stored in a folder associated with that database. The standard place for these for a Topicscape called, for example, "Reference Material" would be "C:\Documents and Settings\<username>\My Topicscapes\Reference Material". Its database would be "Reference Material.3DT". When installing Topicscape you can give another location than C:\Documents and Settings\<username>\My Topicscapes.

You can move complete Topicscape folders around with no problem, provided that they are on your computer. Topicscape Personal Editions (Pro, Lite and SE) do not support network connections (see Network access to files). Unless you delete files in the Topicscape folder yourself, you won't have any trouble with those files.

If you use the "Link" option, Topicscape occurrences can be linked to files anywhere else in your computer. Then it is more likely that the link can be broken. This is why we recommended choosing the "Move" option when introducing files to Topicscape, so the file will be moved into that Topicscape's own directory and is less likely to be moved.

In addition to occurrence files, when you describe a topic / occurrence with a Description+ in the detail panel, another file (an RTF file) is generated. The check for missing files will report any Description+ files that are missing as well.

If occurrences exist that were originally associated with a file but the file is no longer there, then the Data Repair Wizard will give the message "Some occurrences had no corresponding files". This means that you (or some other software perhaps) have moved or deleted the files. The Wizard will show you which files they are, and will give several options for fixing this.

Fixing missing topic links occurrences

The links between different Topicscape files are inter-Topicscape links. These files can be present in one Topicscape as an occurrence which can link to a topic when you try to open the occurrence inside Topicscape. You can also link to a Topicscape without specifying a target topic.

During data repair, these inter-Topicscape links are checked for the presence of the linked Topicscape and the topic that it relates to. When one or more link destination is found to be missing, a report displays the results and you can select to output a spreadsheet (CSV) report or to delete the occurrences concerned.

Fixing doubtful text

Text in your database (for example, topic names, occurrences or description) will often contain additions made by Topicscape itself (examples are fileless-; `251`; and .fes). You will not normally see this type of text. If you do, something has gone slightly wrong. Data repair will detect such "doubtful text" when it appears in the wrong place. Topicscape can clean this up for you (recommended), or you can such items yourself, one by one.

Very large databases

3DT files can grow very large because old "versioning" information is retained until cleared up. If you find that a 3DT file has grown very large, running it through the Data Repair Wizard can do the necessary clearing up and will make it much smaller. A suddenly-smaller database after Data Repair is not an indication that something has gone wrong.

Check results log file

A log file of what happened during the check appears here:

C:\Documents and Settings\<username>\My Topicscapes\.log\scan_ Topicscape_debug_<current_date_time>.log

Checks performed (if you're really interested)

1. Check for invalid associations (checks that associations link topics that exist)
2. Check that Topicscape Orphans and Topicscape Islands exist (these should always be present)
3. Check generator value (an internal technical check)
4. Check for incorrect flags in association (an internal technical check)
5. Check for incorrect flags in topics and occurrences (an internal technical check)
6. Check for null in topic occurrence table (an internal technical check)
7. Check for empty OrigFileNames in topic table (an internal technical check)
8. Check Stored Procedures and Triggers (an internal technical check)
9. Check topic table Delete status (an internal technical check)
10. Check for stray files
11. Refresh the keyword indexes
12. Check database by gfix
13. Check if OAT-OIT > 100 (an internal technical check that also removes old data and makes your 3DT file smaller by removing old versioning information.)
14. Check that Topicscape Box folder exists (Topicscape expects to find this folder for various functions, so if it is missing, the Data Repair Wizard will repair that)
15. Check for status of occurrences in Topicscape Box
16. Check for History List update
17. Check for Occurrence File Extension correctness
18. Check for Isolated Topics
19. Check for unused description+ files
20. Check for files described in the occurrences table that are actually not at the described location (file occurrences should have a file)
21. Check for topics / occurrences with missing description+ file
22. Check for topics / occurrences with doubtful text
23. Check for invalid topic links
24. Check for invalid user defined topic attribute table

"Topicscape Box occurrences are not synchronized"

Items in Topicscape Box have information associated with the files and shortcuts you put there - like Description and Source. This extra information is stored in the Windows Registry. The Topicscape Box files are stored in a folder which (if you accepted the suggested settings on the first run of Topicscape after installation) will be, for example, C:\Users\Harry\My Topicscapes\.Topicscape Box If you move files from there using Windows Explorer instead of Topicscape Pro or Topicscape Box, the Registry will become out of sync and the Repair Wizard will fix it (if you accept it's suggestion to do so).

Separate checks from within Topicscape

There are some checks that can be run without using the full Data Repair Wizard. They are included in the wizard process, so they only need to be run when you want to do a specific and limited check.

List all missing occurrence files

Most occurrences are made with files, and if a file is accidentally deleted or lost in some other way, the occurrence will show up in black in the Topic Center, with a tag 'Missing". This is most likely to happen when you make a 'Link' for an occurrence -- in other words, the file is left in a directory outside Topicscape's control.

To run a check for missing files, select 'List all missing occurrence files' from the 'Tools' menu. This will report the number found and give you the option of finding them, one by one and repairing the link, making a mass change, deleting them, or leaving them to sort out later. The mass change can be useful if, for example, you have many links that all point to files in a specific directory that has been renamed or moved.

Check for stray files

If you want to, you can check to see if there are any files in a specific Topicscape's folder that don't belong to that Topicscape: In other words, files that are not identified in Topicscape as part of an occurrence. When you delete occurrences, for safety Topicscape does not delete the file, it just deletes the record of it in Topicscape. During that run, Topicscape can then perform an Undo, and after that run you could navigate to the folder and recover the file if you wanted to.

To clear these out, use the "Check for stray files" option in the Tools menu. It only checks the files for the Topicscape that is open at that time and it only looks in that Topicscape's directory.

To give an example: If you have a Topicscape called "Hobby Notes", it and all its moved or copied files will be in a folder called C:\Documents and Settings\Yourname\My Topicscapes\Hobby Notes (unless you chose a different option at install time or moved the folder).

This process simply checks that all the files found in the Hobby Notes folder are defined as occurrences in your database 'Hobby Notes.3DT' and reports any that are not, in an action panel. Using the panel, you can choose to leave them where they are, delete them or move them to another directory for further action.

The stray files check will be done whenever you run the Data Repair Wizard, but you can also run it by itself, using the entry in the Tools menu of Topicscape.

When you see the report, press the "Check all" button, look through the list presented and uncheck any you want to keep and press "Delete checked" to permanently delete the rest.

---

Next: Appendix E: Security of Topicscape documents and data