Bit by Bit Backup

The following is AI-generated (aka vibe-coded) software. The software is specific to my needs and my system and may not work for you or your system. Support is limited, but if you report a bug I'll certainly look into it. All code is provided in a zip file for you to review should you choose.

Bit by Bit Backup records a checksum (SHA-256) of every file you choose, then reads the files again later and reports any whose contents changed while their size and modified time did not. That is the signature of bit rot: a program that edits a file updates its modified time, but a failing disk, cable or memory doesn’t. From two computers’ records it compares their copies, repairs a damaged copy from the good one, and backs one computer’s folders up to the other, never copying a damaged file.

It began as a copy of Bit Rot Detector, now retired, which it extends with Backup mode. Bit Rot Detector’s records are the same format, so this app opens them; its project files aren’t read.

It is a Python program, started with bit-by-bit-backup.py (the code is in the bit_by_bit_backup folder beside it), with a window for choosing files and looking at results, and a batch mode for Windows Task Scheduler. It needs nothing beyond Python and its standard library.

Contents

• Requirements
• Using it
• Projects
• The database and the working folder
• The ignore list
• Comparing two databases
• Repairing from a backup copy
• Backing up
• Batch mode and Task Scheduler
• The activity log
• The working folder
• Command line
• Keyboard shortcuts
• How it is fast
• Testing
• Troubleshooting
• Limitations

Requirements

• Python 3.8 or later, from python.org (its installer includes Tkinter). On Debian or Ubuntu, sudo apt install python3-tk for the window; batch mode works without it.
• Optional: Pillow, only to show the window icon outside Windows.

Start it with python bit-by-bit-backup.py (or pythonw bit-by-bit-backup.py on Windows, for no console window).

The icon, bit-by-bit-backup.ico beside the script, is a blue tile of nine bits with the middle one red: one bit gone bad among good ones. The window shows it in its title bar and taskbar button; it holds each size from 16 to 256 px, each drawn on whole pixels, so it stays sharp at every display scaling.
python tools/make_icon.py (needs Pillow) draws it again.
For a PyInstaller build, build from bit-by-bit-backup.py (PyInstaller follows its imports into the bit_by_bit_backup folder), give the icon as the executable’s (--icon bit-by-bit-backup.ico) and keep the .ico beside the .exe too: the window loads it from beside the executable when frozen.

Using it

Scan and Compare scans, at the left of the toolbar, switch the window between its two modes. In Scan mode, described here, the window has the tree on the left and the progress and results on the right. Compare scans mode compares two databases across the whole window (see Comparing two databases). The project row serves both.

One window runs at a time on a computer. Starting the app again while it is open brings the open window to the front instead (restored if it was minimized), and a project named on that command line isn’t opened: load it in the open window. Scheduled batch runs aren’t held back by an open window (see Batch mode and Task Scheduler).

1. Choose what to check. Tick drives, folders or files in the tree. A click on a check box (or Space on the selected row) ticks it with everything inside it, or unticks it. Untick a folder inside a ticked one to leave it out; the folder above then shows a partly ticked box. Folders open as you expand them, so a drive with a million files opens at once; one slow to list (a sleeping disk, a network share) shows “Loading…” until it is, and the status bar says “Listing FOLDER…”, then “Listed FOLDER.”; the window stays usable meanwhile. A folder that opens on nothing says why in a row of its own: “Empty”, or, under a filter, how many items the filter hides and that Show all shows them. Drive labels fill in the same way.
   The choice is saved as rules (“include D:\Photos, leave out D:\Photos\cache”), so files added to a ticked folder later are covered without ticking them. Deselect all unticks everything, after asking. Filters ▾ chooses what the tree shows: Show all; Show ticked only, just the ticked folders and files with the folders leading to them; Show ignored within ticked, just what the ignore list (below) leaves out inside the ticked folders, marked “(ignored)”, with the folders leading to it; Show unticked within ticked, just the folders and files unticked inside a ticked folder (the one exception in a long list of ticked folders), marked “(unticked)”, with the folders leading to them and anything ticked again inside them; or Show ticked and unticked within ticked, both together: everything Show ticked only shows, and the exceptions inside it, marked “(unticked)”. One whose folder or file is gone is marked “(not found)”; ticking an exception again takes it out of Show unticked within ticked, and its mark away in the combined view.
   The button names the filter in use (“Filters: ticked ▾”), and going to a path shows the whole tree again. In every view, what the ignore list leaves out is gray and struck through. Right-click a row (or press Shift+F10 or the menu key) for Copy full path, which puts the row’s path on the clipboard, and Add to ignore list, which, after asking, adds that place (only it, not every folder of that name) to the ignore list. To jump to a folder, paste or type its path in the field above the tree (quotes from Explorer’s Copy as path are fine) and press Enter or Go to: the tree opens down to it and selects it. A path on a drive or share the tree doesn’t list, such as \NAS\Photos, gets a row of its own at the top.

2. Catalog (F5) lists the chosen folders and records each new file’s checksum. A file whose size and modified time are unchanged is not read at all, so a second Catalog of an unchanged collection takes moments (0.17 seconds for 20,016 files in a test here).
   Catalog also notes:

   • Modified files (a new modified time or size), which it reads again: an intended change, not bit rot.
   • Missing files: gone from their folder.
   • Moved files: a new file with the size and modified time of a missing one is taken as that file moved or renamed, with the same name preferred. It keeps its checksum and history and isn’t read again, so renaming a folder of photos costs nothing.
   • Added files, in folders that were recorded before (the first Catalog of a folder doesn’t list every file).
   • Skipped files and folders: symbolic links and junctions (which can loop), cloud-only OneDrive files and offline files (reading them would download them), and folders that can’t be listed (access denied). They are listed, not read. Windows’ own folders (System Volume Information, $Recycle.Bin, Recovery: folders marked both hidden and system) are left out of the tree and the run altogether, so a chosen drive doesn’t report them each time. So are junctions in the tree (C:and Settings, which points to C:). A folder with another volume mounted on it (a disk mounted in a folder, a Cryptomator vault mounted in a folder) is not a link to another folder, so the tree shows it and Catalog reads what is inside, as it does a drive.

3. Verify (F6) reads every chosen recorded file again, the ones checked longest ago first, and compares them with their checksums. Stop ends it early; the next Verify starts with the files not checked yet, since they are now the ones checked longest ago.
   Bypass the cache reads each file from the disk even if the system still holds it in memory from a recent read (see Speed). A file found edited since it was recorded is reported as modified, not as bit rot.

   While Catalog hashes or Verify reads, the progress panel shows the files and bytes done, the reading speed over the last few seconds, and the time left. For the first minute it says “working out time left”.
   After that the estimate is learned from the whole run so far: how long a file takes and how long a MB takes, worked out from how the time went on stretches of small and of large files, then applied to the files and bytes still to read. So a folder of thumbnails or a slow spell (another program using the disk, run at Background) doesn’t send it soaring, and a run half done after two hours says about two hours left. In the tests’ simulated run (small and large files in turn, with slow spells), it stayed within 15% of the run’s length of the true time left after the first two minutes. After it comes when the run should be done, as “done about 2026-10-28 03:20 pm”.

4. Results lists what the last Catalog or Verify found; each run replaces it. What earlier runs found and you haven’t accepted stays in the record: the note under the table counts it, and Show earlier results lists every open finding.
   Problems are possible bit rot and unreadable files (a read error from the disk), in red with ⚠; Information is the rest. Untick Show skipped to hide skipped files and folders from the table and its counts (the note under the table says how many are hidden). After each Catalog and Verify, and when the filter changes, the columns fit their contents (a long path shown whole when there is room) and the File column takes any room left over; a column you drag keeps its width until then. Click a heading to sort by it, again to reverse. Ctrl+C copies the selected files’ paths. The table shows the newest 10,000 results; Export CSV saves them all.

5. Right-click a result to open its folder, or show the file in Explorer (a missing file opens the folder it was in).
   Accept the selected results once dealt with (Select ▾ beside it selects All the rows shown, None, or Inverts the selection). For possible bit rot or an unreadable file, restore it from a backup (or decide it is fine), then accept: the file is read again and the checksum of what is there now replaces the recorded one. Accepting a missing file forgets it. Other results just leave the list; a file or folder skipped again for the reason you accepted isn’t listed again.

6. Export CSV saves the results the filter shows to the working folder (see below), for Excel, and the status bar names the file. Working folder ▾, Open in Explorer, opens it.

Stop (Esc) ends a run at the next read; what it did is kept. A Catalog that is stopped hashes the rest next time, and reports nothing missing, since a listing cut short can’t tell. Closing the window stops a run too.

A drive that isn’t connected is shown in the tree as “(not connected)” when something on it is chosen. Its files are never reported missing: Catalog and Verify say it wasn’t found and leave its record alone. A chosen folder deleted from a drive that is there does make its files missing.

Projects

A project keeps a set of folders to check with its own settings and its own database, so a photo collection and a backup drive, say, can each be checked on their own. The Project row under the toolbar holds the project’s short Name and Description. Its settings (Run priority, Bypass the cache, and Keep damaged copies) are in the toolbar above. The Projects ▾ button, at the right of the toolbar between Working folder and Database, opens a menu with Load project…, Save project as…, Save project, Close project, Manage projects… and Set projects folder…. Close project leaves the open project (offering to save its changes) and uses this computer’s database again, as the window starts: its ticks (showing only them), no results, no name or description.

• Save project as… (Ctrl+Shift+S) asks for the folder to save the project in (any folder, with Browse…, whatever --working-folder-path says) and the name its files start with, offering the short name to confirm or change. It saves the name, description, chosen folders, settings, and the databases and folder pairs of Compare scans and Backup, in that folder as NAME.bit-by-bit-project.json, with a copy of the database in use as NAME.sqlite holding only what is ticked: the files and findings of folders ticked earlier and not now are left out (a million-file record halved in 5 seconds in a test), and the database copied from is left as it was. The copy is made in the background, the progress panel saying so (about 10 seconds for a million-file record here); until it is done, the tree’s choices, Catalog, Verify, loading a project and closing the window wait for it. The window then uses the project: its database, and beside it its reports (NAME-report-...csv), comparisons, batch log (NAME-log.txt) and repair log (NAME-repair-log.txt).
• Save project (Ctrl+S) saves the changes to the open project; with none open, it is Save project as.
• Load project… (Ctrl+O) opens a project file from any folder: its folders, settings and database. A file that can’t be read, or isn’t a project, is reported and left as it is.
• Manage projects… opens a window listing the projects in a folder (the last project’s, at first; type another and press Enter, or Browse…), with each one’s file name, short name, description and number of files. A project file that can’t be read is listed in red, and can still be renamed or deleted.
  • Rename… asks for the new short name and the name its files start with (which follows the short name until you change it), and renames every file of the project: the project file, the database (with the copies Clear kept and the pending updates backups wrote to it), the logs and the reports. It won’t use a name whose files would replace others. Renaming the open project opens it again under its new name.
  • Delete… lists the project’s files and, once you say Yes, sends them all to the Recycle Bin (the Trash on Linux and macOS), the database first, so they can be restored from there. A network share or a USB drive has no Recycle Bin: the question says so, and there the files are deleted for good. The open project can’t be deleted: load another one first.
  • Neither runs while a run (in this window, another or a scheduled one) is checking the project’s database, or while a comparison is going, and no run can start on the project’s files until they have all been renamed or moved. A file another program has open stops a rename with nothing renamed, or a delete before the database goes.
  • Only the project’s own files are touched: a project named Photos never takes Photos-2024.sqlite or Photos notes.txt.
• The title bar shows the open project’s name, with a * while it has changes not saved. Closing the window or loading another project asks whether to save them.
• Set projects folder… chooses the folder that holds your projects. Load project, Save project as and Manage projects then start there, this time and the next (it is kept in the settings, [projects]).
• Load project, Save project as and Manage projects start in the folder given with --projects-folder-path, else the one Set projects folder chose, else the folder of the project opened or saved last (the working folder before the first).
• The window opens a project only when --project-file-path names one. Otherwise it opens on this computer’s own record, with the folders ticked that the record has ticked (the ones batch mode checks) and Filters’ Show ticked and unticked within ticked on when there are any, so the tree shows just them and the folders and files unticked inside them; no name or description, no results, and nothing chosen in Compare scans. The progress panel says “Nothing chosen yet” while nothing is ticked, then how many of the ticked files are tracked (have a checksum recorded), their size, and the record’s last Catalog and Verify, whenever they ran (an earlier session’s, or a scheduled batch run’s; a project’s panel shows the project’s). They are counted in the background: with a million files recorded, “Counting the chosen files…” shows for about half a second after each tick. Earlier runs’ findings (a scheduled batch run’s, say) stay in the record: the note under the table counts them, and Show earlier results lists them. Load project… opens a project the same way (with Show ticked and unticked within ticked on when it ticks anything), its findings behind Show earlier results until a run. The folders you tick are saved in the record as you tick them, and batch mode without a project checks those.
• Batch mode runs a project with --project-file-path NAME.bit-by-bit-project.json: its folders, its settings (an option given on the command line overrides one) and its database.

The project file is JSON; a setting in it that isn’t valid (a hand edit) is the default instead.

The database and the working folder

Without a project the window uses this computer’s own database, MACHINE-bit-by-bit-backup.sqlite in the working folder (named after the computer), and so does batch mode without a project; with one, the project’s own database. A database keeps every file recorded in it until it is told otherwise: unticking a folder leaves its files there, and a deleted file stays as missing until accepted. The toolbar ends with Activity log (see The activity log) and three menus: Working folder ▾, Projects ▾ and Database ▾.

Database ▾ names the database in use and shows its size (Database: MACHINE (12.3 MB) ▾, or the project’s name, Database: Photos (3.1 MB) ▾; 1 MB is 1,048,576 bytes). Its menu acts on that database:

• Forget unticked files… removes the files outside the folders ticked in the window, and their findings. Nothing ticked would forget everything, so it says to use Clear instead.
• Forget missing files… removes every file recorded as missing, and the findings that it went missing, as accepting each one in the results would.
• Clear… removes every file, finding and run, keeping the ticked folders. It first keeps a copy beside it, as MACHINE-bit-by-bit-backup.cleared-<date and time>.sqlite (or Photos.cleared-...sqlite): delete it once you no longer need it.

Working folder ▾ has Open in Explorer and Change working folder…, which moves the window to another working folder until it closes (see The working folder). It asks first, saying that the computer’s files in the current folder are not deleted or moved, and whether the new folder already has this computer’s database or a new, empty one (with a settings file, if it has none) will be made there. The settings so far are saved in the old folder. An open project stays open; without one, the window uses the new folder’s database, with its ticks.

Each asks first, saying how many files it touches; those that remove files give the room back to the disk, so the size on the button drops.
None of them runs during a Catalog, Verify, comparison or backup, or while a scheduled batch run uses the database.

The ignore list

Some files change all the time and mean nothing when they do: browser and app caches, compiled Python, Office’s lock files, partial downloads, temporary files. Checking them only fills the results with changes no one needs to see. Ignore list…, beside Filters above the tree, opens the list of files and folders that are never recorded, checked, compared or counted, by any run, in any project or batch run on this computer.

• A name (__pycache__, *.tmp, Thumbs.db) leaves out every file or folder of that name, and everything inside such a folder.
• A pattern with a separator leaves out the folders (and files) whose path ends that way: photos\raw takes any ...\photos\raw, and *\AppData\Local\Temp any user’s temporary folder.
• One that starts at a drive or a share (D:\Scratch, \NAS\Temp) takes that place only.
• * and ? are wildcards (* crosses folders), and case doesn’t matter.
• Patterns are shown and kept with backslashes, Windows’ standard; a pattern typed with / works too and is changed to \. (On Linux and macOS they are shown with /.)

The list opens sorted by each pattern’s first letter or digit, in any case (*.tmp under t, *\AppData\Local\Temp under a); patterns you add go at the end until it opens again. Type a pattern and press Enter or Add, or paste several lines (a list of patterns, one a line) into the field to add them all at once, those already on the list left out; select patterns and press Delete or Remove. The Origin column says whether each pattern is one of the defaults or one added by you. Restore defaults (its label says how many defaults are missing from the list, “Restore defaults (2 missing)”) sets the list back to the defaults alone, taking your own off; Merge defaults adds the missing defaults and keeps your own, taking off any pattern listed twice (in any case or slash). Save keeps the list and at once forgets, from the database in use, the files already recorded that it now leaves out, with their results, so the counts drop; other databases (projects not open, the computer’s when a project is open) forget theirs at their next run, which says how many. They are forgotten, not reported missing. Cancel keeps nothing. Compare scans and Backup leave an ignored place out on both sides, so a backup neither copies it nor deletes the Destination’s copy.

To see what the list leaves out of your folders, choose Filters ▾ > Show ignored within ticked above the tree: it looks through the ticked folders as Catalog would (a large drive takes a while; “Looking for ignored files and folders…” shows meanwhile) and shows each ignored file or folder, with the folders leading to it, the first 5,000 at most. Saving the list shows the new list’s. In the whole tree, ignored files and folders are gray and struck through, and a row’s right-click menu has Add to ignore list, which adds that one place (D:\Photos\cache) and forgets its recorded files, as Save does.

The list starts with __pycache__, *.pyc, .pytest_cache, .mypy_cache, *\AppData\*cache* (whatever an app keeps in AppData with “cache” in its name: Chrome’s and Edge’s Cache, Code Cache and GPUCache, Firefox’s cache2), *\AppData\Local\Temp, *\Library\Caches, ~$*, *.tmp, *.crdownload, *.part, *.partial, *.repair-tmp and *.rotten (a repair’s temporary copy and the damaged copies it keeps), Thumbs.db, .DS_Store, and a database’s temporary files (*.sqlite-wal, *.sqlite-shm, *.sqlite-journal, *.db-wal, *.db-shm, *.db-journal), which come and go whenever it is opened; then Windows’ own: $RECYCLE.BIN, System Volume Information, pagefile.sys, hiberfil.sys, swapfile.sys, C:\Windows\Temp, C:\Windows\Prefetch, C:\Windows\SoftwareDistribution\Download, *\AppData\Local\Microsoft\Windows\Explorer (thumbnail caches), *\AppData\Local\CrashDumps, ehthumbs.db and IconCache.db; then PortableApps’: PortableApps.comLauncherRuntimeData-*.ini, *\Data\Temp, *\Data\profile\*cache* (a portable browser’s cache2, startupCache, Cache, Code Cache, GPUCache and ShaderCache), *\Data\profile\thumbnails, *\Data\profile\*.lock and parent.lock.
(INetCache and the graphics drivers’ shader caches are already taken by *\AppData\*cache*.) A list saved before these were added keeps what it had: Merge defaults adds them. A bare Cache isn’t on it, since it would also take folders of your own that happen to be called that; add it if you want it. The list is kept in the working folder as MACHINE-bit-by-bit-backup-ignore.json; a file there that isn’t the app’s list is left alone, and runs stop with a message until it is fixed or removed.

Comparing two databases

Compare scans mode, at the left of the toolbar, compares the checksums two records hold: this computer’s and the one kept for a backup drive or another computer, say, or two folders in this computer’s own record. A file whose two copies have the same size and modified time but different checksums means one of them has bit rot; Verify on each side says which.

It takes the whole window below the project row: the tree, the progress panel and Scan’s toolbar (Catalog, Verify, Bypass the cache) give way, so the differences have the full width. F5, F6 and Esc do nothing here. The databases and pairs are at the top, in the order you fill them in, with + Add pair and Compare below the last pair, then the table of Differences with the same filter as the results (Problems: damaged copies and differing contents; Information: the rest; All), the Repair buttons (see Repairing from a backup copy) and Export CSV.

1. First database and Second are the two databases. The lists offer every computer’s database in the working folder (this computer’s first) and the two chosen in Backup; Browse… picks one anywhere else, and From Backup takes Backup’s Source and Destination as the first and second, and Backup’s folder pairs with them (with their Reached at), replacing the pairs here. ↻ reads a database again after Scan has changed it, so the folders it has recorded since are offered in the pairs. Compare always reads both databases as they are, so the comparison needs no reload: click Compare again. The same database may be both. The first is always the database last scanned or opened: a project’s own when it opens, this computer’s after a Catalog or Verify without a project; you can still pick another for a one-off comparison. Without a project or a scan both start empty; in a project the second starts as the one it was saved with, or another database beside the project’s. A project keeps its second database and its pairs, with their Reached at folders.

2. Each pair of folders matches a folder in the first with the folder holding the same files in the second, whatever the folders above them are called: D:\Photos with \NAS\Backup\Photos copy, for instance.
   Choose… beside each folder opens a picker with every folder its database holds files in, ticked or not, as a tree that starts closed at the drives and shares: open a folder to see those inside it.
   Find at its top shows the folders whose path holds what you type, anywhere in it and in any case (2024 finds D:\Photos24 and D:\Video24 trip), in bold, with the folders leading to them open; a folder inside one found is reached by opening it. Enter or a double-click chooses; Enter in Find chooses the only folder found. Enter in a pair’s field opens the picker searching for what you typed. A folder can also be typed in full, in any case. With a million files recorded (52,000 folders), the picker showed them in a third of a second and each search took 0.03 seconds in a test here. Reached at, under each folder, says where this computer reaches it when that isn’t where its database recorded it, such as \PC1\Photos for a D:\Photos recorded on PC1; blank means the recorded folder. It is used to repair files and to open them. For a database of another computer (opened through its share, or named after it), left blank or with only the computer’s name (\PC1), Compare finds the share as Backup does and fills it in; when none holds the files, Repair and the rows’ menu leave that side out, saying what to fill in. + Add pair, below the last pair, adds another; Remove takes one away.
   Right-click a difference in the Path in first column to open the first folder or show the first file in Explorer, and in Path in second for the second; other columns have no menu.

3. Compare first waits while a Catalog or Verify is using either database, on this computer or the one it belongs to, saying so: read mid-run, a database lacks what the run hasn’t saved yet, and SQLite doesn’t promise its recent changes read right from a second computer meanwhile. It then matches the files of each pair by their path inside the pair’s folders (ignoring case and whether the path uses \ or /). Stop comparison, beside Compare, ends it at once and leaves the line it was on in the status (“The comparison was stopped. It was at: …”), so a record or share that is slow to answer can be found; a read already waiting on the network carries on by itself, and what it finds is dropped. The table lists the differences, possible bit rot first in red with ⚠:

   • Damaged copy: both records hold the same checksum, but Verify found one copy changed (possible bit rot) or unreadable. The other copy can repair it.
   • Contents differ: same size and modified time, different checksums.
   • Different version: the size or modified time differs too (the detail says which copy is newer).
   • Only in first and Only in second: a file in one pair folder and not the other.
   • Not compared: a file not hashed yet on one side; run Catalog there first.

   Results ▾, beside the filter, shows one of these kinds only (All, the first entry, shows every kind), each with how many the filter shows; it applies under Problems, Information and All alike.

   Files that match are only counted, in the line under the pairs. Each time the table fills (a comparison, a filter change, coming back to Compare scans), the columns fit the differences: the paths are shown whole when there is room, shared between them when there isn’t, and the File column takes the room left. Click a heading to sort by it.
   Export CSV saves the differences the filter shows to the working folder as STUDIO-PC-bit-by-bit-backup-comparison-2026-10-25-1431.csv.
   The comparison stays while you switch to Scan and back, until another project opens.

The comparison reads the records, not the files: each side is as its last Catalog and Verify left it. Both databases are opened read-only and never changed; one on a network share works too (see Troubleshooting if it offers no folders). Comparing two records of 200,000 files each took 1.6 seconds in a test here. The databases and the pairs are kept with the open project (without one, only until the window closes), and the table’s layout in the settings.

Repairing from a backup copy

After a comparison, Repair first from second… and Repair second from first… replace damaged files on one side with the good copies on the other: the selected rows, or every damaged copy and every file whose contents differ when none is selected (Select ▾ above the table selects All, None, or Inverts the selection). The two machines can be different computers: give each pair’s Reached at the share each folder is reached through, and run the repair from either computer.

1. A list shows each file to repair, its damaged copy and the good copy, as this computer reaches them. Rows that can’t be repaired from that side are listed as left out with the reason: a good copy marked as damaged in its own record, a different version, a file on one side only.
   Nothing changes until Repair N files. Keep damaged copies (.rotten), beside it and on unless you turn it off (it is remembered, and kept with a project), decides whether each damaged file is kept beside its repaired copy; it can’t be changed once the repair starts.
2. Each file is checked again just before it is replaced. The good copy must still have its recorded size, modified time and checksum. The damaged copy must still have its recorded size and modified time, and other contents or a read error. A file that fails a check is skipped and left as it is, so an edit made since the records were taken is never overwritten.
3. The good copy is copied to NAME.repair-tmp beside the damaged file and its checksum checked. It is given the damaged file’s modified time and permissions, the damaged file is renamed NAME.rotten (NAME.2.rotten when that is taken), and the copy takes its name. A failure on the way removes the copy and puts the damaged file back. With Keep damaged copies off, the copy replaces the damaged file in one step, and the list and the log say the damaged copy was not kept. A file whose contents differ with neither record marking it damaged is always kept as NAME.rotten, whatever the check box says: either copy may be the damaged one, and the repair may go the wrong way.
4. Each file’s outcome (Repaired, Skipped or Failed, and why) shows in the list and is written to STUDIO-PC-bit-by-bit-backup-repair-log.txt in the working folder (or the project’s NAME-repair-log.txt), with the date and time. Stop ends after the file in hand.

The records aren’t changed. Run Verify on the computer that recorded the damaged copy, and it finds the file good again; Accept its finding to clear it from the results.

Damaged copies…, beside the Repair buttons, lists the .rotten files the repair log says were kept and that are still there, with their size, when the repair was made and the repaired file (marked “(not found)” in red when it is gone). Once you are happy with a repair, select its damaged copy (Select ▾ selects all, none or the others) and press Delete selected: after asking, the copies are deleted for good, not moved to the Recycle Bin, and each deletion is written to the repair log, so it isn’t listed again. Only files named .rotten are ever deleted. A copy deleted by hand drops off the list.
*.rotten is on the ignore list’s defaults, so Catalog doesn’t record them (an ignore list saved before gets it from Merge defaults).

When the contents differ and neither record marks a copy as damaged, the app can’t tell which copy is good: open both before choosing the direction.

Backing up

Backup mode, at the left of the toolbar beside Compare scans, makes folders on the Destination (usually another computer, reached over the network) match folders on the Source. It works from the two computers’ records: files the Destination lacks or has in another version are copied to it, and files the Source no longer has are deleted from the Destination for good. It is in the window only, with no batch mode.

1. Source is the record in use (this computer’s, or the open project’s) unless you choose another.
   Destination is the other computer’s record, through its share (\OFFICEPC\D\Bit by Bit Backup\OFFICEPC-bit-by-bit-backup.sqlite) or Browse…. Under each are its files and its last Catalog and Verify, with how long ago: in green when it was minutes ago, orange when hours, and red when a day or more, or never run. A record on another computer is read from a copy on this one, as the plan reads it (see step 3), and ↻ beside it copies it again: click it after a Catalog on that computer, since the share can report the record unchanged for a few seconds after it changed. Records are only as fresh as their last Catalog, so decide from these dates whether to run one first: the Source’s in Scan, the Destination’s on its own computer.
   From Compare scans takes Compare’s first and second databases as the Source and Destination, and Compare’s folder pairs (first as Source, second as Destination, with their Reached at) in place of the pairs here; and each list offers those two, and the last 5 Destinations planned to, kept between sessions (in this computer’s settings file, [recent]).
2. Pair folders as in Compare scans: the Source folder on the left, the Destination folder that should match it on the right. The Destination folder may be new, with nothing recorded in it yet. Reached at under the Destination folder is where this computer finds it, the other computer’s share (\OFFICEPC\D\Photos for its D:\Photos). Left blank, or with only the computer’s name (\OFFICEPC), Plan backup finds the share itself when the record is another computer’s (opened through its share, or named after it, OFFICEPC-bit-by-bit-backup.sqlite): it tries shares named after each folder on the way (\OFFICEPC\Photos), after the drive (\OFFICEPC\D\Photos) and the drive’s administrative share (\OFFICEPC\D$\Photos), and takes the first that holds the files the record lists there, with their sizes and modified times (4 in 5 of up to 20). It fills in Reached at with it and says so. That computer’s own path (D:\Photos) is never used from here, since it could be a folder of this computer’s. A folder new to the backup is reached through the share of the folder above it. For this computer’s own record, blank means the folder as recorded. A project keeps Backup’s records and pairs apart from Compare’s, starting from Compare’s pairs until you change them.
3. Plan backup compares the records (waiting first, as Compare scans does, while a Catalog or Verify is using either), then looks at each file to act on, as it is on disk now (its size and modified time, nothing read): each folder is listed once, 16 folders at once, so over wifi each file doesn’t wait its turn on the network, and nothing inside a folder the Destination doesn’t have is looked at (a first backup is mostly new folders). The status line says “Looking at each file to copy or delete on the disks… 1,200 of 52,318 (\OFFICEPC24)” meanwhile: the files looked at so far and the last folder started on. Stop plan ends the plan at once and leaves that line in the status (“The plan was stopped. It was at: …”), so a share or folder that doesn’t answer can be found. A look already waiting on the network carries on by itself, and what it finds is dropped. Plan from records, beside it, skips that look and trusts the Destination’s record, so the plan is ready as soon as the records are compared. Use it when the Destination’s record is fresh (its last Catalog is shown under it): a file changed on the Destination since that Catalog isn’t flagged in orange. What earlier backups copied and deleted is known from their pending updates (see below), so it isn’t listed again. Start backup still looks at each file as it comes to it, so a file already there with the Source’s size and time isn’t copied again, and one already gone isn’t deleted; both count as Already done. A record on another computer (a share or a mapped network drive) is first copied to this computer’s tmp folder in large reads and read from there; the next comparison or plan uses the copy again while the record is unchanged, so only the first one after that computer’s Catalog reads it over the network (“Copying … to this computer, to read it faster”). The table lists each file’s action, its Source and Destination paths, its size and a detail, with the filter To do (Copy, Replace, Delete), Reports, Already done and All. Beside it, Actions ▾ narrows To do to All, Copy, Delete or Replace, each with its count (lit only on To do). The table shows 10,000 rows at a time: ◀ Previous and Next ▶ below it move through the rest of the list, on every tab and with the filter applied, and the note says which rows are shown (“Rows 10,001 to 20,000 of 52,318”); Export CSV saves them all. The # column numbers the files to copy, replace or delete in the order the backup takes them: the copies and replacements pair by pair and file by file, then the deletions. Click its heading to sort by it (the pages follow that order too): during a backup the files being worked on show at the top of To do as Copying…, Replacing… or Deleting… in bold, and each one done leaves To do, so the next ones move up. With several copies at once, the files in hand are the next few numbers. #, Action, Source, Destination and Size are as wide as their longest cell; Detail takes the room left, cut short when there isn’t enough (Source and Destination give up room only when even that isn’t enough):
   • Copy: not on the Destination.
     Replace: on the Destination in another version, whichever side is newer. A Destination file changed since its record is still replaced or deleted, and shown in orange with a note.
   • Delete: on the Destination but not in the Source’s record (and, when the backup comes to it, not in the Source’s folder either). A folder or file left out on either side is left alone on both: one the ignore list takes (a place added by its full Source path, C:\Data\Scratch, also takes the same place in the Destination folder), one unticked inside a ticked folder in the Source’s record or the Destination’s, and one the Source’s record doesn’t choose at all (a pair of D:\Photos when the Source ticks only D:\Photos24 leaves the Destination’s other folders alone: the Source’s record knows nothing of them). It is neither copied nor deleted, even when the Source’s record still holds files from before the untick. The ignore list is this computer’s together with the one the Source’s record was last cleaned with on its own computer, so a plan made on any computer leaves the same files alone.
   • Report, in red: a damaged copy (the same size and modified time on both sides but other contents, or a file its record marks as possible bit rot or unreadable). Nothing is done to it; Compare scans can repair it.
   • Already done: the Destination already has the Source’s size and modified time (within 2 seconds, for drives that round them), or the file to delete is gone. A pair is left out, with the reason under the pairs, when its Source folder can’t be reached (an unplugged drive or a share offline never empties the Destination), when its Destination can’t be reached, or when its two folders are the same or one holds the other, however each is reached (C:\Photos and the share \THISPC\Photos of it are one folder). A file whose recorded path would lead out of its pair’s folder (a damaged record, or one changed by someone else on its share) is reported, and nothing is done with it. Export CSV saves the rows shown. Rows aren’t selected, since nothing acts on chosen rows.
     Right-click a row’s Source to open the source folder or show the source file in Explorer, and its Destination for the destination’s (a file not copied yet opens the nearest folder above it); other columns have no menu.
4. Start backup asks first, with the number and size of the copies, replacements and deletions, and names each folder whose Reached at the plan filled in itself, from a sample of the files recorded there taken from all over the folder: check it is the right one, since its files will be replaced and deleted. When a pair’s deletions are more than a tenth of its Destination files, a second question asks whether the Source is complete. Copies come before deletions. Several files are copied (or deleted) at once: over a network each file waits on several round trips (opening, creating, renaming, setting its time) that leave the link idle between small files, and a few at once fill those gaps. In a test with 40 small files and a 50 ms wait on each, 4 at once took 0.75 seconds where 1 took 2.8. Every backup starts with 4 at once and adjusts as it goes, between 1 and 16: every 10 seconds it tries one more, or one fewer, and keeps the change only when the files a second plus MB a second rose by a tenth or more, so it finds what the network and the files allow (more for small files, fewer when large ones already fill the link); a try that didn’t help is undone.
   There is nothing to set. A Source file that still has the size and modified time its record gives must also have its recorded checksum, taken as it is read, or it is skipped as possible bit rot (“Tried to copy, but its contents no longer match … run Verify there”). A Source file changed since its record (another size or time: an edit, not bit rot) is copied as it is now, and its row says so; one that changes while it is copied fails that try. A file no longer in the Source is skipped.
   Before each deletion the backup looks in the Source too: a file that is there after all (put back since the Source’s last Catalog) is skipped, “Tried to delete, but it is still in the Source …; run Catalog there”, and with the Source’s folder out of reach the deletion fails and is tried again like a copy. The copy is written to NAME.backup-tmp beside its target, flushed to the disk, given the Source’s modified time, then swapped in. A read-only target is made writable first. A file that fails goes back on the queue and is tried again, 10 seconds later, at most 3 more times; a try that failed while its folders couldn’t be reached isn’t counted. When 5 copies or replacements fail in a row, no new files start and the backup checks that the pair’s Source and Destination folders can be reached, again every minute for up to 5 minutes (a share over wifi can drop and reconnect by itself; the line under the pairs says it is checking). Once they can, the failed files are tried again first and the backup goes on.
   If they still can’t, the backup pauses itself, says why, and asks you to click Resume once the connection is back; the failed files are then tried again first. The Detail of a file skipped or failed says what was tried: “Tried to copy, but it failed after 4 tries: The network path was not found (…)”, or “Tried to replace, but …”. Folders the deletions left empty are removed unless the Source has them. Copies aren’t read back over the network: the Destination’s next Catalog reads the new files on their own disk, and Compare scans shows any that differ. Stop ends after the files in hand, leaving no temporary file. Pause, beside it, also finishes the files in hand and then starts no more (“Pausing: finishing the 2 files in hand”, then “Paused, 120 of 400 files done”); its label turns to Resume, which goes on where it left off. The time paused is left out of the speed and the time left. Stop works while paused too.
5. The line under the pairs shows the files and bytes done, the number copied at once, the speed and the time left, worked out as Scan’s is.
   Bytes are counted only once they are on the Destination’s disk (each copy is flushed every 32 MB), since over a network share Windows takes writes into its cache at once and sends them later, so the speed is taken over the last minute rather than 5 seconds. A file found already there, skipped or failed counts as done at once but is left out of the time a file and a MB take: counted in, a backup planned from the records after one that was stopped, with its first files already there, said 35 seconds were left where 50 minutes were in a simulation (the owner saw 3 hours left after 10 hours for 10 of 28.5 GB). A file tried again after a failure counts its bytes once, so the speed never goes below 0. Each row then shows what was done (Copied, Replaced, Deleted, Already done, Skipped or Failed, and why): a file copied, replaced or deleted, or found already done, moves from To do to Already done, and one skipped or failed stays in To do. Every action is written to STUDIO-PC-bit-by-bit-backup-backup-2026-10-29-1431.csv in the working folder (or the project’s NAME-backup-...csv), for Excel. Plan backup again to see what is left.

The backup holds the run lock of both records, so no Catalog or Verify runs on either computer meanwhile; when the Destination’s can’t be taken, it asks whether to go on without it. It checks there is room on the Destination’s drive for the copies, less what they replace, with 1 GB to spare. The Source’s record is opened read-only and its files are only read.

The backup doesn’t write into the Destination’s record over the network (SQLite can’t safely be written that way from another computer).
Instead it writes each file it copied, replaced or deleted, with the copy’s checksum taken as the Source was read, to a small file of its own beside that record, named after the computer that backed up and when (LAPTOPX1-bit-by-bit-backup.pending-STUDIO-PC-20261101-120400-000.csv), so backups from several computers never share one. It is written as ...csv.writing and renamed when the backup ends. Plans and comparisons read these files as if they were part of the record, so a plan made right after a backup, even Plan from records, has nothing left to do. The Destination’s next Catalog or Verify, on its own computer, folds them into its record, oldest first, and deletes them: a copied file is recorded with the backup’s checksum only if it still has the size and modified time the line gives, and a deleted file is forgotten only if it is gone; anything else is left to the Catalog, which reads the file as before. So the copies aren’t read again or listed as added, the deleted files aren’t reported missing, and the run’s summary says how many files were recorded from backups. A pending file whose folders can’t be reached there (a drive not connected) waits for a later run; one left as .writing by a backup that never ended is folded in once it is a day old. Each pending file processed, or kept for later, gets a row in the activity log (Pending updates, Processed or Kept) with its name and what was done with its lines, and batch mode writes the same line to its log. While it folds them in, the progress panel says “Recording what backups copied and deleted” with a count, and Stop ends the run there; what was folded in is kept, and the next run folds the rest.

Batch mode and Task Scheduler

--batch-command catalog, verify or catalog-and-verify runs without a window, on the folders last ticked in the window without a project (or a project’s, with --project-file-path), and exits. What it did goes to the console and to the log in the working folder, each problem on a line of its own:

python bit-by-bit-backup.py --batch-command catalog-and-verify

Exit status: 0 when the run finished, whether it found problems or not; 1 when it couldn’t run (the database is locked or damaged, another run is going on, nothing is chosen) or a chosen drive wasn’t connected; 2 for a mistake on the command line.

To run it weekly with Windows Task Scheduler: Create Task, a weekly trigger, and an action “Start a program” with the program C:\Path\To\python.exe (or pythonw.exe), the arguments "C:\Path\To\bit-by-bit-backup.py" --batch-command catalog-and-verify, and “Start in” set to the script’s folder. Choose “Run whether user is logged on or not” so it runs in the background. Only one run happens at a time: a scheduled run that starts while the window is running one exits with status 1, and the window refuses to start one while a batch run goes.

Batch mode doesn’t count as a window, so a scheduled run goes ahead while the window is open (it is often left open for days, and a weekly check silently skipped would be worse) and the window opens while a batch run goes. Only a run on the same database waits for the other, as above; a run on another database, such as a project’s, goes ahead.

The activity log

The app keeps a log of when each of these started and ended, one row for each, with the project open (blank for none) and a detail: Catalog and Verify (in the window, and in batch mode, where the detail starts “Batch mode:”); Accept; loading and refreshing (↻) the First and Second databases in Compare scans and the Source and Destination in Backup; Compare scans’ comparisons; Plan backup and Plan from records; Start backup, with its pauses; and each pending file from a backup that a Catalog or Verify processed (Processed) or kept for a later run (Kept), by its name. A record is logged as loaded when you choose it (from a list, Browse…, From Backup or From Compare scans, or a path typed in full), not when the window or a project opens with it, nor when Compare scans’ first database follows a scan. Each start has an ending row: Finished, Stopped (Stop, or the window closing) or Failed, with what happened or what went wrong.

It is MACHINE-bit-by-bit-backup-activity.csv in the working folder, one per computer, written as each row happens (CSV for Excel: UTF-8 with a byte-order mark, Windows line ends): Time, Activity, Event, Project, Detail. It is never cut short; delete it, with the app closed, to start again.

Activity log, at the right of the toolbar, opens a window listing it, newest first (click a heading to sort by it, as in the other tables). It reads the file every second, so rows appear as they are written, by this window or by a scheduled batch run. Finished rows are green, Stopped and Failed rows red. The window can stay open beside the main one, and opens where it was last closed.

The working folder

The app keeps everything in its working folder: the folder holding the script, unless --working-folder-path says otherwise.
Change working folder…, in the Working folder menu, moves the window to another one until it closes: the next start uses --working-folder-path again, so change it in the shortcut or Task Scheduler task to stay in the new folder. The files in the old folder are left as they are; copy the MACHINE-bit-by-bit-backup* files across (with the app closed) to carry the record, settings and logs over. Every file it writes starts with the computer’s name (platform.node() in upper case, the same as %COMPUTERNAME%), so several computers can share one folder, on a USB drive or a network share, without touching each other’s files:

• The record is one SQLite database per computer, holding each file’s path, size, modified time, checksum and when it was first seen, last hashed and last checked; the chosen rules; the runs; and the results. Only changed rows are written, in batches every 2,000 rows or 2 seconds, so a crash loses at most that much work. A database that isn’t valid is kept as ...damaged-<date and time>.sqlite and a new record starts; one that is locked, isn’t this app’s or was made by a newer version is left alone, and nothing runs until it can be used.
• Settings are in [window] (size, place and maximized; the window always opens in Scan), [panels] (the tree’s width), [dialogs] (the results table’s sort and the column widths you dragged, and the Compare table’s; and each popup window’s size and place, so every one opens as it was last closed, even one still open when the app closed), [verify] (Bypass the cache), [run] (Run priority), [results] (the filter and Show skipped), [projects] (the folder Set projects folder chose) and [recent] (the folder of the project used last). A value that can’t be read falls back to its default.
• Reports are CSV for Excel: UTF-8 with a byte-order mark and Windows line ends, a heading line (Kind, Path, Found, Detail, Recorded checksum, Checksum found), the rows in the table’s order.

Checksums are SHA-256, so any of them can be checked with another tool: certutil -hashfile FILE SHA256, PowerShell’s Get-FileHash, or sha256sum.

Command line

python bit-by-bit-backup.py [--working-folder-path FOLDER] [--project-file-path PROJECT] [settings]
                           [--projects-folder-path FOLDER]
python bit-by-bit-backup.py --batch-command catalog-and-verify [--project-file-path PROJECT] [settings]

Every option takes a value, as --name VALUE or --name=VALUE, in any order; an option not given is blank.
python bit-by-bit-backup.py --help lists them with their defaults.

The settings match the window’s fields one for one. Given when the window opens, they fill in its fields (over the project’s, which then shows as changed); in batch mode they apply to the run. Blank means the project’s setting, else in the window the one used last, in batch mode the default:

A mistake (an unknown option, a value out of range or not one of the choices, a Verify option with --batch-command catalog, --show-results, --show-skipped or --repair-keep-damaged in batch mode, a bare option, a positional argument) prints the error and exits with status 2.
--verify-hours, --verify-older-than-days, --threads-per-ssd, --background-priority and --full-speed were taken out on 2026-10-31 and are now unknown options: take them out of a scheduled task’s arguments (--run-priority normal is what --background-priority FALSE was).
--backup-copies and --backup-adjust were taken out on 2026-11-02, with the window’s Copies at once and Adjust: every backup adjusts the files at once by itself.

Keyboard shortcuts

How it is fast

The app runs at the disk’s read speed and skips reading wherever it can.

• Catalog reads only what changed. Folders are listed with os.scandir, which on Windows gives each file’s size, modified time and attributes with the listing, and a file whose size and modified time match the record isn’t opened.
• One reader per hard disk, several per SSD. Files are grouped by drive, and every drive is read at the same time, each by its own readers: one on a hard disk (two would make its heads seek between files), two on a network share, and on an SSD a number worked out for that drive (Windows is asked whether the disk has a seek penalty, and which bus it is on). An NVMe drive (3,000 MB/s and up) gets four, a SATA one (about 550 MB/s) two, since more readers would only wait on the drive; neither gets more than half this computer’s processors, so a scan leaves it usable (Full speed gives more). On an SSD, files under 1 MB get one reader of their own: Python threads handing tiny reads to each other ran slower than one thread (9,000 small files a second with one reader against 3,400 with four, here).
• Large sequential reads into a reused 4 MB buffer with the sequential-scan hint, and SHA-256 from OpenSSL, which releases Python’s lock while it hashes, so several readers really hash at once.
• Batched writes. Readers hand results to one writer, which commits in batches, with WAL journaling.
• The window never waits on a disk. Counting the ticked files, Save project as’s copy, listing a folder in the tree and reading a drive’s label run on threads; the window waits at most 50 ms for them, so quick ones show at once and slow ones show “Loading…” or “Counting…” until they are done. With a million files recorded, a tick held the window 0.45 to 0.64 s before, and holds it 0.05 s now, the count showing 0.46 to 0.58 s later; Save project as held it for its whole 10 seconds, and now for 0.05 s.

Measured in the cloud container the tests run in (Python 3.12, Linux, a virtual disk), not on a Windows PC, so take them as a sketch: SHA-256 hashed 1.26 to 1.34 GB/s on one thread. Verify of 1.1 GB in 20,016 files (16 of 64 MB and 20,000 of 4 KB), with the system’s cache emptied first, took 3.7 s with an SSD’s readers and 6.3 s with one reader; the 16 large files alone ran at 780 to 850 MB/s. With the files in the cache, small files ran at about 9,000 a second and the large ones at 1.7 to 1.9 GB/s with four readers. The virtual disk’s own cache makes these vary from run to run. For scale, a hard disk reads roughly 150 to 250 MB/s, a SATA SSD 550 and an NVMe drive 3,000 to 7,000 (typical figures, not measured).

Run priority ▾, at the top right of the window in every mode (--run-priority, kept with the settings and in a project), has three choices:

• Background, to begin with, keeps the computer usable while a scan runs: every thread of a Catalog or Verify that reads, hashes and writes takes processor time and disk reads only after whatever else is running (Windows’ background mode for a thread; nice elsewhere). The window itself is never put in the background, so it stays as responsive as ever. A run takes longer while something else is busy, which is the point.
• Normal runs as any program does.
• Full speed uses what the computer has to finish sooner, leaving it slower to use meanwhile, as below.

It is read when each run, comparison or plan starts: a change applies from the next one. At Full speed:

• Catalog and Verify run without background priority, an NVMe drive gets a reader for each processor up to 8 (a SATA one stays at two, which it can’t outrun; a network share gets four), and files under 1 MB get two readers instead of one, whose waits on the drive overlap.
• Catalog lists eight folders at once, on threads, so waits on the disk or the network overlap; one thread still writes the record.
• Compare scans and Plan backup split a pair of 50,000 files or more into slices of at least 20,000, each compared in a process of its own, up to one per processor. Comparing is Python work that one process can’t spread over the processors (threads made it slower). Two records of 400,000 files took 7.3 s in one process and 3.5 s in four, in the cloud container the tests run in (4 processors), with the same rows. Plan backup also looks at 64 Destination folders at once rather than 16.

A Verify soon after a Catalog may read files from the system’s file cache rather than the disk, and so prove little; the status bar warns when files hashed less than a day ago were checked. To check again at once (when testing, say), tick Bypass the cache (--verify-bypass-cache TRUE): on Windows each file is opened unbuffered (FILE_FLAG_NO_BUFFERING), so every read comes from the disk; elsewhere its cached pages are dropped first. It is slower when the files would have been in the cache, and doesn’t fill the cache with them.

Testing

The tests use pytest and drive the real window under Xvfb on Linux:

python -m pytest                 # all 178, about 90 seconds
python -m pytest -m "not gui"    # the 101 that open no window, which need no Tkinter either

The tests import the bit_by_bit_backup package beside them. To check that a test fails when its feature is broken, copy bit-by-bit-backup.py and bit_by_bit_backup to another folder, break the feature there, and run the test with APP_ROOT set to that folder (APP_ROOT=/tmp/broken python -m pytest -k name).

They cover the rules and the tree’s check boxes; Catalog (unchanged files not read, added, modified, missing, moved and returned files, links and the app’s own files skipped, a deleted folder against an unplugged drive); Verify (bit rot, unreadable and locked files, bypassing the cache, oldest first and every file read, one reader per hard disk and several per SSD, Stop); Full speed (more readers, no background priority, folders listed on several threads with the same record, a comparison split across processes giving the same rows); folders with a volume mounted on them followed; one run at a time; one window at a time, batch mode not held back; Accept, and accepted skips staying accepted; damaged, foreign and newer databases; comparing two records (paired folders of other names, case, separators, a drive’s root, one record against itself, unusable records, damaged copies, waiting while a run uses a record); repairing from a backup copy (reached under another name, both copies checked again, edited and changed files left alone, the .rotten copy kept or not, the log, the damaged copies listed from it and deleted); backing up (copies, replacements and deletions with the folders they leave empty, nothing copied twice, damage reported and never copied, a Source changed since its record copied as it is now, unreachable and overlapping pairs left out, the share found for another computer’s folder, big deletions asked about again, the pending updates left beside the Destination’s record, read by plans and folded in by its Catalog after checking each file, each pair’s files counted as the comparison reads them, a record on another computer read from a copy and the copy used again until it changes, Destination folders listed rather than each file looked at, failures retried and leaving no temporary file, several files at once adjusted to keep only what helps, 5 failures in a row checking the connection and the failed files tried again once it is back, a connection still lost pausing the backup, the run locks, room on the drive, long Windows paths (in copies and repairs too), Backup mode in the window and its choices in a project, folders ignored or unticked on either side left alone on both); the ignore list (names and paths, its file, runs and comparisons leaving its files out, forgetting those recorded before); project files, and batch mode with a project; the CSV format; the machine-named files; batch mode as a scheduled task runs it, with exit statuses, without Tkinter too; command-line mistakes; option values written out for Scriptary; and the window (progress, results, filter, Show skipped, both modes, sort, columns fitted after each run and comparison, the window’s size and place and other settings saved and hand-edited, palette colors, scaling, fitting at the smallest size, closing during a run, projects, command-line settings in the fields, the repair list, going to a pasted path, Backup’s Actions filter and pages, sorting by the backup’s order with the files in hand marked, Plan from records, Stop plan, Stop comparison, Pause and Resume, the Activity log window, Deselect all, Show ticked only, Show unticked within ticked and both together, managing projects, the projects folder, opening on the record’s ticks, the Ignore list window, each run replacing the results, and the work left to threads: counting the choice, Save project as’s copy, folder listings and drive labels, a database slow to answer, the Run priority menu). All 164 pass at 100% and at 150% display scaling (Xvfb :98 -screen 0 3000x1800x24 -dpi 144 and DISPLAY=:98 openbox, then DISPLAY=:98 python -m pytest).

To see the app find bit rot on Windows, damage a copy of a file with simulate-bit-rot.ps1, beside the app’s script:

.\simulate-bit-rot.ps1 -FilePath "D:\Test\sample.jpg"

It flips one byte at a random place (or the one -Offset N names, counted from 0) and keeps the file’s size and modified time, so the next Verify reports possible bit rot. It names the byte it flipped; running it again with that -Offset puts the file back. If Windows won’t run scripts, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned once.
Under Linux and macOS it works with PowerShell 7 too, but their modified times are kept only to 100 nanoseconds (NTFS’s own precision), which the app sees as an edit.

Troubleshooting

• A Catalog stays at “Starting…” or “0 of 0 files”, and Stop only says “Stopping…”: before it listed the folders, the Catalog was folding in a backup’s pending updates, and forgetting each file the backup deleted went through every open finding in the record (12 ms a file at 200,000 findings, 10 minutes for 50,000 deletions) without heeding Stop. Findings are now found by the file’s index (under a millisecond for 500 files), the panel says what the run is doing before the walk (“Forgetting the files the ignore list leaves out”, “Recording what backups copied and deleted”), and Stop ends it there.

• “Bit by Bit Backup is already open on this computer”, or starting the app only brings a window forward: one window runs at a time. Use the open one (for a project, Projects ▾ > Load project there). If no window is to be seen, another user signed in to this computer has it open, or it is still closing; it can also be an app run as administrator, which a window started normally can’t bring forward.

• A folder opens on “N items hidden by the filter” (often after loading a project, which opens on Show ticked and unticked within ticked): the filter shows only what is ticked and the exceptions inside it, so an unticked folder’s contents are hidden. Choose Filters ▾ > Show all to see and tick them.

• “The backup stopped on an unexpected error: …”: something the backup didn’t expect, such as the disk holding the backup log filling up. It stops at once and says what happened in the line under the pairs and in the backup log; what was copied or deleted before is in the log. Free the room (or report the error), plan again and start the backup again: files already copied count as Already done.

• Files listed as missing on one Catalog and back again on the next, with nothing changed: their paths were longer than Windows’ 260 characters, which the folder’s listing shows but opening a file doesn’t reach without Windows’ long form. Every file is now read through it, so they are read like any other. Backup and Repair use it too, on both sides, for paths from 240 characters (a new folder’s path must be under 248).

• The same files are listed as Missing and Added after every Catalog, even after Accept: files that come and go between Catalogs. In a working folder inside a ticked folder, any computer’s files of this app or of Bit Rot Detector (records, their -wal and -shm files, logs and reports) are now left out, and the ignore list’s defaults leave out every database’s temporary files. A list saved earlier gets them from Merge defaults. For anything else that comes and goes, add it to the ignore list.

• “Paused: 5 files failed in a row and … couldn’t be reached for 5 min” during a backup: the Source or Destination folder stayed out of reach after the failures. Shorter drops are waited out. Check the share in File Explorer, then click Resume: the failed files are tried again first. The Detail column of each Failed row, and the backup log, say what Windows reported. The backup’s own adjusting takes fewer files at once when more don’t help.

• Failed: “Cannot create a file when that file already exists” in a backup: Windows’ error 183, about the file’s folder, not the file. Making the folder failed because Windows said it was already there while Python couldn’t see it as a folder, which a share’s own top folder (\LAPTOP\Downloads, where a pair’s top files go) or a share’s cache just after another copy made the folder can cause. A folder that is there is now used as it is. The Detail column now ends with the path the error was about, so a failure says which file or folder Windows refused.

• “Waiting for the Catalog or Verify using … to finish” in Compare scans or Backup: that database’s run lock is held, by a run in this window, a scheduled batch run, or the app on the computer it belongs to. The comparison starts by itself when the run ends; Stop gives up waiting.

• “… hasn’t answered in 15 seconds” in Compare scans or Backup: a database is on a computer that is busy, asleep or out of reach. It is read on a thread of its own, so the window can still be used (and closed); the comparison or plan goes on by itself once it answers. One a run is updating isn’t opened when it is chosen (“being updated by a Catalog or Verify”), and a lock held for more than 10 seconds is reported as the database not being readable.

• “Another run is going on”: a scheduled batch run, or the window on another computer sharing the folder under the same computer name, holds the record. Wait for it to finish.

• “… is not this app’s database” or “made by a newer version”: the file named isn’t one this version can use. It is left as it is; move it away to start a new record.

• “Not found, so not checked (is the drive connected?)”: a chosen drive or share wasn’t there. Connect it and run again; its record is untouched. In batch mode this exits with status 1.

• Possible bit rot on a file you edited: a program changed its contents without updating its modified time (some tag editors and backup tools can be set to keep it). If the change was yours, Accept it.

• “Can’t be opened (Access is denied)” in the tree: this computer’s account may not list that folder. Catalog skips it and lists it under Skipped, and files recorded in it before are left as they are, not reported missing. To check it, run the app as administrator (for batch mode, tick “Run with highest privileges” in Task Scheduler), or give your account read access in the folder’s Properties, Security tab. Windows’ own protected folders are left out of the tree, so they don’t show this.

• A Cryptomator vault (or a mapped network drive) isn’t in the tree: Cryptomator mounts an unlocked vault for your own account, as a drive letter or in a folder. An app run as administrator can’t see drives mounted for your account (Windows keeps them apart), so run Bit by Bit Backup without administrator rights, or use a folder in the vault from the account that unlocked it. A vault mounted in a folder is shown as that folder. A vault mounted as a network drive (WebDAV) shows as its drive letter; its \cryptomator-vault@42427\DavWWWRoot\... path can also be pasted in the path field. Unlock the vault before the window opens, or paste its path, since the drive list is read when the window opens.

• No folders offered for a database in Compare scans: the picker offers the folders that database holds files in, so one with none recorded yet offers none; the line under the pairs says so, and says why when the app can’t read a database.

• “Can’t be read with its latest changes” in Compare scans: the database is in a folder this computer can’t write in (a read-only share, or one SQLite can’t share its index on), so its -wal file, which holds its recent changes until it is closed, can’t be read with it. Close Bit by Bit Backup on the computer that uses it, or let its run finish, then click ↻ beside the database.

• “The Destination folder … can’t be reached” in Backup: the share in Reached at isn’t there or this account can’t open it. Open it in File Explorer first (which may ask for the other computer’s password), then plan again.

• “… none of its shares checked holds its files: fill in Reached at” in Backup: the other computer shares the folder under a name the plan doesn’t try, or not at all. Share the folder (or a folder above it) on that computer, open the share in File Explorer, and type it in Reached at (\LAPTOPX1\Downloads for its C:\Downloads).

• “The Destination’s record … is in use” when a backup starts: Catalog or Verify is running on the other computer, or this computer can’t write the lock file beside its record. Wait for the run, or go on without the lock when you know nothing runs there.

• A backup skips a file as “possible bit rot”: it has the size and time its record gives but other contents. Run Verify on the Source, and repair the file from a good copy before backing it up.

• Many skipped OneDrive files: they are cloud-only. Make them “Always keep on this device” to check them.

• The window doesn’t start under pythonw: a mistake on the command line is shown in a message box; otherwise run it with python to see the error.

Limitations

• Comparing two databases, repairing from a backup copy and backing up are in the window only; batch mode does none.
• Verify reads every chosen file each time (its time limit and the days to skip went on 2026-10-31). On a large collection, Stop it when it must end: the next Verify starts with the files not checked yet.
• A backup knows only the files the two records hold: a Destination file its record doesn’t list is never deleted, and a Source file not recorded yet isn’t copied. Empty folders aren’t copied, and only the modified time and the read-only mark are kept, not other attributes or permissions.
• A moved or renamed file is copied again and its old copy deleted, which takes time over wifi after a large reorganization.
• Backup speed over wifi hasn’t been measured yet; the tests copy between local folders. Plan backup was measured on two records of 400,000 files on one computer: 18.1 seconds before the 2026-10-30 changes, 7.7 after. Looking at 40,000 Destination files with a simulated 2 ms network wait took 87 seconds one at a time and 1.6 seconds by folder. With that wait, 4,000 files in 2,000 folders of two took 1.1 seconds to look at with eight folders at a time and each file looked at alone, 0.36 seconds with 16 folders each listed once, and 0.07 seconds when the folders were all new to the Destination.
• A copy of another computer’s record is used again while its size and modified time are unchanged. Windows may report a share’s file sizes and times up to about 10 seconds late, so a comparison started within seconds of a change on the other computer can read the record as it was just before. ↻ beside the record copies it again whatever its size and time say.
• A repair reads both copies over the network to check them, then copies the good one: about three times the file’s size crosses the network.
• The Windows-only parts (asking whether a disk is an SSD, drive labels, the drive list, junctions and volumes mounted in folders, OneDrive file attributes, Windows’ own hidden and system folders, unbuffered reads for Bypass the cache, file locking with msvcrt, Task Scheduler) haven’t been run by the tests, which run under Linux.
• Two partitions of one hard disk count as two drives and are read at once, which makes the disk seek between them.
• Paths are recorded as given by the system, drive letter and all: a drive that comes back under another letter is a new place to the app (its old record shows as not connected).
• WAL journaling needs a folder the computer can share memory through; on a network folder where it isn’t available, SQLite’s default journal is used instead, with slower commits.
• Verify can’t tell bit rot from a program that writes a file and then sets its modified time back.