The Problem
A bunch of DNGs were corrupted but I didn't want to restore everything and lose recent changes. This system allowed me to restore from a backup of the photo library files on a remote Linux machine using Rclone, an SQLite3 query, and a bit of Python.
The Solution
Collect Photos
First, you'll need to put all of the photos needing restoration in a collection. I named mine `INVALID DNG` and populated it using the DNG validation tool builtin to Lightroom Classic.
SELECT List of Photos Needing Restoration
Use the query below - I ran it via a Jupyter notebook in Jetbrain's DataSpell after connecting the catalog's `.lrcat` via the SQLite3 database connector. If you named the collection something other than `INVALID DNG` then you'll need to update the WHERE condition for `Col.name`.
One important note: You don't need to close out Lightroom Classic to access the SQLite3 database. However keep in mind if you hold a lock too long on anything it'll hold up Lightroom Classic for that period. I also don't recommend making ANY changes; treat it as read only.
SELECT Col.name AS CollectionName,
LibFile.idx_filename AS FileName,
concat(Root.absolutePath,Folder.pathFromRoot) AS FileDirectoryAbs,
concat(Folder.pathFromRoot, LibFile.idx_filename) AS FilePathRel,
concat(Root.absolutePath,Folder.pathFromRoot, LibFile.idx_filename) AS FilePathAbs
FROM
AgLibraryCollection AS Col
JOIN AgLibraryCollectionImage AS ColImg
ON Col.id_local = ColImg.collection
JOIN Adobe_images AS Img
ON ColImg.image = Img.id_local
JOIN AgLibraryFile AS LibFile
ON LibFile.id_local = Img.rootFile
JOIN AgLibraryFolder AS Folder
ON LibFile.folder = Folder.id_local
JOIN AgLibraryRootFolder AS Root
ON Folder.rootFolder = Root.id_local
WHERE
Col.name = 'INVALID DNG'
;Restore The Photos
The Python block below is how I restored the photos from backup. You'll need to replace `Z:\restore\folder` with the folder that holds the copy of data to restore from. If you have your restore organized differently than the photo library or use multiple roots, then you might need to update the script below based on your situation.
import os,os.path
import shutil
res={}
for index, row in ToClean.iterrows():
print(f"Working on {repr(row['FileName'])} in {repr(row['FilePathAbs'])}")
if row['FilePathRel'] not in res:
res[row['FilePathRel']] = dict(
FileName = row['FileName'],
FilePathAbs = row['FilePathAbs'],
FilePathRel = row['FilePathRel'],
BackupFilePath = row['FilePathAbs']+'.bak',
BackupFileName = row['FileName']+'.bak',
BackupedUp = None,
RestoredFromPath = None,
Restored=None,
)
if res[row['FilePathRel']]['Restored']:
print("Already restored")
continue
restore=os.path.join(r'Z:\restore\folder',row['FilePathRel'])
if os.path.exists(restore):
print(f"Going to restore {repr(row['FilePathAbs'])} from {repr(restore)}")
res[row['FilePathRel']]['RestoredFromPath'] = restore
if not res[row['FilePathRel']]['BackupedUp']:
if os.path.exists(row['FilePathAbs']) and not os.path.exists(restore):
print(f"Backing up {repr(row['FilePathAbs'])} to {repr(res[row['FilePathRel']]['BackupFilePath'])}")
os.rename(row['FilePathAbs'],res[row['FilePathRel']]['BackupFilePath'])
res[row['FilePathRel']]['BackedUp']=True
elif os.path.exists(restore):
print(f"Restore file {repr(restore)} already exists - not backing up")
res[row['FilePathRel']]['BackedUp']=True
else:
print(f"Orig file {repr(row['FilePathAbs'])} doesn't exist - not backing up")
res[row['FilePathRel']]['BackedUp']=False
else:
print("Skipping backup - already done")
print(f"Restoring {repr(row['FilePathAbs'])} from {repr(restore)}")
shutil.copy(restore,row['FilePathAbs'])
res[row['FilePathRel']]['Restored']=True
else:
res[row['FilePathRel']]['Restored']=False
print(f"Restore file {repr(restore)} doesn't exist")
print(f"Done with {repr(row['FilePathAbs'])}")
If you need to load the restore files from a remote machine/system, you might be able to use rclone to mount the data locally.