Writing XMP Files in Lightroom (why you shouldn’t and when you should)
This is a follow-up for reason #7 from my post on Monday called “If your Lightroom is running slow, it’s probably one of these seven reasons” (here’s the link in case you missed the other 6). #7 was about turning off having Lightroom automatically write .xmp files each time you make an adjustment in Lightroom and I mentioned that most Lightroom users should probably turn this auto writing of xmp files off. So, after some questions, I thought today I’d do a quick Q&A about XMP, when you should use them, and why in most cases you shouldn’t).
Q. What is a .xmp file?
A. It’s a separate text file that holds any edits or metadata changes you made to your RAW file. That’s one seen above — the .xmp file for the RAW file with the same name, but its file extension is .xmp.
Q. Is that text file referred to as an .xmp file?
A. You’ll hear it called either an xmp sidecar, just a ‘sidecar file’ or simply an xmp file. So, if somebody says to you, “Send me the RAW and the sidecar” you know they need your RAW file and that .xmp sidecar file — two files.
Q. I thought Lightroom kept all those changes stored right in the catalog?
A. It does.
Q. So why would you also need to have a .xmp file?
A. If you made edits to a RAW file (let’s say you make the Exposure brighter, you pulled back the highlights, added some Clarity and cropped the image), and you needed to give this RAW image to someone else; maybe a retoucher or another photographer in your studio, and you needed that RAW file to have all those edits you made to the file — that’s when you would need to create a .xmp file. That .xmp text contains instructions for all those changes to your RAW image. Now, when they open that RAW file in a program that supports .XMP (like Photoshop for example, or the Adobe Bridge, or even a different copy of Lightroom) as long as both the RAW file and the .xmp are there together, they would see the RAW file with all your edits.
Q. So, I don’t need the ‘Automatically Write Changes to’ .XMP preference turned on?
A. I wouldn’t recommend it — it slows Lightroom down. In fact, most likely you’ll only need a .xmp sidecar file if and when you’re sharing a RAW image you’ve edited with someone else.
Q. Do I need an .xmp file with my JPEG, TIFF, or PSD files?
A. You do not — those edits you make are baked in the single file when you export them from Lightroom.
Q. Is there a way to create just one .xmp file without having Lightroom slowing things down all the time by writing to a .xmp file every time I move a slider?
A. Absolutely, when you’re done editing, and you’re ready to share the file with someone, in Lightroom just click on the file and then press Command-S (PC: Ctrl-S) and it creates a separate .xmp sidecar file with all your changes written into it.
Q. What would happen if I gave someone one of my RAW files without a .xmp sidecar file?
A. Then none of your edits/changes/metadata would appear when they opened the file on their computer. It will be as though they downloaded the original RAW image from your camera’s memory card.
Q. Is there a way to share a RAW file, with my edits included, but without the 2nd sidecar file?
A. Actually, there is. Convert the file to a .DNG file. That way, the .xmp data is written right into the DNG file itself — no 2nd file needed. You would do this in the Library module, under the Library menu up top; choose ‘Convert Photo to DNG’
Q. How do I turn off Lightroom’s Automatic writing of .xmp sidecar files again?
A. You turn it off by going to the Lightroom menu (PC: Edit), go Catalog Settings; click the Metadata tab, and turn off the checkbox beside “Automatically write changes into XMP”
Q. Hey, isn’t tonight the live broadcast of British photographer Ian Munro’s gallery opening at The Gallery at KelbyOne?
A. It is, and you’re invited to come check out Ian’s interview; see his amazing work, and learn about the man behind the camera — all tonight at 8 PM ET. It’s free and open to everyone. All the details are over on my daily blog today at scottkelby.com – no .xmp file required.
Hope you found that helpful. Have a great weekend, everybody! 🙂
Best,
–Scott
[…] where I need to keep my RAW edits intact when sharing the original RAW file with another user (see this article), so combining the .XMP file and the RAW original into one single file isn’t a benefit to me (I […]
If a person decides they don’t not want or need xmp files for their workflow is there an easy way to delete them all at one time? If so what are the steps? I’m using an iMac and it looks like I would need to go into every sub folder on my hard drive and then have to delete them sub folder by sub folder. I have more 102,000 xmp files now.
Thanks,
Lon
Go to the top level folder and do a file search on your File Explorer (or Finder on Mac) for *.xmp and that should list all your XMPs, Then you can select them, and delete them.
great article. thank you.
not sure why lightroom cc is creating the .xmp files when the box is unticked??
If xmp files are not required (unless sharing), then why is there a LR warning when you uncheck this option warning that the file edits will not be visible in other programs (eg. Photoshop)? I always used to generate amp files, but have never used them. After many years, they have hogged up quite a large drive space.
Thanks for this helpful post! Can I safely delete the old XMP files now that I’ve turned off this option in LR? I want to free up some space.
XMP files are tiny text files that hardly take up any space. You could spend a lot of time deleting them and maybe recover a few hundred MB of space. There are probably better ways to free up space than focusing on XMP files.
I had my catalog corrupt irrecoverably about 6 months ago, where NONE of my MANY backups would restore. There are many similar stories without resolution on the web – apparently a known issue. XMP was the ONLY thing that carried on working: 5k+ edits lost, 40k+ saved by XMP. I now check my XMP settings are on every time I start LR, and copy the sidecars across drives. Rebuilding the catalog is relatively trivial to my mind. I agree, time is very valuable indeed…
Build a catalog for each separate shoot, or project, or whatever. Don’t have to put everything in one, and your corruption catastrophe is a good reason not to! Also, you can backup catalogs. XMP is not the best answer.
[…] where I need to keep my RAW edits intact when sharing the original RAW file with another user (see this article), so combining the .XMP file and the RAW original into one single file isn’t a benefit to me […]
I started shooting with Fuji cameras years ago because the jpgs it creates are better than what I was able to create from the Canon RAW photos I was taking, and obviously a lot faster. I changed my Lightroom settings a long time ago to write changes to xmp I think because I noticed that Lightroom was saving any changes I made to the jpg file itself. I always liked the idea of the photo I take being the digital negative and all changes being stored in Lightroom but when I started shooting jpg I wasn’t sure if this was the case. Does anyone know if it makes any difference whether you are shooting RAW or jpg? I am sure I took someone’s advice but I can’t remember who!!
Thanks
By default, Lightroom only saves changes to the catalog regardless of file type being worked on.
BTW, I thought the graphic “side car” was very clever.
Would you recommend converting to DNG instead to retain the edits
I no longer convert to DNG (it takes longer with no big benefit, except maybe the lack of a sidecar, but I don’t’ generally share my RAW images with someone who needs my edits, so I don’t use .xmp very often. Actually, rarely is more the case).
I take online backup. When done editing, backing up just the XMP (or none, if you prefer not so save it) is much faster than a full DNG (rsync could solve this). For some raw files, like FujiFilm, the DNG is much smaller, but to sum up:
Space is cheaper than time.
One of the benefits of writing xmp data all the time, it serves as a backup of the edits to the image. Should your catalog become corrupted or unusable for any reason, your raw file will still have the edits in the XMP file on disk. Speed vs paranoia – its a trade off.
I’ve had to do it 3 times now (but in earlier versions, like pre-6) (i’ve been on this tool for a long, long time…)
Collections won’t be retained if you them and try to restore using this fall-back technique.
If you use Virtual Copies, they won’t either unless you manually create a snapshot, then you can choose that snapshot after import to recreate your snapshot. (October 11, 2017 Comment section as excellently explained to me by Rob Sylvan)
Right. Flags and history panel data is not written to XMP either.
someone pinged me on my comment. the file the VC snapshot is written to is the BASE image filename… and to write that metadata, you move over to the base file and do a CTRL-S there… there’s not a separate file for the VC is the point, it’s data in and saved associated with the base filename.
Backing up the catalog also serves as a backup to the catalog. 🙂
Perfectly articulated, Mr. Sylvan. 🙂
Consider the following scenario with a setup that is actually recommended for several reasons that have been explained elsewhere:
1. your computer’s internal HDD or SSD containing the LR catalog (.lrcat) and preview files (.lrdata)
2. a first external hard disk to which your HDD or SSD is being backed up very regularly, e.g. using Time Machine on Mac OS (except for the .lrdata file which is usually pretty huge and a backup of it would not be essential)
3a. your second external hard drive containing all the RAWs + XMPs (LR saving the edits to its own catalog file AND to the XMPs)
OR
3b. your second external hard drive containing all the RAWs but no XMPs (LR not saving the edits to the XMPs but only to its catalog file)
4. your second external hard drive is connected via Thunderbolt, USB 3, or Firewire 800 (the latter being my case)
In case a (small) disaster happens and your internal HDD or SSD gives up on you, you have a backup on external disk 1 to restore your system and all data on it. And you still have all your photos (possibly tens of thousands of them, sometimes even more if you’re a pro) that you made and loved and treasured and edited and saved for the last ten years or so, on the second external HDD. Now, of course, you also need to have at least one backup of that 2nd HDD with your photos (RAWs) on it, but that is of no direct concern in this scenario.
However, thinking everything will be fine again after the small disaster of your internal HDD or SSD giving up, as soon as you can restore the backup, horror starts to unfold itself into the scenario when you find out that your backup cannot be restored or refuses to be restored or is simply unreliably restored (with several files missing, amongst which your LR catalog file…).
If that horror scenario happens and you have setup 3a, you will have some work to do re-starting from scratch with the OS and re-installing your applications, BUT you still have all your RAWs AND all the edits, meaning (most of) the work you have done on the photos for the last ten years.
If that horror scenario happens and you have setup 3b, you will have some work to do re-starting from scratch with the OS and re-installing your applications, AND ALL OF YOUR WORK ON YOUR PHOTOS OF THE LAST 10 YEARS IS LOST TOO.
Crazy scenario? Don’t think so as it happened to one of my colleagues a couple of years ago.
Summarising: one backup is just one backup. In my eyes, LR users who take their work, their photos and the time and energy they put in editing them, need to leave the option for LR to save the edits into the XMPs definitely always and everywhere ON.
Which doesn’t make it a luxury to try and have a second backup of your OS and data (LR catalog file…) and of course ALWAYS have at least one backup of your RAWs and XMPs as well.
I have been a faithful LR user right from the first beta version in Summer 2006. I have never had such a disaster as described above (touching wood 3 times including my head), but I would surely not want to see it happen.
Over the years, I have tested the speed of many different LR versions (almost all) with the option to save to XMP either on or off. In my case the difference was minimal every single time. In my eyes, speed isn’t everything, but safeguarding all of your photos and all of your work on them is.
If your catalog get corrupted, you may not notice for some time. If you restore the backup, you will loose the last edits. So the sidecar files can be quite helpful at this time.
I had this problem in the past, so it is not pure theory.
If you run the integrity check with each backup, and frequently back up the catalog, then I have little worry of backing up a corrupted catalog without realizing it.
Every decision in this world of photography/technology comes with weighing the pros against the cons for our own situations. We all have our own risk calculus. Your workflow sounds solid, and you have confidence in it. I have confidence in mine. It is a wonderful time to be alive and have so many options. 🙂
Genius! If only other people had brains and used them!
The interesting twist comes into play if your workflow first involves using Photo Mechanic(PM) where you are populating exif data & possibly including cropping to your raw files. At this point xmp’s are generated via PM. Importing via LR will see the xmp and bring in that info. If I read your article correctly, Scott, after the import via LR the xmp is no longer needed and can be deleted. Assuming of course writing xmp’s is turned off in LR.
It isn’t needed, but there’s nothing to be gained in a practical sense to deleting them either. They only take up a few KB of space, and are ignored by Lightroom after import (unless you are writing to them from Lr).
So, I’m one of those weird people that only keep images to edit in my catalog. When edited and exports to tif and then made into a jpg for web, I delete the series from the LR catalog.
In my case, I assume I should keep the sidecar files, as if I need to reimport years later my catalog will not know what edits I made?
thanks
Yes. Or, just don’t delete them from the catalog? 😀
This article addresses exactly something I was wondering about! I typically work in Lightroom. If I want to make adjustments in Photoshop, Lightroom creates a new file which gets saved back into Lightroom. I am experimenting using Adobe Camera Raw instead of Lightroom because it is easier to move back and forth between the two programs without creating a new copy of the file. Would using a dng or sidecar file allow me to use either workflow? Is there a downside to using a dng rather than the camera’s raw file?
There’s no difference in how Camera Raw works versus Lightroom works when going back and forth between them and Photoshop as far as a copy being made or not.