View the profiles of people named Mike Bombach. Join Facebook to connect with Mike Bombach and others you may know. Facebook gives people the power to. Bombich Software offers a range of backup and cloning software for Mac OS platforms. The company’s main product, Carbon Copy Cloner, was the first software to allow the creation of a bootable Mac OS X backup. The cloner supports block-level clones for fast upgrades; takes incremental backups of items that have only changed after the last. Note: Mike Bombich, developer of Carbon Copy Cloner, recently expounded on the use of the app with macOS 11 Big Sur. It’s worth reading if you’ll be upgrading your Mac to Big Sur. Mike Bombich is on Facebook. Join Facebook to connect with Mike Bombich and others you may know. Facebook gives people the power to share and makes the. By Mike November 3, 2020 CCC 5.1.23+ can make bootable backups of Big Sur on Intel-based Macs. Update Nov 24: CCC 5.1.23 can now make bootable backups of a Big Sur startup disk on Intel-based Macs.
In a recent blog post, Carbon Copy Cloner developer Mike Bombich revealed that he had uncovered a bug in the Apple File System, or APFS, as a result of his work with 'sparse' disk images. Until Apple issues a macOS update that resolves this problem, Bombich Software are dropping support for APFS-formatted disk images.
Apple's APFS file system which is now part of macOS High Sierra apparently suffers from a disk image vulnerability that in certain circumstances can lead to data loss when working with sparse disk images.
PTE: What are Disk Images?
Mike Bombich: Disk images are handy devices. They're files, but they act like a hard drive – you mount a disk image by double-clicking the file, then it behaves like it's another hard drive attached to your Mac. macOS has been using disk images for decades, and we find them particularly useful when making backups to network volumes. By formatting the disk image volume using an Apple-native format, we can do things like back up system files.
PTE: Who is affected by this bug?
Mike Bombich: While the underlying problem here is very serious, this is not likely to be a widespread problem, and will be most applicable to a small subset of backups. Disk images are not used for most backup task activity, they are generally only applicable when making backups to network volumes. If you make backups to network volumes, read on to learn more. What I describe below only applies to APFS sparse disk images. Ordinary APFS volumes (e.g. your SSD startup disk) are not affected by this problem
PTE: Understood. Tell us what the problem is and how you found it?
Mike Bombich: Naturally, when Apple introduced APFS in macOS High Sierra, we sought to offer support for using APFS on destination disk images when doing so would match the format of the source volume. As far as creating and mounting disk images is concerned, APFS and HFS+ are easily interchangeable, so adding support for APFS was very straightforward. Unnoticed by us, Apple, and thousands of developers, however, is a very subtle behavioural difference that is specific to APFS on a sparse disk image.
Earlier this week I noticed that an APFS-formatted sparse bundle disk image volume showed ample free space, despite that the underlying disk was completely full. Curious, I copied a video file to the disk image volume to see what would happen. The whole file copied without error! I opened the file, verified that the video played back start to finish, checksummed the file – as far as I could tell, the file was intact and whole on the disk image. When I unmounted and remounted the disk image, however, the video was corrupted. If you've ever lost data, you know the kick-in-the-gut feeling that would have ensued. Thankfully, I was just running some tests and the file that disappeared was just test data. Taking a closer look, I discovered two bugs in macOS's 'diskimages-helper' service that lead to this result.
In the past with HFS+ formatted disk images, the disk image volume would automatically adjust its free space to accommodate any differences between the disk image volume's capacity and the actual amount of free space on the underlying disk. So for example, if you had created a disk image with a capacity of 500GB on a 500GB network volume, but then you added 400GB of stuff to the network volume outside of the disk image, now there's only 100GB of space for stuff on the disk image. Accordingly, when you mount the disk image, it would report its own disk usage as 400GB and its free space as 100GB (even if there is literally nothing on the disk image volume). The math always felt weird, but the result was right – the disk image can't practically accommodate more than 100GB of data, so the free space should reflect that. This behaviour is documented in Apple's hdiutil man page:
Mike Bombich
To prevent errors when a filesystem inside of a sparse image has more free space than the volume holding the sparse image, HFS+ volumes inside sparse images will report an amount of free space slightly less than the amount of free space on the volume on which image resides.
This behaviour has been a known quantity for many, many years. HFS+ still performs this adjustment on High Sierra, so it does not appear to be a regression, rather just an oversight that is specific to APFS.
If this were the only bug, however, this issue would be just an annoyance. The larger issue occurs when any application tries to write more data to the disk image volume than the underlying disk can accommodate.
The diskimages-helper application works quietly in the background, responding to filesystem requests made to the disk image volume. It's essentially a broker, or middle-man. There's a disk image file on disk, but applications don't interact directly with the file, they need to interact with a filesystem. So the diskimages-helper application presents a filesystem interface on top of that disk image file. When you make a write request to a mounted disk image volume, the request goes to the diskimages-helper application, which translates that request into changes to the disk image file.
When you initially create a 'sparse' disk image file, that file is very small, e.g. <100MB. It's just large enough to hold some pre-allocated space for the filesystem structures. As you copy files to the disk image volume, the file grows. Herein lies the bug. Following the earlier example, suppose you attempt to copy 200GB of data to that 500GB disk image file. This shouldn't be possible because there was only 100GB of free space left on the underlying disk. The APFS disk image reports that there's 500GB of free space available, though, so what the heck, let's do this! The first 100GB of data does successfully get written into the disk image file – the disk image file has grown now to 100GB. But now the underlying disk is completely full, and the disk image file can no longer grow – the diskimages-helper application is getting 'No space left on device' errors when trying to write data to its band files. At this point, you'd think that the diskimages-helper application would do one of the following:
- Report a 'No space left on device' error to the process making the write request
- Refuse additional write requests – sorry, no more space
- Unmount the disk image – we have to stop this insanity
- Quit – please, just stop pretending to do something that you're not actually doing!
- Panic the kernel – we're writing data into a VOID, STOP!
Alas, none of those things happen (and no, it should never panic the kernel, but writing to the void is equally unreasonable). diskimages-helper continues to accept writes, and the application asking to write the data continues to send data, eventually completing with apparent success.
After files failed to actually make it to a physical disk somewhere, you'd think (hope?) that perhaps, at least, the file would appear smaller on the disk image. This is probably the most alarming part of this bug – because the filesystem structures are stored on a section of pre-allocated space on the underlying disk, the diskimages-helper application has no trouble updating filesystem metadata. So file size, modification date, permissions, etc – all of those attributes are fine. In yet another bizarre twist, we found that many times a truncated file would even validate a checksum test. Presumably, the diskimages-helper retains some of the file data in RAM, because again, the data never made it to the underlying disk. This part is perhaps the hardest to explain in text, so I created a video to demonstrate the problem:
PTE: How should we proactively avoid data loss?
Mike Bombich: Carbon Copy Cloner (CCC) creates and uses disk images when you select 'New disk image...' from the Destination selector. Starting in CCC 5.0.4 and up to CCC 5.0.8, CCC will automatically create an APFS-formatted disk image if the source is an APFS-formatted volume. We have posted an update to CCC (5.0.9) that revokes support for creating APFS-formatted disk images. Additionally, if you have a task that is currently backing up to an APFS-formatted disk image, CCC will issue a warning when that task completes indicating that APFS-formatted disk images are no longer supported by CCC. This will not prevent you from using an APFS-formatted disk image, and indeed, if your underlying destination is not overly full, there's no need for panic here. Nevertheless, disk images eventually fill up, so our recommendation to our users will be to migrate away from an APFS-formatted disk image at your earliest convenience.
PTE: What should the average user do about this problem?
Mike Bombich: The average CCC user will be unaffected by this APFS shortcoming. CCC 4 users are completely unaffected; users not yet on High Sierra are unaffected. Our usage statistics indicate that less than 7% of CCC backup tasks leverage a sparse disk image, and of those, less than 12% are APFS-formatted. We recommend that CCC 5 users update to 5.0.9, and then make a brief review of your backup tasks – open CCC and select each task, and read the task plan. If the task is configured to back up to a disk image, it will plainly state that, e.g.:
CCC will copy selected items from Macintosh HD into a disk image at Macintosh HD.sparsebundle on NAS Backup.
If you do have a task that is configured to back up to a sparsebundle or sparseimage disk image, hover your mouse over the source icon. A tooltip will indicate how the source volume is formatted. If your source's filesystem is APFS, then your destination disk image might be formatted as APFS as well. When you run that backup task, CCC will update the disk image as usual, and then when the task completes, CCC will issue an error if that disk image is formatted as APFS. If you see that warning, we recommend deleting the destination disk image at your earliest convenience. Again, if the underlying destination is not very full and has never been near capacity (e.g. if it has always had more than 50GB of free space), then there's no reason to be alarmed and you can remove the destination disk image on a weekend or at night when you have ample time to allow CCC to recreate an HFS+ formatted disk image. If your underlying disk has ever filled up to capacity, though, you should delete the disk image and allow CCC to replace it with an HFS+ formatted disk image.
PTE: Are DMG files affected by this problem?
Mike Bombich: Disk images that end with a .dmg suffix are not affected by this problem. The storage for dmg disk image files is pre-allocated when those files are created, so their storage space is guaranteed. CCC specifically does still provide support for APFS formatted dmg files.
PTE: Is this a problem specific to CCC?
Mike Bombich: No, this problem will affect any application that writes to APFS-formatted sparse disk images that reside on a full or nearly-full disk. I tested this scenario with copies via the Terminal application, Finder copies, and even exported a file from QuickTime. In every case, the application that was copying or creating the file was completely unaware that any problem had occurred when writing the data to disk. In the QuickTime case, I was able to immediately open the exported file and play it start to finish. After ejecting and remounting the disk image, QuickTime couldn't open the file. The core of this problem resides in macOS's diskimages-helper application and can only be resolved by an update to macOS.
Until Apple resolves this disk images bug, we strongly recommend that people avoid using APFS-formatted sparse disk images for any purpose with any application.
PTE: Thanks for this and you can find out more on the Bombich Software website.
May 11, 2012 @ 11:22 pm
hi,
i am trying to make a full clone backup drive, but when i boot offthe newly cloned drive neither 'dropbox' nor 'google drive' areloading. and CCC suddenly appears on the desktop...? i have triedthree times now. any ideas? thanks... corky
CCC Report ID: 9699
I've heard of the Dropbox issue, but the Google Drive one isnew. I added that to this section of the documentation:
Non-registration-related, application-specific oddities [LINK NO LONGER EXISTS]
Mike
hi mike,
thanks! regarding 'google drive'; it says that it can't loadbecause there is no 'google drive' folder, even though thereactually is one! and the app is in the applications folder...
On Fri, May, 11, 2012, at 18:21 , Mike Bombich wrote:
PLEASE REPLY ABOVE THIS LINEAny part of your response that is below this linewill be discarded
Hi Corky:
I tracked down the problem on this one. Google Drive keeps adatabase at /Users/yourname/Library/ApplicationSupport/Google/Drive/snapshot.db. This database records, amongother things, the inode number of your Google Drive folder. Fileand folder inodes are physical addresses that indicate where withinthe filesystem a particular file resides. If you read the contentsof this database, you can compare the values to the inode numberson your volumes, e.g.:
So in this case, I'm booted from the backup volume. You can seethat Google Drive has my Google Drive folder's inode numberrecorded as '1222230' and that's what the inode is for that folderon the original volume, 'Lion'. When I look at the inode value ofthe folder on the backup volume, it's different. The inode numbercan't be preserved in a file-level backup, so Google Drive willalways break when you boot from your backup volume.
Unfortunately there's no easy workaround for this. As long asGoogle Drive records inode numbers, it's only ever going to workwith the original source volume. It doesn't matter how accuratelyCCC preserves the contents of these folders or the databases thatstore information about them, the inodes will always be differenton a new volume.
Bombich Support
To summarize:
Google Drive reports that the Google Drive folder does notexist
Disconnect your account from Google Drive, then sign in again toGoogle Drive to re-establish the 'new' folder on your destinationvolume as the Google Drive folder.
Mike
hi mike,
impressive! i think that did the trick. thanks. and dropbox justmakes you log in again. appreciated the help... corky
On Mon, May, 14, 2012, at 13:30 , Mike Bombich wrote:
PLEASE REPLY ABOVE THIS LINEAny part of your response that is below this linewill be discarded