Login Register
MacInTouch Home Page MacInTouch Discussions

MacInTouch Amazon link...
Time Machine and OS X throttling
 


2016-10-02 at 22:10 #6514   (1)
The other day I received a replacement backup hard drive to replace one that went bad: The problem emerged when it took 40 hours to run a Time Machine backup after upgrading from Mt. Lion to El Capitan a couple of weeks ago. I ran some tests, and the supplier (OWC) agreed to send a replacement drive. (I use a 2012 model MacBook Pro.)

I transferred all my non-Time Machine files onto the new drive quite easily but remain puzzled by Time Machine. I don’t want to lose my recent TM backups, as I’m still not 100% confident that the El Capitan installation went properly, due to some lingering glitches.

First attempt to copy over Backups.backupdb ran for about 5 out of 7 hours indicated, and then gave an error message, saying I lacked some necessary permission. (After installing El Capitan, I ran Onyx to fix permissions and any other issues. I do not use multiple accounts on my Mac.) Only the oldest 10 out of about 60 backup folders (those span almost 1.5 years) had copied over, and the oldest backup folder was missing a file called mach_kernel, which would not transfer, even manually; I now have that saved in a separate folder.

Then I manually copied over just my TM backups from this September (6 backup folders), and it took about a day, not too surprising, since the old drive only reads and writes at about a third of the specified speed. But that had to go in a separate folder from those first 10 that are now in Backups.backupdb.

Yesterday I was referred to the instructions at
  https://support.apple.com/en-us/HT202380
and confirmed that the new drive is formatted properly. I cannot see how to implement the instruction saying,

Make sure "Ignore ownership on this volume" at the bottom of the "Sharing & Permissions" section of the Get Info window is deselected (unchecked).
All I see in the Finder info for the new drive is the standard Sharing & Permissions chart, confirming that I have read/write permission. But everything else I did before matches those instructions.

Can anyone advise me how to proceed?


2016-10-03 at 10:27 #6525   (2)
(2016-10-02 at 22:10)brianvt wrote:  ... Can anyone advise me how to proceed?
Use Carbon Copy Cloner or similar app and clone your old Time Machine drive to the new hard disk.


2016-10-03 at 14:14 #6542   (3)
(2016-10-02 at 22:10)brianvt wrote:  The other day I received a replacement backup hard drive to replace one that went bad: The problem emerged when it took 40 hours to run a Time Machine backup after upgrading from Mt. Lion to El Capitan a couple of weeks ago.
There may have been nothing at all wrong with the hard drive.

Starting with OS X 10.9 Mavericks, Apple make a kernel change that causes it to aggressively throttle low-priority I/O that hits spinning metal hard drives (SSDs are apparently given a pass). Rumor is that this was an attempt to fix performance problems, but it has a major side effect, because Time Machine backups are run as low priority I/O.

The result is that Time Machine backups run at a snail's pace, even if no other applications are running.

You can disable throttling of low priority I/O by entering a terminal command:
sudo sysctl debug.lowpri_throttle_enabled=0
This makes a world of difference. This weekend I attached a new drive to Time Machine on a SSD-less machine running El Capitan. It was going to take it a week to backup a couple hundred gigabytes. After disabling the throttling, the backup completed in an hour or so.

Also, El Capitan tries (and fails, unfortunately) to fix the problems where Time Machine misses and drops backups of files. It appears that the first time Time Machine is run under El Capitan it does a very deep scan, to try and repair the missed backups from previous OS X's. So the deep scan combined with the I/O throttling makes the first backup run a very long time.


2016-10-04 at 12:31 #6646   (4)
(2016-10-03 at 14:14)Michael Schmitt wrote:  Starting with OS X 10.9 Mavericks, Apple make a kernel change that causes it to aggressively throttle low-priority I/O that hits spinning metal hard drives (SSDs are apparently given a pass). Rumor is that this was an attempt to fix performance problems, but it has a major side effect, because Time Machine backups are run as low priority I/O.
Now this is something I'm going to have to look into. My boot drive is an SSD, but I have an internal RAID of spinning drives for most of my documents and media. My Time Machine drive is an external FireWire drive cradle with a 3.5" spinner. I've not had problems with Time Machine being slow during normal use, but it would nearly die when I'm watching videos in VLC player. Throughput to the backup drive would go from 80+ MB/sec to maybe 2 MB/sec at best. Simply pausing the video would make it jump back up again.

It's also caused me a lot of headaches when importing photos from my camera card with a USB card reader in Capture One Pro, as it also begins generating previews while the import is happening, so that would also cause the import to slow to a crawl, as well. In that situation my catalog/library file where the previews are stored is on the SSD, but the RAW photos are stored on the spinning RAID, so during import and preview generation it's writing to and reading from that at the same time.

I looked for a while for any sort of I/O throttling information many months ago to try to explain this and was never able to find anything. How'd you find out about it?


2016-10-04 at 14:09 #6656   (5)
(2016-10-04 at 12:31)jjakucyk wrote:  I looked for a while for any sort of I/O throttling information many months ago to try to explain this and was never able to find anything. How'd you find out about it?
I googled for something like Time Machine slow, and among the top hits was this StackExchange post.

It is possible that the throttling is exacerbated by other factors. Like the original poster, I too have an older MacBook Pro, which by today's standards is no spring chicken.


2016-10-04 at 14:21 #6657   (6)
(2016-10-03 at 14:14)Michael Schmitt wrote:  Starting with OS X 10.9 Mavericks, Apple make a kernel change that causes it to aggressively throttle low-priority I/O that hits spinning metal hard drives (SSDs are apparently given a pass). Rumor is that this was an attempt to fix performance problems, but it has a major side effect, because Time Machine backups are run as low priority I/O. The result is that Time Machine backups run at a snail's pace, even if no other applications are running.
Possibly a dumb question: does this throttling affect everything? I use Carbon Copy Cloner and Chronosync to back up to various hard drives, and I haven't noticed these backups slowing to a crawl.


2016-10-04 at 17:30 #6671   (7)
For what it's worth, I haven't noticed any issues with slow Time Machine backups, on a variety of Macs,all running 10.10 (Yosemite). Until this discussion, I hadn't even heard about OS X throttling "low priority" I/O, and have never modified a setting related to it.

While I've been sticking with Yosemite on all systems I manage, including the kids' laptops, my wife decided to give El Capitan a try. I upgraded her MacBook Pro on Saturday, and started from scratch with a new Time Machine backup.
~400 GB of stuff backed up very quickly and without incident.


2016-10-05 at 08:40 #6706   (8)
(2016-10-04 at 17:30)moondigger wrote:  For what it's worth, I haven't noticed any issues with slow Time Machine backups, on a variety of Macs,all running 10.10 (Yosemite). Until this discussion, I hadn't even heard about OS X throttling "low priority" I/O, and have never modified a setting related to it.
I hadn't noticed a problem with Yosemite, but I did try to do new, full Time Machine backups with a couple of El Cap machines before upgrading to Sierra.

Time Machine was projecting 2-3 days to back up one with 500 GB of data, and about half that for one with 250 GB.

Disabling throttling via the sysctl command allowed both backups to complete in a couple of hours. Interestingly, neither machine's performance seemed to suffer significantly while the backups were being performed.

Terminal tip: if you don't use the Terminal often, a quick way to re-execute the

sudo sysctl debug.lowpri_throttle_enabled=0
command is to press the up arrow, which will cycle through previously typed commands. Simply press the up arrow until the command reappears, then hit Return. This will help to minimize the chance of mistyping the command.


2016-10-05 at 08:59 #6708   (9)
(2016-10-04 at 14:09)Michael Schmitt wrote:  I googled for something like Time Machine slow, and among the top hits was this StackExchange post.
Using tips from that post, I checked an OS X 10.9 (Mavericks) system that is not running Time Machine, using the following command, where capturing can be ended by hitting Control-C or quitting the terminal emulator:

sudo fs_usage > ~/Desktop/fs_usage-log.txt
I found the following entries scattered among many others in the capture file:

07:53:35    THROTTLED       0.025209 W launchd    
07:55:11    THROTTLED       0.025771 W mdworker    
07:55:11    THROTTLED       0.050480 W mdworker    
07:55:11    THROTTLED       0.049289 W mdworker    
07:55:11    THROTTLED       0.051006 W mdworker    
07:55:11    THROTTLED       0.023338 W mdworker    
07:55:11    THROTTLED       0.025647 W mdworker    
07:55:11    THROTTLED       0.025137 W mdworker    
07:55:11    THROTTLED       0.025541 W mdworker    
07:55:11    THROTTLED       0.039352 W mds        
07:55:11    THROTTLED       0.050695 W mdworker    
07:55:11    THROTTLED       0.025555 W mdworker    
07:55:11    THROTTLED       0.025508 W mdworker    
07:55:11    THROTTLED       0.025186 W mdworker    
07:55:11    THROTTLED       0.025420 W mdworker    
07:55:11    THROTTLED       0.025166 W mdworker    
07:55:12    THROTTLED       0.025226 W mdworker    
07:55:12    THROTTLED       0.025709 W mds        
07:55:12    THROTTLED       0.026024 W mds        
07:55:12    THROTTLED       0.025497 W mds


2016-10-05 at 12:36 #6735   (10)
(2016-10-05 at 08:40)josehill wrote:  Time Machine was projecting 2-3 days to back up one with 500 GB of data, and about half that for one with 250 GB. Disabling throttling via the sysctl command allowed both backups to complete in a couple of hours. Interestingly, neither machine's performance seemed to suffer significantly while the backups were being performed.
Ow. 2-3 days is long, but on the other hand, 500GB is a lot of data - especially when Time Machine needs to compare files as it goes to determine which have changed (and need to be copied to the backup volume) and which have not (and therefore need to be hard-linked to the prior version.)

WRT turning off throttling, I'm glad it worked for you, but I would only use it when absolutely necessary. Although interactive apps may not show much degradation, apps that require more horsepower may be slowed down, and timing-critical apps (e.g. audio/video capture) may end up with glitches if TM doesn't run at its normally-low priority.


Kurt L. Show this Post
2016-10-05 at 13:33 #6743   (11)
Kurt L.
I'm not sure if this is related, but I've been frustrated with I/O slowing down when Carbon Copy Cloner or QRecall is running with the screen locked. I have been in the habit of starting a clone or backup operation on my main user data, locking the screen for safety, and then going to bed. What I find is that the whole clone takes about three times as long with the screen locked. Once, as an experiment, I started up Activity Monitor first, and set it to display Disk activity. The graph initially showed 50 MB/sec, a fairly normal read speed while Carbon Copy Cloner examines files. Then I locked the screen and let it do its thing for a few minutes. Then I unlocked the screen, and I could see the major jump in Disk I/O speed from somewhere below 20 MB/sec back up to 50 MB/sec!

Full disclosure: my user data is on a ZFS dataset, so perhaps that affects things. However, I believe that I've seen this on my El Capitan boot disk clone operations as well. I will repeat the experiment on an HFS+ only drive.

Here's another I/O weirdness I noticed the other day: I was ripping an audio CD in iTunes, and in the middle of the rip process I switched focus from iTunes to another program that completely obscured the iTunes window. After a minute or two the CD tray stopped making sounds. Thinking that the rip was complete, I switched back to iTunes, only to find that iTunes had stopped ripping the disk altogether! As soon as I switched focus back to iTunes I heard the CD tray spin up and the rip process resumed!

I think that Apple is optimizing the operating system for laptop users, and not taking into account that there are still some of us that don't care whether the processor runs at full throttle all the time - I don't have a battery running my classic Mac Pro!


2016-10-06 at 10:19 #6776   (12)
(2016-10-05 at 12:36)David Charlap wrote:  WRT turning off throttling, I'm glad it worked for you, but I would only use it when absolutely necessary. Although interactive apps may not show much degradation, apps that require more horsepower may be slowed down, and timing-critical apps (e.g. audio/video capture) may end up with glitches if TM doesn't run at its normally-low priority.
My Time Machine just took 15 minutes to back up 751 MB of data. That's slow. But even with throttling, which I assume is on (running El Cap on a 2012 MacBook Pro Retina with a 1TB SSD), I find that Logic Pro can't run many voices and plugins during TM backups – the CPU usage gauge shows as basically full. I'll try turning off the throttling, as perhaps that way, at least, the TM will finish its business quickly. Of course, I could turn off TM backups when I run Logic – if I remembered.


2016-10-06 at 11:10 #6780   (13)
Markus
(2016-10-05 at 13:33)Kurt L. wrote:  I'm not sure if this is related, but I've been frustrated with I/O slowing down when Carbon Copy Cloner or QRecall is running with the screen locked.
...
Here's another I/O weirdness I noticed the other day: I was ripping an audio CD in iTunes, and in the middle of the rip process I switched focus from iTunes to another program that completely obscured the iTunes window. After a minute or two the CD tray stopped making sounds. Thinking that the rip was complete, I switched back to iTunes, only to find that iTunes had stopped ripping the disk altogether! As soon as I switched focus back to iTunes I heard the CD tray spin up and the rip process resumed!

I think that Apple is optimizing the operating system for laptop users, and not taking into account that there are still some of us that don't care whether the processor runs at full throttle all the time - I don't have a battery running my classic Mac Pro!
You are right, here. The OS turns off interactive programs that do not show any window (or whose windows are completely obscured) to preserve battery. An interactive program that the user cannot see does not need to run. This is called App Nap. Call up the Finder info on the respective application, and you'll see a flag called "Prevent App Nap". Tick this, and the app should continue to run in the background (maybe you have to restart the application). Mind you, not every application supports App Nap, and thus not every application does have that flag.


2016-10-06 at 11:33 #6785   (14)
(2016-10-06 at 11:10)Markus wrote:  The OS turns off interactive programs that do not show any window (or whose windows are completely obscured) to preserve battery. An interactive program that the user cannot see does not need to run. This is called App Nap.
Wow, thanks for pointing that out. Just the other day, I was running ClamXav against a 256GB SSD (in a USB3 dock) and trying to figure out how to optimize performance, since it was taking a long time. I thought that hiding its window might help, since it was redrawing the window to list the files it was checking, but that might have slowed it down instead of speeding it up!

I noticed that Onyx provides an option to turn off App Nap (in the Parameters: General tab in the Mavericks version).


2016-10-06 at 12:34 #6792   (15)
(2016-10-05 at 12:36)David Charlap wrote:  Ow. 2-3 days is long, but on the other hand, 500GB is a lot of data - especially when Time Machine needs to compare files as it goes to determine which have changed (and need to be copied to the backup volume) and which have not (and therefore need to be hard-linked to the prior version.)
Fair enough, but, as I mentioned, these were fresh, full backups, so there was no backup history that needed to be reviewed. After disabling throttling, the full backup of 500 GB completed in less than a full afternoon. (Unfortunately, I got distracted by meetings and forgot to time it.)

It's also fair to say that always running the system with throttling off may not be the best idea and may lead to unforeseen consequences. That said, I haven't run into any issues when I've forgotten to re-enable throttling (same sysctl command line as before, except ending with a "1" instead of a "0"), and throttling resets back to the default (on) upon reboot.

For what it is worth, my experience with similar functions on other flavors of Unix suggest that while fiddling with sysctl throttling may not be "recommended" as a general practice, it is generally a fairly low-risk procedure, and, as here, is easily reversible.

One other point worth a mention: the sysctl command can be issued during a Time Machine backup procedure, and the effect is immediate.


2016-10-06 at 14:47 #6807   (16)
(2016-10-06 at 11:33)Ric Ford wrote:  I noticed that Onyx provides an option to turn off App Nap (in the Parameters: General tab).
In Onyx for El Capitan, it's under the Misc. tab in Parameters.


2016-10-06 at 18:48 #6827   (17)
(2016-10-04 at 14:09)Michael Schmitt wrote:  I googled for something like Time Machine slow, and among the top hits was this StackExchange post.
It turns out that we had a note about it on MacInTouch, too:

Time Machine speed trick


2016-10-09 at 12:51 #6953   (18)
(2016-10-03 at 14:14)Michael Schmitt wrote:  You can disable throttling of low priority I/O by entering a terminal command:
sudo sysctl debug.lowpri_throttle_enabled=0
This makes a world of difference. ...
Thanks, I try this and get an error message that says:

sysctl: must specify -w to set variables
Uro


2016-10-31 at 17:35 #8232   (19)
I've realized that throttling is reenabled after restarting the computer. Is there any way to make this setting persistent?