Obscure techniques for developers to minimise macOS CPU, disk I/O and power consumption

updated Sat 11 Feb 2017

Assuming you’ve done all the obvious things, e.g.: remove unwanted apps, close apps that don’t need to be open, remove System Preference pane based apps you can do without, tidy the Login Items list, free up sufficient disk space, upgrade RAM, remove unnecessary browser extensions etc.

Exclude every website, Git repository and virtual machine on your Mac from Spotlight search. This made an incredible difference in CPU activity for me (the mdworker and mds processes are what to look for in Activity Monitor).
Reduce the quantity of files in your Dropbox folder. It can’t cope with hundreds of thousands of files, or at least the indexing process on initial login can become lengthy and CPU intensive as you near Dropbox’s own estimate of 300,000 files. Avoid having any Git repositories in there – just make sure you (a) have local backups (Time Machine only backs up certain files and directories) (b) you’re pushing to GitHub, Bitbucket or somewhere else off-site regularly. Download AWS CLI and sync a .tar.gz occasionally with an S3 bucket.
If you use PhpStorm, turn off all the Language Injections you don’t need, turn off all the Inspections you don’t need and remove any unnecessary plugins. You’ll almost certainly have a handful of candidates for each of those three areas – i.e. languages or frameworks you never use. Just familiarise yourself with what’s available every now and again so if you do work on an unusual project you have a better chance of remembering to turn the necessary options back on.

(P.S. I’m still using El Capitan 10.11.6 and have no plans to upgrade for several months.)

Mac load average

$ sysctl -n vm.loadavg
{ 1.29 1.38 1.45 }

If you think the Mac load average seems high compared to Linux, here’s an explanation of how it’s calculated. The CPU idle % is a more useful measure of how loaded your system is (the above measurement was taken with 97% idle CPU).

Why developers should turn off smart quotes

Ocassionally I write commands in my notes app before copying and pasting them into iTerm. It’s very easy to type “regular quotes” and not notice macOS has converted them into “smart quotes”.

Then your command doesn’t run, and you get an error message that doesn’t make any sense because the arguments aren’t being parsed correctly, but you don’t notice at first it’s the quotes that are wrong because the font is too small and your mind is fixated on looking for spelling mistakes and syntax problems…

To turn them off, on a Mac go to:

System Preferences > Keyboard > Text > uncheck “Use smart quotes and dashes”

Tips: Updating the Adobe Flash plugin for Firefox, Safari, Opera on Mac

I’ve routinely found the standard Adobe Flash player updater for Mac (OS X/macOS) fails right at the end, without explanation.

However there is a version that always works. Go to:

https://helpx.adobe.com/flash-player/kb/installation-problems-flash-player-mac.html

… and choose Flash player for Safari and Firefox – NPAPI – the download URL is normally:

https://fpdownload.macromedia.com/pub/flashplayer/latest/help/install_flash_player_osx.dmg

The opening dialog box should look a bit different – like this:

Adobe Flash Installer

Opera users

Use this version (last tested 10 Feb 2017):

https://fpdownload.macromedia.com/pub/flashplayer/latest/help/install_flash_player_osx_ppapi.dmg

Firefox plugin checker

This is a very fast and handy way of seeing if your copy of Flash or Silverlight is up to date.

https://www.mozilla.org/en-US/plugincheck/

macOS keyboard shortcuts

Lorna Jane started this off with a list of useful apps (Spectacle and Vimium in particular I’d recommend) and I’ve contributed some of the keyboard shortcuts I’ve picked up.

Troubleshooting Composer timeouts (IPv6 issue)

If composer hangs and you eventually see this error:

The "https://getcomposer.org/versions" file could not be downloaded: failed to open stream: Operation timed out

it could be a problem with IPv6 (for example if you’ve lost your IPv6 address for some reason.)

On MacOS you can turn IPv6 on and off via the command line. This:

networksetup -listallnetworkservices

…will give you a list of network interfaces (e.g. Ethernet, Wi-Fi, any VPNs)

networksetup -getinfo Wi-Fi

…will list all it’s settings; IP address, MAC and IPv6 status.

networksetup -setv6off Wi-Fi

…will turn IPv6 off. (To turn it back on, you’ll typically want -setv6automatic, though there’s also an option to set it manually.)

In my experience turning it off fixed the problem immediately, and composer worked straight away.

Obviously it’s preferable to fix your IPv6 and turn it back on, but this may be inconvenient if you have to reboot router etc. or beyond your control if there’s a problem with your ISP.

Useful page to test your IPv6 connection: http://ipv6test.google.com/

Recover deleted AVI files from an SDHC card when you’ve emptied the trash, then repair them

Scenario: You’ve accidentally deleted a bunch of AVI files on an SDHC memory card. You’ve also emptied the Trash in OS X, which would ordinarily mean they’re permanently lost.

Install testdisk via Homebrew
Use mount to identify the disk format (in this case a FAT-32 partition from Keedox Trail camera)
Unmount the disk, diskutil unmount /dev/<whatever> (or you can open Disk Utility and use the GUI)
sudo diskutil (if you don’t run as root, chances are you won’t see the full list of drives)
Tell it to create a log when it asks you
The drive you want should be displayed without RO to indicate it’s accessible, select it
It’ll ask for partition type – for FAT-32 use Intel
Follow the rest of this procedure (you navigate to the correct partition, directory, the deleted files are in red, you press : (colon) to select them, then C to copy, choose a destination folder, etc.

AVI files not loading

I now had AVI files which were correctly named (apart from the initial _) and timestamped, but looked about twice as big as they should be and wouldn’t play due to unrecognisable header info. They wouldn’t open in OS X Quick Look or VLC, DivFix++ kept giving me “Seek error” and trying to recover them by regenerating with ffmpeg:

ffmpeg -i input.avi -c copy output.avi

…didn’t work either.

The equivalent did work in mencoder though, and it’s very fast:

mencoder -idx input.avi -ovc copy -oac copy -o output.avi

I had a whole bunch of files (_ICT0001.AVI, _ICT0002.AVI etc.) for which you can use ‘find’ and a bash script:

find * -name '_ICT*.AVI' -type f -exec ~/avifix.sh {} \;

avifix.sh: (test it on a single file first)

#!/bin/bash
mencoder -idx $1 -ovc copy -oac copy -o repaired-$1

OS X El Capitan upgrade tips

(updated Sun 3 April 2016)

Yesterday I upgraded my primary computer (a 2012 Mac Mini) to El Capitan (OS X 10.11).

It’s normal for me to wait 6 months or so for the .3 or .4 OS X release, to allow Apple to fix hardware, networking, performance problems or random bugs and – crucially – for other developers to do the same with their applications (by no means everyone is actively testing software on the beta versions.) I’d recommend this to others.

Installation notes/tips:

Backup first (obviously.)
I recovered around 40GB of free space after installation (and 17GB on a Macbook Air upgraded soon after.)
performance is generally snappier (the CPU graph in Activity Monitor looks flatter when the system is idle, also considerable improvements when previewing files – not just PDFs but video as well)
performance will degrade considerably immediately after installation (less so on an SSD, but the Mac Mini’s HDD + Fusion Drive suffered a lot) as Spotlight reindexes everything (you’ll see sustained high disk IO and high CPU from md5 and associated processes.) If there is more than one user of the computer, this will happen the first time each user logs in, as each has a separate Spotlight database. If you use Dropbox, temporarily quitting that will help it complete faster. Keep Activity Monitor open and once indexing has finished, disk IO will return to zero.
I recommend a clean restart after that to check everything is ok.
You’ll need to upgrade the usual things, e.g. XCode, any Text to Speech voices you have installed.
Homebrew requires a change of ownership for /usr/local/ – see discussion on Stack Exchange – to the best of my knowledge chown -R is perfectly safe, but you certainly shouldn’t start messing around disabling SIP.
SuperDuper – a program that does disk backups and cloning – requires you delete and recreate any existing scheduled backups, otherwise they won’t run. More info
Expect to do one large Time Machine backup afterwards (again, this was smaller on the Macbook Air.)

Software compatibility:

I only had one program that was incompatible, a version of GPG (encryption).
If you still have Photoshop CS4, it needs the old version of Java. This is painless – on attempting to run it a Dialog Box informs you of this, the More Info button links to an Apple support page with a direct download to the file. You just install it and it works straight away.
No issues at all with PhpStorm (Jetbrains had display problems last year because of java bugs.)

Security:

Previously, Apple developed two-step authentication, with El Capitan they added two-factor authentication. The former is still supported, the latter is more secure – “It uses different methods to trust devices and deliver verification codes” – but it requires first turning 2-step off, adding security questions (note your answers are max 32 characters) and then setting up 2-factor on an iOS device (which’ll discard the security questions you just created.) Instructions (9to5mac)

Note that, given the current Apple/US government iPhone case, if you can’t get in with two-factor there is a recovery process (unlike if you lose your FileVault recovery key, say) but it’s not immediate. The KB article refers to a confirmation email to your registered account, possibly being required to confirm credit card details etc.